Apache Hadoop Security - Ranger

© Hortonworks Inc. 2011 – 2014. All Rights Reserved
Apache Hadoop Security: Ranger
Sep 16, 2015
Madhan Neethiraj

Agenda
Control
access into
system
Flexibility
in defining
policies
• Authorization & Auditing with Ranger
• Centralized security administration for HDFS, Hive,
HBase, Knox, Strom, YARN, Kafka, Solr, ..
• Audit logs to Solr, HDFS, RDBMS, Log4j, ..
• Extensible Architecture – custom conditions, context
enrichers, easier addition of new components

Security in Hadoop
Authentication
Authenticate users and systems
Apache Knox, Native Kerberos
Authorization
Provision access to data
Apache Ranger
Audit
Maintain a record of data access
Apache Ranger, Hadoop native
audit
Data Protection
Protect data at rest and in motion
HDFS encryption + Ranger KMS,
Vendor solutions
Administration
Central management & consistent security
Apache Ranger

Authorization and Auditing with Ranger
HBase
Ranger Administration Portal
HDFS
Hive Server2
Ranger Policy
Store
Ranger Audit
Store
Ranger Plugin
Hadoop
Components
Enterpris
e Users
Log4j
Knox
Storm
RDBMS
YARN
Kafka
Solr
HDFS
Solr
Ranger Plugin
Ranger Plugin
Ranger Plugin
Ranger Plugin
Ranger Plugin
Ranger Plugin
Ranger Plugin

Central Security Administration
Apache Ranger
• Delivers a ‘single pane of glass’
for the security administrator
• Centralizes administration of
security policy
• Ensures consistent coverage
across the entire Hadoop stack

Ranger Authorization
Ranger Plugins authorize access to resources in following Hadoop components:
Component Resources Access Types
HDFS Files/Directories Read, Write, Execute
Hive Databases, Tables, Columns Create, Alter, Drop, Select, Update, All
HBase Tables, Column-Families, Columns Read, Write, Create, Admin
Knox Topologies, Services Allow
Storm Topologies Topology:
submit/activate/deactivate/reblance/kill/get/get-
info/get-user/get-conf, File: upload/download, Get
Nimbus Conf
YARN Queues Submit-application, Admin-queue
Kafka Topics Publish, Consume, Configure, Describe, Admin
Solr Collections Query, Update, Others, Admin

Ranger Auditing
• Ranger plugins generate detailed audit logs for accesses to protected resources.
Audit logs include details like: user, resource, type of access, time of access, client IP
address, access-result, ID of the policy that allowed/denied the access
• Audit logs to one or more destinations – Solr, HDFS, RDBMS, Log4j, ...
• Interactive view of audit logs using Ranger Admin

Ranger Policy - Hive
Allow Marketing group users ‘select’ access on few columns in customer_details table

Ranger Policy - HDFS
Allow Marketing group users to access /demo/data/Customer* directories and files

Ranger Policy - Kafka
Allow Marketing group users to access Marketing topic

Extensible Architecture

Extensibility: Ranger Stacks
• Customers and partners can easily add Ranger authorization and auditing
support for new components
• Describe component details (like resource structure, access-types) in JSON
and register with Ranger
• Implement component authorizer to authorize resource accesses using
Ranger policy engine
• Ranger Admin provides UI for policy administration, based on component
details in registered JSON

Extensibility: Ranger Stacks - example

Extensibility: Dynamic Policy Conditions
• Provides ability to evaluate custom conditions to drive authorization decisions
• Custom conditions can evaluate various data available in the request – like
user, groups, resource, IP-address, context, etc.
• Register custom conditions via component description JSON
• Ranger Admin provides UI to specify condition values
to be satisfied
Allow accesses
from 10.0.2.* IP
addresses only!

Extensibility: Dynamic Policy Conditions - sample
• Register the custom condition in the component description JSON:
• Implement the custom condition and make it available to Ranger plugin:
• Ranger Policy Engine will call the custom condition while evaluating policies

Extensibility: Context Enrichers
• Provides ability to add context data to access requests
• Context data added can be used by condition evaluators to drive
authorization decisions
• An example: from the client IP address in the request, a context enricher
adds location data (like COUNTRY, STATE, CITY, AREA-CODE) to the
request context. A custom condition can then restrict access depending upon
the location data in the context.
• Context enrichers should be specified in component description JSON

Extensibility: Context Enrichers - sample
• Register the context enrichers in the component description JSON:
• Implement the context enricher and make it available to Ranger plugin:
• Ranger Policy Engine will call all registered context enrichers before evaluating policies

Extensibility: Context Enricher + Condition - sample
• Implement a custom condition that verifies that the access is from specified countries only:
• Register the custom condition in the component description JSON
• On receiving authorization request, Ranger Policy Engine calls
LocationDataProviderEnricher enricher, which adds location data to the request.
• When evaluating policies, Ranger Policy Engine calls LocationCountryCondition,
which allows accesses only from the countries specified in the policy

In Development: allow/deny/exceptions in policies
• Ability to explicitly deny access to resources
• Ability to allow/deny access to a wider group, like employees/public,
but specify exceptions to a subset, like part-time employees/vendors/ip-
addresses, etc.
• Policy evaluation order:
• All deny-policies for the resource are evaluated first
• If the request matches a deny-policy, and not its deny-exceptions, access will be denied
• If the request is not denied by deny-policies, allow-policies will be evaluated
• If the request matches an allow-policy, and not its allow-exceptions, access will be allowed
• Development in tag-policy branch of Apache Ranger

Features under Development..

In Development: allow/deny/exceptions in policies
allow access to
finance group
and falcon user
deny access from
outside of Switzerland
for everyone,
except falcon user
Policy to:
- deny access from outside Switzerland to everyone, except falcon user
- allow falcon user to access from anywhere
- allow finance group users to access from Switzerland only

In Development: tag-based policies
• Ability authorize access based on tags associated with resources
• A single tag-based policy, like for PII tag, to authorize access to
resources across components – like HDFS, Hive, HBase, ..
• Available to all components that use Ranger authorization
• Similar policy structure as existing resource-based policies
• API to integrate with tag providers – like Apache Atlas
• Development in tag-policy branch of Apache Ranger

In Development: tag-based policies
Policy to authorize access to resources tagged as PII,
in HDFS/Hive/HBase/Kafka/Solr, only to audit users

Apache Ranger: how to contribute?
• Ranger Home Page - http://ranger.incubator.apache.org
• Ranger Wiki - https://cwiki.apache.org/confluence/display/RANGER
• Ranger JIRAs - https://issues.apache.org/jira/browse/RANGER
• Project Mailing Lists
• Users: user@ranger.incubator.apache.org
• Developers: dev@ranger.incubator.apache.org
• Commits: commits@ranger.incubator.apache.org

Q&A…
Discussion

Apache Hadoop Security - Ranger

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Apache Hadoop Security - Ranger

Similaire à Apache Hadoop Security - Ranger (20)

Plus de Isheeta Sanghi

Plus de Isheeta Sanghi (7)

Dernier

Dernier (20)

Apache Hadoop Security - Ranger

Notes de l'éditeur