BlueTalon-Isilon-Validation

1© Copyright 2014 EMC Corporation. All rights reserved.© Copyright 2014 EMC Corporation. All rights reserved.
BLUETALON AUDITING AND AUTHORIZATION WITH
HDFS ON ISILON ONEFS V8.0
Boni Bruno, Chief Solutions Architect, EMC

Data-Centric Security
Secure, Fast, Flexible Hadoop Data Security Solution for Enterprises
Data
users
Authorizatio
n
Masking
Hadoop Compute
HDFSAuditing
• Manage your enterprise data in a high-
performance flexible grow-as-you-go storage
system that scales-out
Info Sec
• Analyze data at any scale or speed with your
favorite Hadoop framework
• Simplify data security
with a central policy
and audit

Hadoop
BlueTalon & Isilon provides Hadoop control & visibility at the Data Layer
• Transparent enforcement
– End users use existing apps without change
– Minimal performance overhead for security
• Contextual auditing
– Tagged with policy, role and actions
• Dynamic masking
– Selective for users without duplicating data
• Precise authorization
– Granular: file, sub-file, row, column, cell, sub-cell
– Decisions based on business data
Enforcement
PointsPolicy
Engine
Audit
Engine
Any Application
Data
Stewards
Security
admins
Auditors Data
ScientistsAnalysts
Business
users
Developers Machines

Example of policy, enforcement and audit in BlueTalon
Data users
(e.g. analysts, data scientists)
Auditors
Data
Stewards
Security
admins

Performance benchmark with BlueTalon on Isilon
6.9
125.55
7.03
124.98
Teragen
Terasort
1 TB data Job Elapsed Time (mins)
Without BlueTalon With BlueTalon
• Teragen – Measures Write I/O from Hadoop cluster to Isilon cluster.
• Terasort – Measures entire MapReduce performance across HDFS I/O
between Hadoop and Isilon, local disk I/O, CPUs usage, memory usage, etc.
Hadoop
Enforcement PointsPolicy
Engine
Audit
Engine
• 4 nodes, 100 TB, OneFS 8.0.0.1
• HDP 2.4, 7 compute nodes
• 40x7 cores, 252 GBx7 mem
• Minute performance difference with large map reduce jobs without and with BlueTalon

• BlueTalon Enforcement Points in diagram
– Filesystem EP (installed on each compute node)
• Policy and Audit in diagram
– Policy Engine and UI
– Audit Engine and UI
• Clients shown in diagram
– FsShell (hdfs)
– Hive cli (mapreduce)
This is how BlueTalon’s customers using a Hadoop cluster for compute and Isilon cluster for HDFS storage deploy BlueTalon.
Security admins create rules and view audit through UI (or API) that drive run-time Policy and Audit Engines on a
management node of the compute cluster. All file system requests from the compute cluster go through the local FSEP, which
proxies the Isilon NameNode over HDFS (not webhdfs) protocol. There is one instance of FSEP per compute node. The FS EP
proxy connects to OneFS using SmartConnect to maintain scalability and performance.
6
BlueTalon Validation in SA Lab
Isilon node 1

• Isilon storage cluster (Hopkington SA Lab)
• 4 node Isilon cluster with hdfs enabled and webhdfs not enabled.
• Webhdfs was disabled on Isilon to make sure BT only used HDFS
on the backend with Isilon. It does. This means we can use BT
to proxy both HDFS and WEBHDFS to Isilon HDFS on the
backend!
• HDP compute cluster
• 8 node HDP cluster configured with HDFS, YARN, Ambari Metrics,
etc.
• 7 compute nodes with 40 cores and 252 GB each
• Ambari UI: http://xx.solarach.lab.emc.com:8080
• BlueTalon EPs : Filesystem EP installed on each compute node
• BlueTalon Policy and Audit Engines and UIs installed on Ambari node
• Policy UI : http://xx.solarach.lab.emc.com:8111/BlueTalonConfig
• Audit UI : http://xx.solarach.lab.emc.com:8112/BlueTalonAudit
• Tests validated (see screenshots)
• FsShell –ls and –cat commands
• Teragen and Terasort mapreduce jobs with 1GB and 1TB data
• Screen on the bottom right shows write throughput on a
teragen mapreduce job running through BlueTalon EP
7
Details of the BlueTalon validation in SA Lab

Details of Validation with OneFS Simulator
• Test1. Functional validation of storage, compute and storage+compute jobs
– 3 Node Isilon OneFS 8.0 Simulator with HDFS enabled on a ESXi host
o HDFS license enabled
o FreeBSD OS
– Single Node HDP 2.3 cluster on EC2 instance
o BlueTalon Policy, Audit and Filesystem EP
o HDFS clients (fs shell and yarn)
o CentOS 6.5 OS
– Ports opened between compute cluster on EC2 and Isilon storage cluster
o Port 8020 for NameNode and port 585 for DataNode process
– Configuration on HDP cluster
o core-site.xml changed to point to FSEP or Isilon for different tests
o Filesystem EP configured in proxy authentication
– Sizing for functional testing
o Isilon VMWare Host: 8 vCPU, 32 GB mem, 500GB disk
o HDP EC2 instance: m3.xlarge = 4 vCPU, 15 GB mem, 80GB diskNote: BlueTalon Engineering runs an HDFS command test suite as part of its release exit
criteria on native HDFS clusters. We ran this checklist Jenkins job against the Isilon
cluster. All 118 tests passed successfully.

Comparison of storage queries with and without BlueTalon
core-site.xml on the
compute cluster
configures the filesystem
 Compute cluster points to
Isilon storage cluster directly
 Compute cluster points FSEP which
points to Isilon storage cluster
Without FSEP: Both alice
& bob can list in alice’s
home folder
With FSEP: bob can’t list
in alice’s home folder
Without FSEP: Both alice
& bob can read data from
a private file in alice’s
home folder
With FSEP: bob can’t
access data from a
private file in alice’s
home folder
Without FSEP: alice can’t
move files in her home
folder because filesystem is
owned by hdfs &
supergroup (required for
Hadoop functionality)
With FSEP: alice can
move data from private
location to public location
to share with bob
• Without BlueTalon • With BlueTalon

Enforcement and Policies applied on Isilon storage cluster
• global_default policy applies to all
users
• If no rule is applicable, then deny is
enforced
• Allowing recursive execute on /
enables traversing the filesystem
meta-data without exposing data.
• Allowing recursive read on
/user/<username>/public enables
users to share data with others
through their home folder
• <username>_default policy
applies to only that user
• Allowing recursive read and
write on /user/<username>
enables users to maintain
their private files in their
home folders
• Each user gets the effect of
permissions from both
global_default and their
<username>_default policies
 Compute cluster points to FSEP which points
to Isilon storage cluster
 alice can list her folder
• Enforcement of BlueTalon policies in HDFS backed by Isilon • Policies created in BlueTalon Policy UI (or automated with rules API)
 bob can’t list her folder
 bob can view
alice’s public data
 alice can make her private
data public by copying it to
public folder
 alice can view
her private data
 bob can’t view her private data

Enforcement and Audit of requests on Isilon storage cluster
 Compute cluster points to FSEP which points
to Isilon storage cluster
 alice can list her folder
• Enforcement of BlueTalon policies in HDFS backed by Isilon • Audit of the requests captured by BlueTalon
bob can’t list her folder 
 bob can view
alice’s public
data 
 alice can make her
private data public by
copying it to public folder
 alice can view
her private data
 bob can’t view her private data 

MapReduce jobs through BlueTalon FSEP on Isilon storage cluster
• Subset of the audit captured during the map reduce test in
BlueTalon UI
 mapreduce compute cluster points to FSEP which points to Isilon storage cluster
 alice doesn’t have any fs-test files
 alice is running a mapreduce job that
goes through BlueTalon FS EP 
 the file system read test run by alice completes successfully
 the file system write test run by alice completes successfully

social security is
selectively masked
data is restricted
to west coast locations
Business Users,
Data Scientists,
Developers
Security
Admins
Policies in BlueTalon
• credit cards and social security are sensitive
• our contracts prohibits use of customer data outside west coast Data
Stewards or
Business

GUI - BlueTalon HDFS Data Domain for Isilon OneFS

GUI - BlueTalon OpenLDAP User Domain for Users and Roles

Audit of HDFS with BlueTalon FS EP on Isilon OneFS
• Not only READ and WRITE, but also OPEN and GETFILESTATUS requests can be audited

Detailed audit of Hive
• user alice, beeline -e “select * from accounts” • user bluetalon, beeline -e “select * from accounts”
{
"Action": "LOGIN","AuditParams": "-",
"Client": "-","ClientIp": "10.255.54.29",
"ColumnList": "-","DataBase": "-",
"Effect": "Authorized",
"FinalQuery": "-",
"GroupName": "bedrock",
"LoggedUser": "alice",
"OrignalQuery": "-",
"PolicySet": "global_default,bedrock_default",
"Schema": "-",
"SessionID": "-",
"Timestamp": "2016-02-04|03:09:34.354",
"UniqueID": "-"
}
{
"Action": "UNKNOWN","AuditParams": "",
"Client": "","ClientIp": "10.255.54.29",
"ColumnList": "-","DataBase": "default",
"Effect": "Denied",
"FinalQuery": "select * from ARCAccessDenied",
"GroupName": "bedrock","LoggedUser": "alice",
"OrignalQuery": "select * from accounts",
"PolicySet": "global_default,bedrock_default",
"Schema": "default","SessionID": "",
"Timestamp": "2016-02-04|03:09:34.711",
"UniqueID": "711655_2027923200_1246552766"
}
{
"Action": "LOGIN","AuditParams": "-",
"Client": "-","ClientIp": "10.255.54.29",
"ColumnList": "-","DataBase": "-",
"Effect": "Authorized","FinalQuery": "-",
"GroupName": "bluetalon","LoggedUser": "bluetalon",
"OrignalQuery": "-","PolicySet": "bluetalon_default,global_default",
"Schema": "-","SessionID": "-","Timestamp": "2016-02-04|03:09:39.819",
"UniqueID": "-"
}{
"Action": "UNKNOWN","AuditParams": "",
"Client": "","ClientIp": "10.255.54.29",
"ColumnList": "ID,NAME,PHONE,BIRTHDATE,SOC_SEC_NO,ZIP,CREDIT_CARD,BALANCE",
"DataBase": "default","Effect": "Policy",
"FinalQuery": "select accounts.ID, accounts.NAME, accounts.PHONE,
accounts.BIRTHDATE, hash(accounts.SOC_SEC_NO) SOC_SEC_NO, accounts.ZIP,
0 CREDIT_CARD, accounts.BALANCE from accounts WHERE (accounts.ZIP >
/*<GCODE>WestCoastZips<GCODE>*/) ",
"GroupName": "bluetalon","LoggedUser": "bluetalon",
"OrignalQuery": "select * from accounts",
"PolicySet": "bluetalon_default,global_default",
"Schema": "default",
"SessionID": "",
"Timestamp": "2016-02-04|03:09:40.214",
"UniqueID": "214805_2027923200_1282238494"
}

Detailed audit of HDFS
{
"audit_type": "audit",
"database": "",
"group_list": ["bedrock"],
"ipaddress": "10.255.54.29",
"modified_request": ["Allow ","GETFILESTATUS ","/bedrock"],
"policy_list": [],
"policy_type": "",
"request": ["GETFILESTATUS ","/bedrock"],
"schema": "",
"time_stamp": "2016-02-04|01:16:11",
"unique_key": "0f885bb4-6635-4e56-9d6c-90c000f24f78",
"user": "alice"
}{
"database": "",
"group_list": ["bedrock"],
"ipaddress": "10.255.54.29",
"modified_request": ["Allow ","READ ","/bedrock"],
"policy_list": [],
"policy_type": "",
"request": ["READ ","/bedrock"],
"schema": "",
"time_stamp": "2016-02-04|01:16:11",
"unique_key": "64d3e4b8-bbee-4e2a-a4f4-b6da6134e045",
"user": "alice"
}
{
"database": "",
"group_list": ["users","bluetalon"],
"ipaddress": "10.255.54.29",
"modified_request": ["Allow ","GETFILESTATUS ","/bedrock"],
"policy_list": [],
"policy_type": "",
"request": ["GETFILESTATUS ","/bedrock"],
"schema": "",
"time_stamp": "2016-02-04|01:16:15",
"unique_key": "fa796aa9-1fb1-4ddc-8dfc-71dcc91981a5",
"user": "bluetalon"
}
• user alice, hdfs dfs -ls /bedrock • user bluetalon, hdfs dfs -ls /bedrock
Output from the bt-audit-kafka service

BlueTalon-Isilon-Validation

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à BlueTalon-Isilon-Validation

Similaire à BlueTalon-Isilon-Validation (20)

Plus de Boni Bruno

Plus de Boni Bruno (9)

BlueTalon-Isilon-Validation