1. Cluster Architecture
7/6/2012
© 2012 MapR Technologies Architecture 1

2. Architecture
Agenda
• Architectural Layers
• Storage Architecture
• Services
© 2012 MapR Technologies Architecture 2

3. Architecture
Objectives
At the end of this module you will be able to:
• Describe the architectural layers and security features of MapR
• Explain how the MapR lockless storage services work
• List the different services running in a MapR cluster
© 2012 MapR Technologies Architecture 3

4. Architectural Layers
© 2012 MapR Technologies Architecture 4

5. Architectural Layers
 MapReduce
 Lockless Storage Services
 Ecosystem & Applications
 Management
 Cluster Security
© 2012 MapR Technologies Architecture 5

6. MapReduce
 JobTracker (with HA)
 TaskTrackers
 Direct Shuffle
– RPC not HTTP
 Fully API-compliant with stock Hadoop
© 2012 MapR Technologies Architecture 6

7. Lockless Storage Services
 API-compatible with HDFS
– Java and C++ APIs
 Real-time read/write/append
 Multiple concurrent sessions
 NFS-mountable
– Supports NFS version 3 and TCP
 Lockless storage services in pure C for performance
 Direct to hardware
© 2012 MapR Technologies Architecture 7

8. Hadoop Ecosystem and Applications
 Most open Hadoop distribution
 Supports
– HBase
– Hive
– Pig
– Mahout
– Oozie
– Flume
– Informatica Hparser
– Datameer Analytics Solution
– Karmasphere Analyst and Karmasphere Studio
– MapR ODBC Connector to Hive
– and more…
© 2012 MapR Technologies Architecture 8

9. Management
 MapR Control System
 Command-Line Interface
 Rest API
 MapR Metrics
 3rd Party integrations
– Nagios
– Ganglia
© 2012 MapR Technologies Architecture 9

10. Management - MapR Control System
10 © 2012 MapR Technologies Architecture 10

11. Management - MapR Metrics
 Graphics on metrics about jobs, tasks, etc.
– Histograms display the distribution of data about jobs and task attempts
– Line charts provide historical data about jobs and task attempts
© 2012 MapR Technologies Architecture 11

12. Cluster Security
 Security throughout the MapReduce stack
– Non-Root user
• All services run as MAPR_USER
– Complies with SELinux policies
– IP address whitelisting
– Secured TaskTracker
• Disallow certain user, all superusers, etc.
– Integration with PAM
© 2012 MapR Technologies Architecture 12

13. Storage Architecture
© 2012 MapR Technologies Architecture 13

14. Lockless Storage Services
 Nodes
 Disks – operating system vs. MapR
 Storage Pools
 Single addressable distributed namespace
 CLDB
– Volumes
– Containers
– Blocks
12 TB 12 TB 12 TB
/ (Total Space = 36 TB)
© 2012 MapR Technologies Architecture 14

15. Services
© 2012 MapR Technologies Architecture 15

16. Services
 Administrative services
– ZooKeeper
– Warden
– CLDB
– JobTracker
– WebServer
– HBase Master
 Data processing services
– FileServer
– TaskTracker
– HBase RegionServer
– NFS*
© 2012 MapR Technologies Architecture 16

17. Questions
© 2012 MapR Technologies Architecture 17