Part of the Hortonworks YARN Ready Webinar Series, this session is about management of Apache Hadoop and YARN using Apache Ambari. This series targets developers and we will feature a demo on Ambari.

1. Apache Ambari
Using Apache Ambari to Manage Hadoop and YARN
Page 1 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Hortonworks. We do Hadoop.

2. Speakers
Pramod Thangali
Senior Director, Engineering
Tom Beerbower
Hortonworks Member Technical Staff
Committer for Apache Ambari
Yusaku Sako
Hortonworks Member Technical Staff
Committer for Apache Ambari
Page 2 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

3. Agenda
• Overview of Apache Ambari
• Managing YARN with Ambari
• Ambari Views
• Demo
• Q & A
Page 3 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

4. Agenda
Ambari
Overview
Managing
YARN
with
Ambari
Page 4 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ambari
Views
Demo Q & A

5. Apache Ambari is a
platform to provision,
manage and monitor
Hadoop clusters
Page 5 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

6. Apache Ambari Driving Themes
Operate Hadoop
at Scale
Page 6 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Integrate with
the Enterprise
Extend for the
Ecosystem

7. Enterprise Operations, Integration and Extensibility
Integration With Existing Operations Tools
Page 7 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Apache Ambari is a 100% open source
platform for provisioning, managing and
monitoring Apache Hadoop clusters
AMBARI WEB
Others
Viewpoint
compute
&
storage
. . .
. . .
. . compute
&
storage
.
EXTEND .
AMBARI REST API
AMBARI SERVER
PROVISION | MANAGE | MONITOR
OPERATE
AMBARI STACKS

8. Agenda
Ambari
Overview
Managing
YARN
with
Ambari
Page 8 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ambari
Views
Demo Q & A

9. Managing YARN with Ambari
• Deploy Hadoop and YARN including MapReduce2
and Tez data processing engines
• Control service lifecycle operations of YARN
including start, stop, restart and refresh
• Manage configurations of YARN including
configuration of Capacity Scheduler
• Monitor health of key YARN components such as
Resource Manager and Node Managers
Page 9 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

10. Slider
Set of tools to make it easy to package, deploy and manage long running
distributed services on YARN
• Avoids having to write custom application masters
• Provides built-in management capabilities via Ambari
Page 10 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

11. Slider View
• Ambari View that manages the life cycle of “Slider”ized apps
• Built using Ambari Views capability
Page 11 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
© Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION

12. Agenda
Ambari
Overview
Managing
YARN
with
Ambari
Page 12 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ambari
Views
Demo Q & A

13. Ambari Views
Goal: enable the delivery of custom UI experiences in Ambari Web
Developers can extend the Ambari Web interface
• Views expose custom UI features for Hadoop Services
Ambari Admins can entitle Views to Ambari Web users
• Entitlements framework for controlling access to Views
Page 13 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

14. Example Views
Capacity Scheduler
Queue Manager
Page 14 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Hive Tez
Query Analyzer

15. Choice of Deployment Model
• For Hadoop Operators:
Deploy Views in an Ambari Server that is managing a Hadoop cluster
• For Data Workers:
Run Views in a dedicated “standalone” Ambari Server
Ambari
Server
Page 15 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
HADOOP
Store & Process
Ambari
Views
Server
Operators
manage the
cluster, may
have Views
deployed
Data
Workers use
the cluster
and use the
Ambari
Views Server
for Views

16. Terminology
Term Description
View Definition Describes the view resources and core view properties such as name,
version and any necessary configuration properties.
View Package Packages the view client and server assets (and dependencies)
View Deployment Deploying a view into Ambari
View Version A specific version of a view
View Instance An instantiation of a specific view version
Framework Services View context, instance data, configuration properties and events
Page 16 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

17. View Packaging
View Definition: view.xml
Resource / Service classes : JAX-RS annotated
UI classes : html, Servlets deployed as web app (WEB-INF/web.xml).
Application logic : Supporting classes
Dependencies : 3rd party jars or classes
├── WEB-INF
│ └── web.xml
├── org
│ └── apache
│ └── ambari
│ └── view
│ └── weather
│ ├── CityResource.class
│ ├── CityResourceProvider.class
│ ├── CityService.class
│ └── WeatherServlet.class
└── view.xml
view.jar
Page 17 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

18. View Versions and Instances
• Deploy multiple versions and create multiple instances of a view
• Manage accessibility and usage
Page 18 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

19. View Components
• Deploy client-side assets (such as HTML + JavaScript)
• Expose server-side resources (such as REST endpoints)
Page 19 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

20. Views Server Side Resources
• Servlets
–Available at URL patterns as defined in standard web.xml
• REST Endpoints
<resource>
<name>files</name>
<service-class>org.apache.ambari.view.filebrowser.FileBrowserService</service-class>
</resource>
• Ambari Spec Managed resources which provide enhanced API services
<resource>
<name>script</name>
<plural-name>scripts</plural-name>
<id-property>id</id-property>
<resource-class>org.apache.ambari.view.pig.resources.scripts.models.PigScript</resource-class>
<provider-class>org.apache.ambari.view.pig.resources.scripts.ScriptResourceProvider</provider-class>
<service-class>org.apache.ambari.view.pig.resources.scripts.ScriptService</service-class>
</resource>
Page 20 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
© Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION

21. Agenda
Ambari
Overview
Managing
YARN
with
Ambari
Page 21 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ambari
Views
Demo Q & A

22. Agenda
Ambari
Overview
Managing
YARN
with
Ambari
Page 22 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Ambari
Views
Demo Q & A

23. Learn More About Ambari and Views
http://hortonworks.com/hadoop/ambari/
Page 23 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
• Ambari Views Framework
https://github.com/apache/ambari/tre
e/trunk/ambari-views
https://github.com/apache/ambari/bl
ob/trunk/ambari-views/
docs/index.md
• View Examples + Contribs
https://github.com/apache/ambari/tre
e/trunk/ambari-views/examples
https://github.com/apache/ambari/tre
e/trunk/contrib/views

24. Thank you!
Page 24 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

25. Next Steps
Page 25 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

26. Next Steps
1. Review Resources
2. Review past recordings
3. Attend Office Hours
4. Attend the next
webinar
Page 26 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

27. Resources
Setup HDP 2.1 environment
• Leverage Sandbox: Hortonworks.com/Sandbox
Get Started with YARN
• http://hortonworks.com/get-started/YARN
Ambari Info
• http://hortonworks.com/hadoop/ambari
Past Recordings
• http://hortonworks.com/webinars/#library
Page 27 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

28. Hortonworks Office Hours
YARN Office Hours
Dial in and chat with YARN experts
We plan Office Hours for September 11th @ 10am PT
Invitations will go out to those that attended or reviewed YARN webinars
These will also be posted to hortonworks.com/webinars
Page 28 © Hortonworks Inc. 2011 – 2014. All Rights Reserved

29. YARN Ready Webinar Schedule
Timeline
Visit: Hortonworks.com/webinars/#library
Page 29 © Hortonworks Inc. 2011 – 2014. All Rights Reserved
Native
Integration
Slider
Ambari
Sept. 4
Office Hours
Sept. 11
Scalding
Sept. 18
Spark
Oct. 2
Office Hours
Oct. 9
Upcoming Webinars
Office Hours
Recorded Webinars:
Introduction
to YARN Ready
Tez
Series: http://info.hortonworks.com/YarnReady-BigData-Webcast-
Series.html
Or sign up for an individual webinar or office hours at the same URL