Apache Ambari BOF Meet Up @ Hadoop Summit 2013 APIs and SPIs – How to Integrate with Ambari http://www.meetup.com/Apache-Ambari-User-Group/events/119184782/

1. © Hortonworks Inc. 2013
Ambari
API
June 25, 2013
Page 1

2. © Hortonworks Inc. 2013
Agenda
• API Overview
• Monitoring
• API Constructs
• Management
Page 2

3. © Hortonworks Inc. 2013
API Overview - Features
• REST
• Monitoring and Management of Hadoop Cluster
• Partial Response
• Query Predicates
Page 3

4. © Hortonworks Inc. 2013
Monitoring
• Read state of Hadoop resources.
Example:
api/v1/clusters/cluster1/services/HDFS/components/NAMENODE
{
"href" : "http://ec2…1/services/HDFS/components/NAMENODE",
"metrics" : {
"boottime" : 1.364912931E9,
"process" : {
"proc_total" : 752.333333333,
"proc_run" : 1.64444444444
},
"rpc" : {
"rpcAuthorizationSuccesses" : 141,
"SentBytes" : 251758,
"rpcAuthorizationFailures" : 0,…
Page 4

5. © Hortonworks Inc. 2013
API Overview – Resource Types
• Collection Resource: This resource type doesn’t refer
to any specific resource; rather it refers to a collection
of resources.
For example: api/v1/clusters/cluster1/services
Returns a collection of services
• Instance Resource: This resource type refers to a
single specific resource.
For example: api/v1/clusters/cluster1/services/HDFS
Refers to the service resource identified by the id “HDFS”.
Page 5

6. © Hortonworks Inc. 2013
API Overview - Response
• JSON
• Collection resource
• Always includes request href.
• Inlcludes “items” of the collection.
• Each item includes href and primary id fields.
• Instance resource
• Always includes request href.
• Always includes primary id fields.
• May include metrics and properties for instance.
• May include href and primary id fields for sub-resources.
Page 6

7. © Hortonworks Inc. 2013
API Overview - Response
Collection Resource Response Example:
api/v1/clusters/cluster1/services/
{
"href" : "http://ec2…api/v1/clusters/cluster1/services/",
"items" : [
{
"href" : "http://ec2…/clusters/cluster1/services/HDFS",
"ServiceInfo" : {
"cluster_name" : "cluster1",
"service_name" : "HDFS"
}
},
{
"href" : "http://ec2…/cluster1/services/MAPREDUCE",…
Page 7

8. © Hortonworks Inc. 2013
API Overview - Response
Instance Resource Response Example:
api/v1/clusters/cluster1
{
"href" : "http://ec2...com:8080/api/v1/clusters/cluster1",
"Clusters" : {
"cluster_name" : "cluster1",
"cluster_id" : 1,
"version" : "HDP-1.2.1"
},
"services" : [
{
"href" :
"http://ec2...com:8080/api/v1/clusters/cluster1/services/MAPR
EDUCE",
"ServiceInfo" : {
"cluster_name" : "cluster1",
"service_name" : "MAPREDUCE"
}
}, …
Page 8

9. © Hortonworks Inc. 2013
API Constructs – Partial Response
• Used to control which fields are returned by a query.
–restrict which fields are returned.
–reach down and return data from sub-resources.
• Properties, categories and sub-resources can be
specified.
• The wildcard ‘*’ can be used to show all…
–categories, fields and sub-resources.
–provides „expand‟ functionality for sub-components.
• Primary id fields of a resource are always shown
regardless of the specifies partial response.
Page 9

10. © Hortonworks Inc. 2013
API Constructs – Partial Response
Query Specific Field Example:
api/v1/clusters/cluster1/services/MAPREDUCE/components/JOBTRACKE
R?fields=metrics/rpc/SentBytes
{
"href" :
"http://…/api/v1/clusters/cluster1/services/MAPREDUCE/components/
JOBTRACKER?fields=metrics/rpc/SentBytes",
"metrics" : {
"rpc" : {
"SentBytes" : 2763921
}
},
"ServiceComponentInfo" : {
"cluster_name" : "cluster1",
"component_name" : "JOBTRACKER",
"service_name" : "MAPREDUCE"
}
}
Page 10

11. © Hortonworks Inc. 2013
API Constructs – Partial Response
Expand Sub-Resource Example:
api/v1/clusters/cluster1/services/MAPREDUCE?fields=components/met
rics/jvm/gcCount
{
"href" :
"http://…/services/MAPREDUCE?fields=components/metrics/jvm/gc
Count",
…
},
"components" : [
{
"href" : "http://…/MAPREDUCE/components/JOBTRACKER",
"metrics" : {
"jvm" : {
"gcCount" : 47
}
},…
Page 11

12. © Hortonworks Inc. 2013
API Constructs – Query Predicates
• Limits the set of resources returned by a query.
• Consists of at least one relational expression.
• Can only be applied to collection resources.
• Relational operators ( =, !=, <, >, <=, >= )
• Logical operators ( |, &, !)
• Functions (in(), isEmpty())
• Brackets can be used to provide explicit grouping of
expressions. Expressions within brackets have the
highest precedence.
Page 12

13. © Hortonworks Inc. 2013
API Constructs – Query Predicates
Query For Started Services Example:
api/v1/clusters/cluster1/services?ServiceInfo/state=STARTED
{
"href" :
"http://…/clusters/cluster1/services?ServiceInfo/state=STARTE
D",
"items" : [
{
"href" : "http://…/clusters/cluster1/services/NAGIOS",
"ServiceInfo" : {
"cluster_name" : "cluster1",
"state" : "STARTED",
"service_name" : "NAGIOS"
}
},…
Page 13

14. © Hortonworks Inc. 2013
Management - Create
• Create a resource.
Example:
Create a cluster named „c1‟ with the property
„Clusters/version‟ = „HDP-1.2.0‟
POST http://…:8080/api/v1/clusters/c1
{
"Clusters": {
"version" : "HDP-1.2.0”
}
}
Page 14

15. © Hortonworks Inc. 2013
Management - Update
• Update a resource.
Example:
Update the state of all „INSTALLED‟ services to be
„STARTED‟
PUT
http://…:8080/api/v1/clusters/c1/services?ServiceInf
o/state=INSTALLED
{
"ServiceInfo": {
"state" : "STARTED”
}
}
Page 15

16. © Hortonworks Inc. 2013
Management - Delete
• Delete a resource.
Example:
Delete the cluster named „c1‟
DELETE http://…:8080/api/v1/clusters/c1
Page 16

17. © Hortonworks Inc. 2013:
Q & A
Page 17