Apache Ambari - What's New in 2.0.0

© Hortonworks Inc. 2011 – 2015. All Rights Reserved
Apache Ambari 2.0.0
April 2015
What’s New

What is Ambari?
Apache Ambari is the open source
operational platform to provision, manage
and monitor Hadoop clusters

What’s New in Ambari 2.0.0
Core Platform
Ambari Alerts (AMBARI-6354)
Ambari Metrics (AMBARI-5707)
Simplified Kerberos Setup (AMBARI-7204)
Automated Rolling Upgrade (AMBARI-7804)
HA Stack Support
Hive Metastore HA (AMBARI-6684)
HiveServer2 HA (AMBARI-8906)
Oozie HA (AMBARI-6683)
Ambari Platform
Handle umask 027 setting (AMBARI-7796)
Ambari Agent non-root (AMBARI-1596)
Blueprints API
Add Host (AMBARI-8458)
For a complete list of changes:
https://issues.apache.org/jira/browse/AMBARI/fixforversion/12327486

Ambari Alerts

Ambari Alerts
• Ambari Web provides centralized management of Health Alerts
• Alerts are configured by default based on the Services in your cluster

Viewing Host Alerts
• View all Health Alerts for each host

Modifying Alerts
• Control thresholds, check intervals and response text

Alert Notifications
• What: Create and manage
multiple notification targets
• Control who gets notified when
• Why: Filter by severity
• Send only certain notifications to certain
targets based on severity
• How: Control dispatch method
• Support for EMAIL + SNMP

Alert Groups
• Create and manage groups of
alerts
• Group alerts further controls what alerts
are dispatched which notifications
• Assign group to notifications
• Only dispatch to interested parties

Terminology
Term Definition
Alert Definition Defines the alert including the description, check interval, type and
thresholds.
Type The type of alert, such as PORT, METRIC, WEB, AGGREGATE, SCRIPT.
State Indicates the state of an alert definition (enabled or disabled). When
disabled, no alert instances are created.
Alert Instance Represents the specific alert instances based on an alert definition. For
example: the alert definition for “DataNode process” will have an alert
instance per DataNode component in the cluster.
Status The status (or “severity”) of the alert instance, such as: OK, CRIT.
Threshold The thresholds assigned to each alert status (dependent on type).
Alert Group Grouping of alert definitions, useful for handling notifications targets.
Notification A notification target for when an alert instance status changes. Methods of
notification include EMAIL and SNMP.

Alert Types
Type Description Status
Thresholds
Configurable?
PORT
Watches a port based on a configuration property as the uri.
Example: Hive Metastore Process
OK, WARN, CRIT Yes (seconds)
METRIC
Watches a metric based on a configuration property.
Example: ResourceManager RPC Latency
OK, WARN, CRIT Yes (variable)
AGGREGATE
Aggregate of status for another alert definition.
Example: percentage NodeManagers Available
OK, WARN, CRIT Yes (percentage)
WEB
Watches a Web UI and adjusts status based on response.
Example: App Timeline Web UI
OK, WARN, CRIT No
SCRIPT
Uses a custom script to handle checking.
Example: NodeManager Health Summary
OK or CRIT No

Ambari Alerting System
1. User creates or modifies cluster
2. Ambari reads alert definitions
from Stack
3. Ambari sends alert definitions to
Agents and Agent schedules
instance checks
4. Agents reports alert instance
status in the heartbeat
5. Ambari responds to alert instance
status changes and dispatches
notifications (if applicable)
Ambari
Server
1
2
4
Stack definition
alerts.json
5
Ambari
Agent(s)
3
email
snmp

Notable Alert REST APIs
REST Endpoint Description
/api/v1/clusters/:clusterName/alert_definitions The list of alert definitions for the cluster.
/api/v1/clusters/:clusterName/alerts The list of alert instances for the cluster. Use partial response syntax to filter.
Example: find all alert instances that are CRITITAL
/api/v1/clusters/c1/alerts?Alert/state.in(CRITICAL)
Example: find all alert instances for “ZooKeeper Process” alert def
/api/v1/clusters/c1/alerts?Alert/definition_name=zookeeper_server
_process
/api/v1/clusters/:clusterName/alert_groups The list of alert groups.
/api/v1/clusters/:clusterName/alert_history The list of alert instance status changes.
/api/v1/alert_targets/ The list of configured alert notification targets for Ambari.

Configuring Alerts During Cluster Install
• Ambari Alerts are configured by
default based on the Services
included in your cluster
• Setting-up email notification is
optional
• Customize Services > Misc

Ambari Alerts <-> Nagios Integration Patterns
Pattern Description Ambari<->Nagios Integration
1
Use Ambari Alerts for
definitions, instances and
instance status.
• Integrate with Ambari REST API.
• Generate Nagios cfgs based on alert information from Ambari.
• Nagios watches Ambari.
• Nagios dispatches notifications based on Ambari reported status.
2
Use Ambari Alerts for definition
and instances.
• Integrate with Ambari REST API.
• Generate Nagios cfgs based on alert information from Ambari.
• Nagios watches the cluster.
• Nagios dispatches notifications based on status determined by Nagios.
3 Do not use Ambari Alerts.
• Use Ambari REST API to obtain cluster information.
• Define and write your own alerts for Nagios cfgs.
• Nagios watches cluster.
• Nagios dispatches notifications based on status determined by Nagios.

EXAMPLE: /api/v1/clusters/MyCluster/alert_definitions/15
{
"href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/MyCluster/alert_definitions/15",
"AlertDefinition" : {
"cluster_name" : "MyCluster",
"component_name" : "ZOOKEEPER_SERVER",
"description" : "This host-level alert is triggered if the ZooKeeper server process cannot be determined",
"enabled" : true,
"id" : 15,
"ignore_host" : false,
"interval" : 1,
"label" : "ZooKeeper Server Process",
"name" : "zookeeper_server_process",
"scope" : "ANY",
"service_name" : "ZOOKEEPER",
"source" : {
"default_port" : 2181.0,
"reporting" : {
"ok" : {
"text" : "TCP OK - {0:.3f}s response on port {1}"
},
"warning" : {
"text" : "TCP OK - {0:.3f}s response on port {1}",
"value" : 1.5
},
"critical" : {
"text" : "Connection failed: {0} to {1}:{2}",
"value" : 5.0
}
},
"type" : "PORT",
"uri" : "{{zookeeper-env/clientPort}}"
}
}
}

EXAMPLE: /api/v1/clusters/MyCluster/alerts/1
{
"href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/MyCluster/alerts/1",
"Alert" : {
"cluster_name" : "MyCluster",
"component_name" : "AMBARI_AGENT",
"definition_id" : 42,
"definition_name" : "ambari_agent_disk_usage",
"host_name" : "c6401.ambari.apache.org",
"id" : 1,
"instance" : null,
"label" : "Ambari Agent Disk Usage",
"latest_timestamp" : 1425530280340,
"maintenance_state" : "OFF",
"original_timestamp" : 1425484200341,
"scope" : "HOST",
"service_name" : "AMBARI",
"state" : "OK",
"text" : "Capacity Used: [1.34%, 7.0 GB], Capacity Total: [525.3 GB]"
}
}

Pattern #1: Alert definitions, instances and status
1. Ambari reads alert definitions and
creates instances for the cluster
2. Use Ambari REST API to read
alert instance information and
generate Nagios .cfg files
3. Nagios watches Ambari REST API
for alert instance status changes
4. Ambari receives alert instance
status info from Agents
5. On status changes, Nagios
dispatches notifications
Nagios
Ambari
Server
1
2
.cfg
4
3
5
email
snmp
CLUSTER
(Optional: Not shown) Configure additional alerts in Nagios to watch
components of the system (Ambari Server, Agents, Hosts)

Pattern #2: Alert definitions in Ambari
1. Ambari reads alert definitions and
creates alert instances for cluster
alert definition information
cluster configuration information
for generating Nagios .cfg files
4. Nagios watches the cluster
Nagios
Ambari
Server
1
3
.cfg
4
5
email
snmp
CLUSTER
2

Pattern #3: No alerts in Ambari
1. Define your own alerts and use
Ambari REST API to read cluster
configuration information for
generating Nagios .cfg files for
those alerts
2. Nagios watches the cluster
Nagios
Ambari
Server
1
.cfg
2
3
email
snmp
CLUSTER

Ambari Metrics

Ambari Metrics System
Built using Hadoop technologies
Default uses local filesystem for
metrics storage (“embedded”) **
Local Filesystem **
HBase
ATS
Phoenix
** Tech Preview “distributed” storage option to use existing HDFS

Terminology
Term Definition
Ambari Metrics (“AMS”) The built-in metrics collection system for Ambari (“AMS”).
Metrics Collector The standalone server that collects metrics, aggregates metrics, serves
metrics from the Hadoop service sinks and the Metrics Monitor. Analogous
to gmetad.
Metrics Monitor Installed on each host in the cluster to collect system-level metrics and
forward to the Collector. Analogous to gmond.
Metrics Hadoop Sinks Plugs into the Service sinks to send Hadoop metrics to the Collector.

Collection System
1. Metric Monitors send system-
level metrics to Collector
2. Sinks send Hadoop-level
metrics to Collector
3. Metrics Collector service
stores and aggregates metrics
4. Ambari exposes REST API for
metrics retrieval
Ambari
Server
Metrics
Monitor
Metrics
Collector
Host1
Sink(s)
3
Metrics
Monitor
Host1
Sink(s)Metrics
Monitor
Hosts
Sink(s)
1 2
4

Simplified Kerberos Setup

Simplified Kerberos Setup
• Wizard driven and automated
Kerberos support
• Works with existing Kerberos
infrastructure, including Active
Directory
• Eliminates time consuming
administration of Kerberos

Automated Kerberos Operations
Once enabled, Ambari works directly with the Kerberos infrastructure to
automate these common tasks:
• Add/Delete Host
• Add Service
• Add/Delete Component
• Regenerate Keytabs
• Disable Kerberos

Terminology
Term Definition
Service Principals Principals required for HDP Service Components.
Ambari Principals
Headless principals used by Ambari to perform “smoke tests” and “health
alert checks”.
KDC Admin Account
An administrative account that will be used by Ambari to create principals
and generate keytabs in KDC.

Principal and Keytab Generation and Distribution
1. User provides KDC Admin
Account credentials to Ambari
2. Ambari connects to KDC, creates
principals (Service and Ambari)
needed for cluster
3. Ambari generates keytabs for the
principals
4. Ambari distributes keytabs to
Ambari Server and cluster hosts
5. Ambari discards the KDC Admin
Account credentials
Ambari
Server KDC
1 2
4
3
5
HDP
Cluster

Ambari + Service Keytab Files
Ambari
Server
HDP
Cluster
Hosts
Keytabs for
Ambari
Principals
Keytabs for
Service +
Ambari
Principals
KDC
Service Principals
Ambari Principals
Ambari and
Service
Principals

Customizable Principal Attributes
• Customize the directory principal attributes for your environment
• Particularly useful when using Active Directory for Kerberos

Manage Kerberos Clients
• Ambari installs Kerberos clients
on cluster hosts
• Optional to not have Ambari
manage krb5.conf client config
OS Client
RHEL/CentOS/OEL krb5-workstation
SLES 11 krb5-client
Ubuntu 12 krb5-user, krb5-config

Configure Ambari and Service Identities

Ambari Automated Rolling Upgrade
For HDP Stack

Rolling Upgrades
Minimized
Service Disruption &
Degradation
Minimizes service disruption by avoiding services
entirely not being available for a period of time, or
suffering performance degradation.
Reliable, Safe, Efficient
Process
Dependable, certified, check-pointed process.
Keep data, configurations, key operational state safe.

Process: Rolling Upgrade
HDP has a certified process
for Rolling Upgrades
Services are switched over to
new version in rolling fashion
ZooKeeper
Ranger
Core Masters
Core Slaves
Hive
Oozie
Falcon
Clients
Kafka
Knox
Storm
Slider
Flume
Finalize
HDFS, YARN, MR,
Tez, HBase, Pig.
Hive
HDFS
YARN
HBase

Process: Rolling Downgrade
ZooKeeper
Ranger
Core Masters
Core Slaves
Hive
Oozie
Falcon
Clients
Kafka
Knox
Storm
Slider
Flume
Downgrade
V
V
V
V
V
V
V
V
V
V
V
V
V
Finalize

Packaging: Side-by-Side Versions
/usr/hdp/current/usr/hdp/2.2.4.0-2633
/usr/hdp/2.2.0.0-2041
/hadoop-hdfs-namenode/hadoop-hdfs
/hadoop-yarn
/hive
/hadoop-hdfs
/hadoop-yarn
/hive
/hadoop-hdfs-datanode

Ambari Automated Rolling Upgrade
Register Install
Perform
Upgrade
Finalize

Register New Version
• Register new version with
Ambari
• Provide the Repository Base
URLs for new version
Register Install
Perform
Upgrade
Finalize

New Version Available Register Install
Perform
Upgrade
Finalize

Install Packages
Click “Install Packages” to start the install of the version across hosts
Register Install
Perform
Upgrade
Finalize

Perform Upgrade Register Install
Perform
Upgrade
Finalize

Perform Upgrade
Click “Perform Upgrade” to start the Rolling Upgrade Wizard
Register Install
Perform
Upgrade
Finalize

Upgrade In Process
Badge indicates upgrade “In Process” or “Action Required”
Register Install
Perform
Upgrade
Finalize

Wizard Driven Experience Register Install
Perform
Upgrade
Finalize
With
verification
and validation

Rolling Upgrade – Success! Register Install
Perform
Upgrade
Finalize

Ambari Agent Non-Root
For HDP Stack

Non-Root Ambari Agent
Agent Runs Commands From the Ambari Server
• Configuration Change
• Service Start
• Service Stop
Some Command require root level access
• /bin/su hdfs -l -s /bin/bash -c /usr/hdp/current/hadoop-client/sbin/hadoop-
daemon.sh --config /etc/hadoop/conf start datanode
Sudo Leveraged
• Configuration for:
– Customizable Users (su hdfs, yarn, etc.)
– Non-Customizable Users (su mysql)
– Commands (yum, mkdir, touch, test, etc.)
Ambari
AgentAmbari
AgentAmbari
Agent
python

Configuring Agent for Non-Root
1. Create and configure a sudoer account
2. Manually bootstrap Ambari Agents
3. Set run_as_user in ambari-agent.ini for the sudoer account

Updated umask Handling

What about umask?
Unix Permissions Basics: (user, group, other)
4 – read
2 – write
1 – execute
rwxr-xr-x == 755
Previous Behavior:
• If (umask > 022); Warning during agent pre-req check
• Installations would fail if ignored
New Behavior:
• If (umask > 027); Warning during agent pre-req check
• Installation will fail if ignored

Blueprints
Add Host

Blueprints Add Host
Add hosts to a cluster based on a
host group from a Blueprint
Add one or more hosts with a
single call
POST /api/v1/clusters/MyCluster/hosts
{
"blueprint" : "myblueprint",
"host_group" : "workers",
"host_name" : "c6403.ambari.apache.org"
}
POST /api/v1/clusters/MyCluster/hosts
[
{
},
{
}
]
https://issues.apache.org/jira/browse/AMBARI-8458

High Availability
For HDP Stack

High Availability Support
HDP Stack Mode Ambari 2.0.0
HDFS: NameNode HDP 2.0+ Active/Standby
YARN: ResourceManager HDP 2.1+ Active/Standby
HBase: HBaseMaster HDP 2.1+ Multi-master
Hive: HiveServer2 HDP 2.1+ Multi-instance
Hive: Hive Metastore HDP 2.1+ Multi-instance
Oozie: Oozie Server* HDP 2.1+ Multi-instance
* Oozie Server must not use Derby database and needs external load balancer to complete HA solution.

Hive HA
Services > Hive > Service Actions
+ Add Hive Metastore
+ Add HiveServer2

Learn More
Page 58
Resource Location
Apache Ambari Project Page http://ambari.apache.org
Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI
Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI

Apache Ambari - What's New in 2.0.0

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (13)

Similaire à Apache Ambari - What's New in 2.0.0

Similaire à Apache Ambari - What's New in 2.0.0 (20)

Plus de Hortonworks

Plus de Hortonworks (20)

Dernier

Dernier (20)

Apache Ambari - What's New in 2.0.0

Notes de l'éditeur