Build a custom metrics on aws cloud

1. Create custom
Cloudwatch metrics on
AWS cloud & New Relic
Ahmad Karawash
PhD in Technology of Information, Book Editor,
CCA, Latece, ACM & IEEE member
12/23/2015 1

2. Amazon Metrics
• Metrics are used to monitor various resources in Amazon Web
Services like EBS volumes, EC2 instances and RDS instances.
12/23/2015 2

3. AWS resources covered by
CloudWatch
 Amazon Ec2
 EBS Volumes
 AutoScaling Groups
 Elastic load Balancers
 Amazon Route 53
 RDS DB instances
 DynamoDB tables
 ElastiCache clusters
 RedShift clusters
 SQS queues
 SNS topics
 Storage Gateways
12/23/2015 3

4. Example of available metrics
• Memory Utilization – Memory allocated by applications and the operating
system, exclusive of caches and buffers, in percentages.
• Memory Used – Memory allocated by applications and the operating
system, exclusive of caches and buffers, in megabytes.
• Memory Available – System memory available for applications and the
operating system, in megabytes.
• Disk Space Utilization – Disk space usage as percentages.
• Disk Space Used – Disk space usage in gigabytes.
• Disk Space Available – Available disk space in gigabytes.
• Swap Space Utilization – Swap space usage as a percentage.
• Swap Space Used – Swap space usage in megabytes.
12/23/2015 4

5. Cloudwatch advantage
• AWS provides some additional monitoring scripts for adding custom
Cloudwatch metrics.
12/23/2015 5

6. Use case Example
• A user might have to take automated actions based on a particular
parameter. This parameter can be the number of active users in a
system which can be stored in an RDS instance. A script can be used
to measure this value and if the number of users is zero for a
particular time period, an alarm needs to be triggered and the EC2
instance needs to be terminated.
12/23/2015 6

7. Prerequisites
• Install python
• Sudo apt-get install python python dev
• Sudo apt-get install python-pip
• Install boto
• Sudo pip install boto
• Create RDS Database or local Database for the new custom metric, for
example RoomTemperatureDatabase.
12/23/2015 7

8. put_metric_data
• put_metric_data ( $namespace, $metric_data, $opt ): Publishes
metric data points to Amazon CloudWatch. Amazon Cloudwatch
associates the data points with the specified metric. If the specified
metric does not exist, Amazon CloudWatch creates the metric.
• If you create a metric with the PutMetricData action, allow up to
fifteen minutes for the metric to appear in calls to the ListMetrics
action.
12/23/2015 8

9. RoomTemperatureMeasure
Database
12/23/2015 9

10. Create a Custom Metric and
connect to external database
Run the python code from Ubuntu console: (how to run check:
https://www.youtube.com/watch?v=zKUu4in858A}
1. from boto.ec2.cloudwatch import connect_to_region
2. import MySQLdb
3. import boto.ec2.cloudwatch
4. db=MySQLdb.connect(“Public DNS","username","password","databasename")
5. cursor=db.cursor()
6. n=cursor.execute(" select value from Temperature ORDER BY id DESC LIMIT 1 ")
7. n1=cursor.fetchone()
12/23/2015 10

11. Create a Custom Metric and
connect to external database
8. s=str(n1)
9. table=string.maketrans( '', '', )
10. number=s.translate(table,"(){}<>,L")
11. numbers=int(number)
12. reg='ap-west-1'
13. conn_cw=boto.ec2.cloudwatch.connect_to_region(reg,
aws_access_key_id=’your_access_key’,aws_secret_access_key=’your_secret_k
ey’)
14. conn_cw.put_metric_data(namespace='my_namespace',name='my_metric',val
ue=numbers,dimensions={'InstanceId':‘instance-val'})
12/23/2015 11

12. Create a Custom Cloudwatch
Metric
• After you connect successively to the database, “put-metric-data”
method will add instance of a new custom metric into you
cloudwatch
12/23/2015 12

13. How to check whether the custom
metric was created
12/23/2015 13

14. How to check whether the custom
metric was created
12/23/2015 14

15. Creating a CloudWatch alarm for
the custom metric
• Now you can monitor you own metric using Cloudwatch
• For example, you can add alarm:
1. from boto.ec2.cloudwatch import connect_to_region
2. import boto.ec2.cloudwatch
3. metric=cw.list_metrics(dimensions={'InstanceId':’i-
a1b2c3d4’},metric_name='my_metric')[0]
4. metric.create_alarm(name='my_alarm', comparison='<=', threshold=0,
period=300,evaluation_periods=1, statistic='Average',
alarm_actions='arn:aws:automate:ap-northeast-1:ec2:terminate')
12/23/2015 15

16. Example for other proposed use
case
• For example:
• if you want to create a new custom metrics that measure the temperature of
the room, you can create your own database on the cloud to store the data
and update it and you have to create a database to be used by cloud watch.
• For that purpose, you can use the same code I have shown in slides 8-9, but
only apply changes to the database query.
12/23/2015 16

17. Real Implementation
• Using PHP and MySQL, I have created my own database that get
temperature value periodically from weather web service
(http://weather.yahooapis.com/forecastrss)
• Using Python, I have created a new custom metric that measure the
temperature called temperature_metric you can check it online
• I can manage this metric easily by pushing my own data to
cloudwatch and benefit from it for monitoring purpose.
• I Can add any notification system compatible with cloudwatch.
• Also, I can do my own alarm system using PHP code to forward emails
when required.
12/23/2015 17

18. New Relic Broadcast alert
12/23/2015 18
In order to Broadcast metric notifications, we can use New
Relic, because it is compatible with CloudWatch and use
various channels such as:
E-mail: Sends an email message to the email
address specified in the configuration.
HipChat: Sends a message to the HipChat room specified in
the configuration.
PagerDuty: Sends a notification through the NetOps
administration toolPagerDuty. This is an advanced option
for the more critical notifications that need to addressed
immediately—the alerts can even be configured to call you
on the phone!
Slack: Sends a notification to the Slack channel specified in
the configuration.

19. New Relic Broadcast alert
12/23/2015 19
VictorOps: Similar to PagerDuty, this option sends a
notification through the NetOps tool VictorOps. The
functionality in the two tools is quite similar, so the choice
depends mostly on what you're already using.
Webhook: Sends a notification to a URL you define. Use
this option if you want to send notifications to a channel
that isn't currently supported directly by New Relic Alerts—
or if you want to create your own custom solution...
Campfire: Sends a notification to the Campfire chat room
specified in the configuration.
OpsGenie: Sends a notification using the OpsGenie NetOps
alert system. OpsGenie is another tool similar to PagerDuty
and VictorOps that can be used to make sure your team
notices the alerts as they arise.

20. New Relic Broadcast alert
12/23/2015 20
There is also a plan to open more notification channels in
the future versions, they may add access to Lambda
functions
For full details about how to connect New Relic to
CloudWatch and start getting notification please read this
web article: http://code.tutsplus.com/tutorials/get-started-
with-monitoring-your-web-application-using-new-relic-
alerts--cms-24187

21. ?? @:
Ahmad.Karawash@gmail.com
12/23/2015 21