Diamond is a python daemon that collects system metrics and publishes them to Graphite (and others). It is capable of collecting cpu, memory, network, i/o, load and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source.

1. Python
Diamond
By,
Vishwanath Kulkarni
1ms12cs135
March 2016
visit www.improvepresentation.com to grab more slides

2. 1
2
3
4
5
INDEX
Introduction
Python Diamond
Fabrics
Controller and
Logging
Conclusion

3. Introduction
Diamond is a python daemon
that collects system metrics and
publishes them to Graphite
(and others). It is capable of
collecting cpu, memory,
network, i/o, load and disk
metrics. Additionally, it features
an API for implementing
custom collectors for gathering
metrics from almost any source.

4. History
• Brightcove Open Source
Projects
• The Google Analytics plugin
for Brightcove Player sends
player and video event data
to your Google Analytics
account. This allows you to
see some of your
player/video data along with
any other analytic metrics
you are collecting all in one
place.

5. History
• Brightcove,
Inc. (NASDAQ: BCOV) is
a Boston, Massachusetts–
based software company
that produces an online
video platform.
• Brightcove Smart Player
plugin to allow tracking with
Google Analytics in a
Brightcove HTML5 player.

6. History
• Then Came Python
Diamond.
• Brightcove was founded in
2004 by Jeremy Allaire.

7. Working
• Graphite Web
• Carbon Cache
• Whisper handler
• Legends

8. Coding
standards
• Genaral
• Unit tests are highly recommended
• PEP-8
• Documentation should be done docstr
style at the start of the collector.
• The use of positional arguments for
functions is strongly discouraged.

9. Coding
standards
Collectors
• Collector default configuration should be in
the get default config method of the class
• All collectors should verify before doing. For
example, if the collector requires the existence
of a file, check to verify that it exists and is
readable before opening.
• use small high-resolution units as default (such
as bits or bytes). this gives you most options
afterwards. note that graphite will show k/M/G
etc prefixes by default if needed.

10. Installation
• Installation/Building Dependencies
• make
• rpm-build
Core Dependencies
• CentOS or Ubuntu
• Python 2.4+
• python-configobj
• python-setuptools
• Python Psutil for non linux system metrics

11. Succuss
stories
• Diamond has successfully been deployed to a
cluster of 1000 machines pushing 3 million
points per minute.
• Diamond is deployed on Fabric's infrastructure,
polling hundreds of metric sources and
pushing millions of points per minute.

12. Logging
• Diamond defaults to logging
at /var/log/diamond/diamond.log. The
following is a sample configuration to have log
sent to
both /var/log/diamond/diamond.log AND
local syslog daemon:

13. Collector
Settings
• Every collector has some default options

14. Fabric
The Tools You Need to Build the Best Apps
Fabric was crafted with both power and ease-of-
use in mind.
creating different accounts, configuring the SDKs,
among others. With Fabric, you’ll never have to
worry about tedious configurations or juggling
different accounts.
It let you get right into coding and building the
next big app.

15. Fabric
The Pulse of All Great Apps

16. Fabric
The Daily users are more than 2.4thousands.
It is rapidly increasing.
9.1 thousands Monthly Active Users.
Sessions 106 thousands.

17. Fabric
Best-in-class SDKs.
Modular Kits for Building the Best Apps.
Developers can verifying users, engaging them,
generating revenue, among others.
Each presents a unique set of hurdles.

18. Fabric
Proactive Alerting.
Intelligent Alerts in Real-time.
Nothing is more important than getting timely
and actionable data.
Fabrics provide it.

19. Fabric
No need to keep revisiting dashboard to stay on
top of apps.
All get actionable crash reports, daily
performance metrics and fabrics intelligent
analyses.
Every number and metric been displayed is up-
to-date and accurate in real-time.

20. Related
Projects
• puppet-diamond - diamond module for
puppet.
• chef-diamond - diamond cookbook for chef
(Reported Broken).
• chef-diamond - Alternative diamond
cookbook for chef (Reported Broken).
• graph-explorer - graphite dashboard that
comes with templates for diamond plugins.
• graph-index - index of graphs for diamond.

21. Advantages
All get actionable crash reports, daily
performance metrics and fabrics intelligent
analyses.
Developers can verifying users, engaging them,
generating revenue, among others.
Real-time logging with 98.5% accuracy with
python diamond and its products.
Analysing all desktops and servers with just few
installations.

22. Disadvantages
It’s being deployed in only linux based
application.
Windows apps and desktops cannot be
analysed.
Has been modified by various organization
which has deplicated in its performance.
No documentation.

23. Thank you
Any Questions?