Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Containing Graphite
(And Herding Cats)
Trying to Make the World at TWC a Better Place
*Drinker of Good Beer
Phil Zimmerman*
@phil_zimmerman
Graphite: Overview
Legos and Herding Cats
Show Me Some Cool Shit
Meet The Players
Registrator
A Self-Serve Interface to Your Cluster
Distributed init For Long-running Services
You Know The Company Behind Consul
(Think Vagrant)
Key/Value Store
Service Discovery (HTTP and DNS)
Failure Detection
Multi Datacenter
Queries Consul and Updates Specified
Templates on the Filesystem
Registrator
https://github.com/gliderlabs/
registrator
Auto Register/De-register Docker Containers
Graphite
Overview
Graphite
Carbon-cache
Whisper
Graphite Web
Carbon-relay
Carbon-cache
Receives and Writes Incoming Metrics
Caches and flushes data
2 Receiver ports, 1 Query Port
Whisper Database
Time-series Data
Graphite Web
Queries carbon-cache
Reads whisper data
AND
Carbon-relay
Load Balances and Proxies Metrics
Can Write To Multiple carbon-caches
For High Volume of Metrics
Carbon Relay
[:2003]
Metrics Metrics Metrics
Carbon Cache
[:2003]
[:2004]
[:7002]
Whisper Data
Carbon Webapp
[:80]
postgre...
Does Anyone See
Containers??
Master Webapp
[:80]
pgsql[:5432]
Clustered Setup
Carbon Relay
[:2003]
Metrics Metrics Metrics
Carbon Relay
[:2003]
Carbon ...
And You Thought
You Were Too Old
To Play With Legos
How Do We Orchestrate
All Of This??
Without Service Discovery
Containers
Service Announcement
Service Discovery
Configuration
What We Need
When Containers Start/Stop, They Register/
De-register Their Services
When Dependent Service Endpoints Change...
Registrator
The Service Containers
Carbon-cache: Dockerfile
The Service Containers
Carbon-relay: Dockerfile
The Service Containers
Carbon-webapp: Dockerfile
The Service Containers
postgresql: Dockerfile
The Service Containers
memcached
The Data Volume Containers
Whisper
Create a carbon-cache container without
running the cmd (use entrypoint)
The Data Volume Containers
graphite-db
Create a postgresql container without
running the cmd (use entrypoint)
The Data Volume Containers
consul-template
Create a consul-template container
without running the cmd (use
entrypoint)
Service Announcement/
Registration
Registrator
Runs in a container
Pass consul url as argument for docker run
Service Announcement/
Registration
docker run -d -P -e
SERVICE_5432_NAME=graphite-db
--volumes-from dashboard-data twcable...
Service Discovery
graphite-master-webapp
local_settings.ctmpl (consul-template)
Service Discovery
graphite-master-webapp
tmpl.json - config for consul-template
Services
Registered In
Consul
Great, Now How Do You
Launch All of These
Containers??
Registrator
Mesos Master
!
Mesos
Marathon
Zookeeper
Mesos Master
!
Mesos
Marathon
Zookeeper
Mesos Master
!
Mesos
Marathon
Zookeeper
Me...
Marathon JSON Deploy
Definition
Post JSON To Running
Marathon Instance
Show Me Something Cool
Helpful Links
http://mesos.apache.org/
https://mesosphere.github.io/marathon/
https://consul.io/
https://github.com/hashic...
Federated Graphite in Docker - Denver Docker Meetup
Federated Graphite in Docker - Denver Docker Meetup
Federated Graphite in Docker - Denver Docker Meetup
Federated Graphite in Docker - Denver Docker Meetup
Federated Graphite in Docker - Denver Docker Meetup
Federated Graphite in Docker - Denver Docker Meetup
Prochain SlideShare
Chargement dans…5
×

Federated Graphite in Docker - Denver Docker Meetup

2 278 vues

Publié le

Slides from talk I gave at Denver Docker Meetup about a Federated Graphite Cluster in Docker using Consul, Consul-template, Registrator, Mesos and Marathon.

Publié dans : Technologie
  • Soyez le premier à commenter

Federated Graphite in Docker - Denver Docker Meetup

  1. 1. Containing Graphite (And Herding Cats)
  2. 2. Trying to Make the World at TWC a Better Place *Drinker of Good Beer Phil Zimmerman* @phil_zimmerman
  3. 3. Graphite: Overview Legos and Herding Cats Show Me Some Cool Shit Meet The Players
  4. 4. Registrator
  5. 5. A Self-Serve Interface to Your Cluster Distributed init For Long-running Services
  6. 6. You Know The Company Behind Consul (Think Vagrant)
  7. 7. Key/Value Store Service Discovery (HTTP and DNS) Failure Detection Multi Datacenter
  8. 8. Queries Consul and Updates Specified Templates on the Filesystem
  9. 9. Registrator https://github.com/gliderlabs/ registrator Auto Register/De-register Docker Containers
  10. 10. Graphite Overview
  11. 11. Graphite Carbon-cache Whisper Graphite Web Carbon-relay
  12. 12. Carbon-cache Receives and Writes Incoming Metrics Caches and flushes data 2 Receiver ports, 1 Query Port
  13. 13. Whisper Database Time-series Data
  14. 14. Graphite Web Queries carbon-cache Reads whisper data AND
  15. 15. Carbon-relay Load Balances and Proxies Metrics Can Write To Multiple carbon-caches For High Volume of Metrics
  16. 16. Carbon Relay [:2003] Metrics Metrics Metrics Carbon Cache [:2003] [:2004] [:7002] Whisper Data Carbon Webapp [:80] postgresql (dashboards) [:5432] Typical Single-node Setup
  17. 17. Does Anyone See Containers??
  18. 18. Master Webapp [:80] pgsql[:5432] Clustered Setup Carbon Relay [:2003] Metrics Metrics Metrics Carbon Relay [:2003] Carbon Relay [:2003] Carbon Relay [:2003] Carbon Cache [:2003] [:2004] [:7002] Whisper Data Carbon Cache [:2003] [:2004] [:7002] Carbon Webapp [:80] MemcachedMemcachedMemcached [:11211] Carbon Relay [:2003] Carbon Cache [:2003] [:2004] [:7002] Whisper Data Carbon Cache [:2003] [:2004] [:7002] Carbon Webapp [:80]
  19. 19. And You Thought You Were Too Old To Play With Legos
  20. 20. How Do We Orchestrate All Of This??
  21. 21. Without Service Discovery
  22. 22. Containers Service Announcement Service Discovery Configuration
  23. 23. What We Need When Containers Start/Stop, They Register/ De-register Their Services When Dependent Service Endpoints Change, Config Files Auto-Update and Trigger Service Restart
  24. 24. Registrator
  25. 25. The Service Containers Carbon-cache: Dockerfile
  26. 26. The Service Containers Carbon-relay: Dockerfile
  27. 27. The Service Containers Carbon-webapp: Dockerfile
  28. 28. The Service Containers postgresql: Dockerfile
  29. 29. The Service Containers memcached
  30. 30. The Data Volume Containers Whisper Create a carbon-cache container without running the cmd (use entrypoint)
  31. 31. The Data Volume Containers graphite-db Create a postgresql container without running the cmd (use entrypoint)
  32. 32. The Data Volume Containers consul-template Create a consul-template container without running the cmd (use entrypoint)
  33. 33. Service Announcement/ Registration Registrator Runs in a container Pass consul url as argument for docker run
  34. 34. Service Announcement/ Registration docker run -d -P -e SERVICE_5432_NAME=graphite-db --volumes-from dashboard-data twcable/ postgresql93:0.2 Start & Register postgresql
  35. 35. Service Discovery graphite-master-webapp local_settings.ctmpl (consul-template)
  36. 36. Service Discovery graphite-master-webapp tmpl.json - config for consul-template
  37. 37. Services Registered In Consul
  38. 38. Great, Now How Do You Launch All of These Containers??
  39. 39. Registrator
  40. 40. Mesos Master ! Mesos Marathon Zookeeper Mesos Master ! Mesos Marathon Zookeeper Mesos Master ! Mesos Marathon Zookeeper Mesos Slave ! Mesos Docker Consul Mesos Slave ! Mesos Docker Consul Mesos Cluster
  41. 41. Marathon JSON Deploy Definition
  42. 42. Post JSON To Running Marathon Instance
  43. 43. Show Me Something Cool
  44. 44. Helpful Links http://mesos.apache.org/ https://mesosphere.github.io/marathon/ https://consul.io/ https://github.com/hashicorp/consul-template https://github.com/gliderlabs/registrator https://grey-boundary.io/the-architecture-of-clustering-graphite/ http://philzim.com/2014/11/12/service-discovery-orchestration- with-mesos-and-consul/

×