This document discusses service registration and discovery with Consul. It begins with an overview of service registration and discovery and how existing approaches like DNS can struggle in Docker environments. Consul is presented as a tool that stores information about services and supports service discovery. Registrator is introduced as a tool that automatically registers and deregisters Docker services with Consul. The document demonstrates how to run Consul and Registrator as Docker containers and have Registrator register container services with Consul. Finally, Consul Template is discussed as a tool to query services registered with Consul and apply configuration templates.
2. $whoami
• Michael Richardson
• Engineer @ odecee
• @m_richo
• http://www.mricho.com
• I like DevOps*
• Docker has grown on me in the last 6 months
* My definition of DevOps is likely different from yours
3. Agenda
• What is Service Registration & Discovery?
• Consul
– tool to store information about services
• Registrator
– tool to register Docker services
• Consul-template
– tool to query registered services and apply
configuration.
• Demo
5. Service Registration and Discovery
What is it?
“Service Discovery Protocols (SDP) are
network protocols which allow automatic
detection of devices and services offered by
these devices on a computer network.”
Source: http://en.wikipedia.org/wiki/Service_discovery
6. Service Registration and Discovery
What is it?
Basically… how do clients determine the
network details of desired services.
7. Service Registration and Discovery
Existing techniques?
• DNS (perhaps Dynamic DNS)
• Puppet exported resources
• Chef Search
• many others
8. Service Registration and Discovery
These approaches can struggle in a docker world
Image source: http://www.ideachampions.com/weblogs/archives/2011/09/the_top_100_lam.shtml
10. Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
11. Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
12. Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP
based DNS isn’t sufficient. *
* SRV records can be an option
13. Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP
based DNS isn’t sufficient. *
• Config Management agents
* SRV records can be an option
14. Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP
based DNS isn’t sufficient. *
• Config Management agents
– I’m resistant to running agents within a container.
– CM agent run intervals are probably not frequent enough.
* SRV records can be an option
24. Consul + Docker
How should I use consul with docker?
Option 1: run 1 Consul client agent in every
container to register its service
Or
Option 2: run 1 consul client agent per Docker
host that is responsible for all the containers on it
25. Consul + Docker
How should I use consul with docker?
Option 1: run 1 Consul client agent in every
container to register its service
Or
Option 2: run 1 consul client agent per Docker
host that is responsible for all the containers on it
Option 2 makes more sense to me
31. Registrator
• https://github.com/gliderlabs/registrator
• Automatically registers/deregisters services for docker
containers
• Supported Registry services include consul, etcd and SkyDNS
2
• Reads events from Docker Socket (/var/run/docker.sock)
• Run on each Docker host*
* Ideally run it as a container
38. How does it work
• Start containers for consul
and registrator
39. How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
40. How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
• Start new container
41. How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
• Start new container
• Registrator informs consul
of service A’s metadata
(IP/port/name/tag)
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader
Consul cluster is distributed, highly available.
Consul agent talks to 1 or more consul servers.
Data is stored and replicated between consul servers
Consul servers elect a leader