Consul at databerries / teemo

Real life targeting & Offline measurement
The leading offline predictive marketing engine

2©Databerries 2017. Ce document est propriété exclusive de Databerries. Toute diffusion sans accord préalable est interdite. 2©Databerries 2017. Ce document est propriété exclusive de Databerries. Toute diffusion sans accord préalable est interdite.
Discover and deploy with Consul and Ansible

OUR BUSINESS
1

©Databerries 2017. Ce document est propriété exclusive de Databerries. Toute diffusion sans accord préalable est interdite. 4
Our solution
GEOLOCATE TARGET MEASUREDISPLAY
1 2 3 4
10+ millions
smartphones
Audiences base on
their offline in-store
behavior
Mobile ads in-app In-store foot
traffic and visits

Evolution of our service dependency system
2

©Databerries 2017. Ce document est propriété exclusive de Databerries. Toute diffusion sans accord préalable est interdite.
Data collection pipeline
Http Server
Java / Vert.x
Message Queue System
(Apache Kafka)
Streaming
server
Java
Geolocation
Events
Google BigQuery
Google Compute Engine
Mobile SDK
Control
server
(Burrow)

Generalization of our system
A D B
Both A,B and C are dependent of D.
C

Hostname hardcoded
A
Hostname of
D hardcoded
D
B
Hostname of
D hardcoded
Pro: easy to setup
Cons: if we move the service D to another machine, we need to change A/B/C source code and redeploy.
C
Hostname of
D hardcoded

Discovery at deployment time
A
Hostname of
C defined at
deployment
time
D
B
Hostname of
C defined at
deployment
time
Pro: easy to setup
Cons: if we move the service D to another machine, we need to change A/B/C source code and redeploy A,B and C.
We provision our instances using an Ansible script.
During the provisioning, Ansible will query the network to retrieve the hostname of the machine providing the D service.
C
Hostname of
C defined at
deployment
time

Service Discovery using Consul
A D B
Service Registry
Retrieve hostname of a machine providing service D Retrieve hostname of a machine providing service D
Registration
Pro:
• If we move D to another machine, A, B, C will automatically pick up the new address of D
• We can have easily multiple instances providing D
• No need to redeploy using Ansible
Cons: The service registry must not be down.
C

Basic architecture of Consul
2

Basic architecture of Consul
• Each node should run a consul agent
• Consul servers are where data is stored and replicated.
• The servers themselves elect a leader.

Details of our usage of Consul
C
Service Registry
A application
Consul agent
DNS (d.service.consul.)
Consul server
Consul agent
config.json:
“Provides service D”
D application
Machine 1 Machine 2 Machine 3
B application
Consul agent
DNS (d.service.consul.)

Fake demo
14

Consul hello world
Instance 1
Consul client
Instance 2
Provides service
“web” on port 80
Nginx
Consul client
Consul server

Setup the consul server
/etc/consul/config.json

Start consul server

Check consul cluster state

Configure consul client on instance 1
/etc/consul/config.json

Start consul client on instance 1

Configure consul client on instance 2
/etc/consul.d/web.json/etc/consul.d/config.json

Consul UI

Retrieve the IP of machine running the service “web”

Redirect port 53 to 8600 for “.consul” domains

Try to dig again using the default port

Access to the service “web”

Summary
• Great compatibility with current systems with HTTP and DNS API
• Benefits of service discovery
• Migration of machines
• Add/Remove multiple machines
• Health checks
• Autoscaling of services
• Be careful of consul downtime

We are hiring!
33
guillaume@databerries.com

Guillaume Charhon
CTO & Co founder
+33 6 41 69 69 37
guillaume@databerries.com

Consul at databerries / teemo

Recommandé

Recommandé

Contenu connexe

Similaire à Consul at databerries / teemo

Similaire à Consul at databerries / teemo (20)

Consul at databerries / teemo

Notes de l'éditeur