9. 9/49
Few numbers of BlaBlaCar today
2 Data Centers in Paris
● 242 servers
● 109 virtual servers (VMWare)
● 110 EC2 instances during working day
– 20 instances the rest of the time
11. 11/49
We have a strong agility culture
● Push our main app on production 10x a day
● Developers make the push themselves
● App can be pushed only after a Bamboo's merge
● Bamboo merge only after thousands of tests
19. 19/49
Improve our Service Deployment
● Started to work on a front web server
– Ease configuration deployment
– Ease a new installation
● Then worked on all servers
– First just for configuration
– Then all service deployment
25. 25/49
Cloud: Why on premise in 2015 ?
● Data privacy concerns
● Limited elasticity required
● Cost efficiency
● History & Culture
26. 26/49
Now – Standard Servers
● Define a single 1U server
– Dual-CPU (12 cores)
– 128 Go of RAM
– Disk configuration vary on 3 flavors:
● Only 2 SSD mirrored (RAID)
● Fast disk (SSD) needed
● Very big capacity (lot of disks)
29. 29/49
Virtualization => Template
● Work well on EC2 (only option for instance
install)
● Never used in VMWare
– Same install as physical server debian based
(thanks to Chef)
30. 30/49
Define our needs to choose a tool
● IP allocation
● Bootstrap chef-client
● PXE preseed debian
● Can install either a physical, virtual or cloud server
● Configure the BMC interface
– HP (iLo)
– Dell (DRAC)
– SuperMicro
31. 31/49
● Because Simon (@s_ourea) chose it !
– More likely to be maintained
– Big community
● Fits our needs
33. 33/49
We need more
● Chef / Foreman, great tools to:
– Install OS
– Install all services and configure them
● No easy solution when you need a specific
hardware on never used server
34. 34/49
Collins !
● Nice tool to manage our hardware inventory.
● A lot of good reasons to choose it:
– Simon (@s_ourea) chose it
– Fit our needs
● Linked with
Foreman
35. Hey ? What about the initial setup of
a physical server ?
36. 36/49
Install servers into DC
● Limited added value in the team racking servers
● Time consuming
● DCs can be far away
● Simon(@s_ourea) & Rémi(@shtouff) pushed a new idea
– Buy servers only when we can use a full rack
– Write an installation documentation
– Choose a company to rack
37. 37/49
Our procedure to have a new rack
● Buy a new rack into the DC
● Our network team installs 2 switchs in the rack.
● They add links to our core network equipments
● Hey! Let's ask to our provider to rack and setup all
servers, following Simon's documentation
38. 38/49
The typical life of a server
● Discovery, and go into Collins
● Another team takes it and installs it via Foreman
● Chef is called via Foreman
● Chef is always running
40. 40/49
Chef and tests
● Yes, we test all of our
infrastructure's services !
● Thanks to Chef and all of it's community:
KitchenCI + vagrant + serverspec, chefspec
41. 41/49
Chef - Cookbook life
● Each cookbook in it's own git repository
● First test and merge with Bamboo
● Push in production with a new version number
● All servers converge to the desired state
42. 42/49
What do you test ?
● Everything! (one day perhaps)
– Server install
– Bootstrapping
– Convergence
– Clustering /! (not automatic)
– Version upgrade ( one day !)
– Coffee quality
43. 43/49
Continuous delivery, really ?
● New cookbook versions pushed 10x times / day
● Provision several new servers / VM / instances by
day
● Buy a rack full of servers, and install them in less
than a day
44. So cool ! But...
What will you do in the future ?
46. 46/49
What we will continue to do
● Test our infra (with or without container)
● Industrialize our processes
● Spread knowledge to all other teams
● Dream solutions, and finally build them
● Listen to Simon (@s_ourea)
