The Three Musketeers: A journey on how Docker and its friends help amaysim deliver software from Development to Production. Slides of my presentation at the Sydney Docker Meetup.

1. The Three Musketeers
By Frederic Lemay - 17 October 2017

2. “At least don’t be boring!”

3. Agenda
• Me
• Amaysim
• CI & Pipelines
• The Three Musketeers
• Demo

4. Me
• Tech lead at amaysim
• 5 years at ThoughtWorks as a Senior consultant
• First 2.5 years as Developer
• Ruby, Java, NodeJS, …
• Last 2.5 years as DevOps
• Docker ❤, AWS, Serverless, Ansible, Jenkins,
TeamCity …

5. amaysim
• Growing company
• Monolithic apps,
Microservices, Serverless
apps
• Go, NodeJS, Ruby, React,
ColdFusion
• Docker, Rancher, AWS, GoCD,
TeamCity

6. CI

7. Snap-CI
Hosted Solution

8. Snap-CI
• No need to manage you own CI infrastructure
• Easy to setup a pipeline
• Restrictive
• Specific language versions
• Limited choice of databases
• Not latest version of docker
• No Docker Compose
Hosted Solution

10. GoCD
Self-Hosted Solution

11. GoCD
• Freedom!
• But in a traditional setup:
• Agents have different
languages and versions
installed (ruby, nodejs,
golang, java)
• Might use different package/
language managers (yarn,
npm, rbenv, nvm)
• Time consuming to manage
and maintain many agents
Many cats to take care of!
Self-Hosted Solution

12. Can we do better?

13. Docker
• Runs on Mac, Windows,
and Linux
• Versatile: test, build,
deploy, run
• Encapsulates
technologies

14. GoCD with Docker
• Agents have fewer tools
installed (Docker, Docker
Compose, git, etc)
• Freedom to developers to use
their own tools within a
container
• Creating and configuring
agents is more straight
forward
Fewer cats, easier to manage!

15. Pipeline

16. BuildTest
Deploy
to Dev
…
Deploy
to Prod
Common pipeline
• Most of the steps are in bash scripts and done directly
through the UI
• Very common across organisations
• How do you test the scripts?
• Some complicated scripts are inside the project repository

17. Can we do better?

18. Makefile
• Already pre-installed in
most systems
• Pipeline steps (code) of a
project is captured and
encapsulated in a
Makefile
• Most of the Makefile
targets would run inside a
Docker container

19. make
build
make
test
make
deploy
…
make
deploy
Common pipeline
• Can be tested locally
• Can be run on my mac, on your linux computer, or your
boss windows machine!

21. Can we do better?

22. Docker Compose
• Manages containers in a neat
way
• Helps the Makefile to stay
concise therefore easier to
maintain
• Shines when a task need more
than one container
• ex: testing with a database

24. I now give you…

25. The Three Musketeers

26. Why using them?
• Consistency:
• Pipelines can be tested and run anywhere!
• Most projects follow the same pattern (test, build, deploy)
• Can move to another CI in less time!
• Control:
• Exact version and dependencies
• Pipeline code is checked in
• Confidence:
• Thorough Unit and Integration tests before committing
• Same tests will be run in CI

27. Some benefits
• Developers:
• Empower them to manage pipelines
• Get up to speed with different projects
• DevOps:
• Agents are simpler to set up and maintain
• Easier to migrate to another CI

28. The Three Musketeers at
amaysim
• Most of the pipelines in GoCD
and projects use them
• GoCD agents only run Docker
Docker Compose, git, etc
• Developers and DevOps
people know about the Three
Musketeers

29. Demo
• Test, build, and deploy a
serverless function written in
Go to AWS
• github.com/flemay/
3musketeers
• github.com/yunspace/
serverless-golang
• github.com/amaysim-au/
docker-serverless

30. thank you

31. Questions?