Talk from CodeMesh 2014. Using Chef with Test Kitchen and Docker to create testable infrastructure code in a flexible way.

1. 1

2. v1.1.0
Building Testable
Infrastructure with Chef, Test
Kitchen, and Docker
mandi walls
mandi@getchef.com
CodeMesh.io 5 November 2014
2

3. whoami
• Mandi Walls
• Professional Services at Chef
• @lnxchk
3

4. v1.1.0
What is Chef
4

5. Automation Platform
• Creates a dependable view of your entire network’s
state.
• Can handle complex dependencies among the
nodes of your network.
• Is fault tolerant.
• Is secure.
• Can handle multiple platforms
• Can manage cloud resources
• Provides a foundation for innovation
5

6. 6

7. Infrastructure As Code
• Programmatically
provision and configure
components
• Treat like any other
code base
• Reconstruct business
from code repository,
data backup, and
compute resources
7

8. Policy-Based
•You capture the policy for your infrastructure
in code
•Chef ensures each node in your
infrastructure complies with the policy
8

9. v1.1.0
Our Goals
9

10. Reliable, Repeatable Infrastructure
• Everyone on the team gets the same config
• Minimize surprises at deploy time
• Test app code against real config
• Test config with the app code
10

11. Testing Chef
• Chef is built on ruby
• Test chef code with ruby tools like rspec
• Integrate with Test Kitchen, ServerSpec, foodcritic
11

12. v1.1.0
My Tools
12

13. Components of Workflow
• My workstation - 3 year old mba
• git, github repo
• chefdk from downloads.getchef.com
• Cloud box - CentOS 6.something
• docker-io installed from peel
• chefdk from downloads.getchef.com
• kitchen-docker gem
• git
13

14. How I’m Working
• Edit recipe code on my machine
• my nice editor settings, not much RAM
• Run local tests - syntax, style, lint, chef spec
• Check into git, upload to repo
• git pull on test box
• kitchen converge, kitchen verify, kitchen test
14

15. Simple Chef Recipe
15
package "httpd"
!
service "httpd" do
action :start
end
!
file "/var/www/html/index.html" do
content "<h1>hello world</h1>n"
end

16. v1.1.0
What is Test Kitchen
16

17. Test Kitchen
• If Chef is code, we should be able to test it
17

18. Test Kitchen
•Test harness to execute code on one or more
platforms
•Driver plugins to allow your code to run on
various cloud and virtualization providers
•Includes support for many testing
frameworks
•Included with ChefDK
18

19. .kitchen.yml
•The configuration file for your Test Kitchen
• driver – virtualization or cloud provider
•provisioner – application to configure the node
•platforms – target operating systems
•suites – target configurations
19

20. .kitchen.yml
20
---
driver:
name: docker
!
provisioner:
name: chef_zero
!
platforms:
- name: centos-6.4
driver_config:
forward:
- 80:80
!
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:

21. .kitchen.yml
• Create a docker container
• With centos-6.4
• I’m testing a webapp, so forward port 80
• Provisions our environment with an apache
webserver as configured by our team in a Chef
recipe
21

22. kitchen create
-----> Creating <default-centos-64>...!
build context to Docker daemon 2.048 kB!
Sending build context to Docker daemon!
Step 0 : FROM centos:centos6!
Pulling repository centos!
---> 70441cac1ed5!
Step 1 : RUN yum clean all!
---> Running in b4ed54c86150!
Loaded plugins: fastestmirror!
Cleaning repos: base extras updates!
Cleaning up Everything!
---> 1c333241ae96!
Removing intermediate container b4ed54c86150!
Step 2 : RUN yum install -y sudo openssh-server openssh-clients which curl
22

23. docker ps
[chef@CentOS63 codemesh]$ sudo docker ps!
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES!
9ddd3300de51 87f714782104 "/usr/sbin/sshd -D - 2 minutes ago
Up 2 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:49157->22/tcp stoic_euclid
23

24. kitchen converge
- install version 2.2.15-39.el6.centos of package httpd!
!
* service[httpd] action start [2014-11-05T00:18:29+00:00] INFO: Processing
service[httpd] action start (codemesh::default line 16)!
[2014-11-05T00:18:29+00:00] INFO: service[httpd] started!
!
- start service service[httpd]!
!
* service[httpd] action enable [2014-11-05T00:18:29+00:00] INFO: Processing service[httpd]
action enable (codemesh::default line 16)!
[2014-11-05T00:18:29+00:00] INFO: service[httpd] enabled!
!
- enable service service[httpd]!
!
* file[/var/www/html/index.html] action create [2014-11-05T00:18:29+00:00] INFO:
Processing file[/var/www/html/index.html] action create (codemesh::default line 20)!
[2014-11-05T00:18:29+00:00] INFO: file[/var/www/html/index.html] created file /var/www/
html/index.html!
24

25. kitchen list
[chef@CentOS63 codemesh]$ kitchen list!
Instance Driver Provisioner Last Action!
default-centos-64 Docker ChefZero Converged!
25

26. show me
[chef@CentOS63 codemesh]$ curl localhost!
<h1>hello world</h1>!
[chef@CentOS63 codemesh]$
26

27. Other Plugins
• kitchen-vagrant (included!)!
• kitchen-docker!
• kitchen-ec2!
• kitchen-rackspace!
• kitchen-gce!
• kitchen-digitalocean!
• kitchen-openstack!
• kitchen-bluebox!
• kitchen-joyent!
• kitchen-lxc
27

28. Reliable Configuration
• Provision onto platforms that match production
• Write Chef recipes once for everyone
• Local development work
• Testing systems
• Production
• Customize environments as necessary
• ports, user accounts, backend services
28

29. v1.1.0
TDI: Test-Driven
Infrastructure
29

30. Testing Bits for Chef
• Pre-testing (no running host)
• rubocop: ruby style and syntax
• ChefSpec: chef-specific unit testing
• foodcritic: chef-specific logic and style rules
• Post-testing (on a running host)
• ServerSpec: integration testing on services
• bats: unix system testing
30

31. rubocop
$ rubocop recipes/default.rb !
Inspecting 1 file!
C!
Offenses:!
recipes/default.rb:17:11: C: Space inside square brackets detected.!
action [ :start, :enable ]!
^!
recipes/default.rb:17:27: C: Space inside square brackets detected.!
action [ :start, :enable ]!
^!
recipes/default.rb:20:6: C: Prefer single-quoted strings when you don't need
string interpolation or special symbols.!
file "/var/www/html/index.html" do!
^^^^^^^^^^^^^^^^^^^^^^^^^^!
1 file inspected, 3 offenses detected
31

32. fix
$ rubocop recipes/default.rb !
Inspecting 1 file!
.!
!
1 file inspected, no offenses detected
32

33. ChefSpec
• Test the intention of your code
• Built on rspec
• Specific to Chef
33

34. Methods
• You can TDD infrastructure code
• Keep on top of regressions
• Give safe configurations to the whole team
34

35. chefspec file
require 'chefspec'!
!
describe 'codemesh::default' do!
let(:chef_run) do!
ChefSpec::Runner.new.converge(described_recipe)!
end !
!
it 'installs apache' do!
expect(chef_run).to install_package('httpd')!
end !
end!
35

36. Run chefspec
$ rspec spec/unit/default.rb !
.!
!
Finished in 0.00735 seconds (files
took 1.85 seconds to load)!
1 example, 0 failures!
36

37. ServerSpec
• Tests running systems
• Run from Test Kitchen at the end of kitchen
converge
• Not dependent on Chef!
• Use from other config tools
• Use standalone!
• serverspec.org
37

38. Serverspec File
require 'serverspec'!
!
set :backend, :exec!
!
describe 'apache' do!
it "is installed" do!
expect(package 'httpd').to
be_installed!
end !
it "is running" do!
expect(service 'httpd').to be_running!
end !
it "is listening on port 80" do!
expect(port 80).to be_listening!
end !
38
it "displays a custom home page"
do!
expect(command("curl
localhost").stdout).to match /
hello/!
end !
end!

39. kitchen verify
apache!
is installed!
is running!
is listening on port 80!
displays a custom home page!
!
Finished in 1.45 seconds (files took 0.6278 seconds to
load)!
4 examples, 0 failures!
Finished verifying <default-centos-64> (0m18.23s).!
-----> Kitchen is finished. (0m39.50s)!
39

40. test vs verify
• kitchen verify will run tests on an instance that is
already running
• kitchen test will start a new instance, run all the
chef recipes, run all the tests, and then destroy the
instance
• nice for different workflow - user watching a test vs a
build server watching for a good return code
• fast with Docker!
40

41. v1.1.0
Further Resources
41

42. Container Ecosystem
• Chef container resources help you manage
container workflow
• Create and deploy images
• https://docs.getchef.com/containers.html
42

43. Testing Tools
• ChefSpec: https://docs.getchef.com/chefspec.html
• Serverspec: http://serverspec.org/
• bats: https://github.com/sstephenson/bats
• Foodcritic:
• More on ChefDK:
• http://foodfightshow.org/2014/08/chefdk.html
• More on this workflow:
• http://www.slideshare.net/misheska/testing-yourautomationcode-
docker-version-v02
43

44. Some URLs
• http://getchef.com
• http://docs.getchef.com
• http://supermarket.getchef.com
• http://youtube.com/getchef
• http://lists.opscode.com
• irc.freenode.net: #chef, #chef-hacking
• Twitter: @chef #getchef, @learnchef #learnchef
•Meetups in your city?!?
44

45. v1.1.0
Questions?
45

46. 46