An introduction to testing paradigms for chef managed infrastructure

1. Automated Infrastructure Testing
Ranjib Dey
System Administrator, ThoughtWorks

2. Part 1
What we are building?

3. What you already know?

4. To Code (develop functionalities)
To Test (ensure if its working fine)

5. To Code (develop functionalities)
Automate
To Test (ensure if its working fine)

6. Enter DevOps/CD

7. To Code (develop functionalities)
To Test (ensure if its working fine)
To Deploy (ensure if it can be released)

8. To Code (develop functionalities)
To Test (ensure if its working fine)
Automate
To Deploy (ensure if it can be released)

9. To Code (develop functionalities)
Automate
To Test (ensure if its working fine)
Automate
To Deploy (ensure if it can be released)

10. To Code
Automate
CI
To Test
Automate
To Deploy

11. To Code
Automate
CI
To Test CD
Automate
To Deploy

12. Part 2
How it is different?

13. Tested code
Automate
Deployed code

14. Infrastructure
Tested code
Automate
Deployed code

15. Infrastructure
Tested code Operations
Automate
Deployed code

16. Infrastructure
Tested code Operations
Automate
Data
Deployed code

17. Infrastructure
Tested code Operations
Automate
Data
Deployed code Compliance

18. Infrastructure
Tested code Operations
Automate
Data
Deployed code Compliance
End users

19. Infrastructure
Tested code Operations
Automate
Data
Deployed code Compliance
End users
Integrations

20. On infrastructure

21. To Code
Automate
To Test
Automate
To Deploy

22. To Code
Automate
Build infrastructure
To Test
Automate
To Deploy

23. To Code
Automate
Build infrastructure
To Test
Automate
Production To Deploy
Infrastructure

24. Part 3
On testing

25. Tested code
Automate
Deployed code

26. Tested code
Automate
Infrastructure as Code
Deployed code

27. Tested code
Automate
Infrastructure as Code
Deployed code
Code

29. Tested code
Automate
Infrastructure as Code
Deployed code
Code
Test

30. Tested code
Automate
Infrastructure as Code
Deployed code
Unit Testing
Code Test

32. Tested code
Automate
Infrastructure as Code
Deployed code
Unit Testing
Code Test

33. Tested code
Automate
Infrastructure as Code
Deployed code
Unit Testing
Code Test
Functional Testing

34. Scenario: Nginx is running as a daemon
When I run "ps aux | grep nginx"
Then I should see “nginx:master" in the output

35. Tested code
Automate
Infrastructure as Code
Deployed code
Unit Testing
Code Test
Functional Testing

36. Tested code
Automate
Infrastructure as Code
Deployed code
Unit Testing
Code Test
Functional Testing
Integration Testing

37. describe_recipe 'nginx:configuration' do
it 'installs version 1.0.15' do
node[:nginx][:version].should == '1.0.15'
file("/etc/nginx/nginx.conf").must_have(:mode, "644")
package("nginx").must_be_installed
service("nginx").must_be_running
end
end

38. Tested code
Automate
Infrastructure as Code
Deployed code
Unit Testing
Code Test
Functional Testing
Monitoring
Integration Testing

39. • Monitoring

40. Tested code
Automate
Infrastructure as Code
Deployed code
Monitoring
Tests Mocks

41. Tested code
Automate
Infrastructure as Code
Deployed code
Tests Mocks
Monitoring
Analytics

42. Part 4
Cloud cometh

43. Cloud

45. I see more code

46. I see more code
I need more tests

47. Cloud
AWS

48. Cloud
AWS
Rackspace
GCE
HP
Joyent

49. Oh yeah
We love cross browser testing?

50. Part 5
Lessons learned

51. Health of infrastructure

52. Health of infrastructure
Battle tested code

53. Health of infrastructure
Battle tested code
Thorough Monitoring

54. • Remember to build feedback system .
– Test harness should publish metrics in the same
system where ops and project metrics are kept.
– Have analytics or trending system on top of the
metrics.

55. Collaborate more with
Ops.

56. Some advice
• Remember MIME types?
• World wont be fully automatic ever. Always
automate the most time consuming task first.
• No easy way out. The function domain
requires you to understand man pages, rfps
etc.
• Choose a text editor rather than an Window
manager based IDE.

57. References
• Infra-automation: http://www.opscode.com/chef/
• Monitoring: http://www.nagios.org/
• Lint tool: http://acrmp.github.com/foodcritic/
• Unit testing: https://github.com/acrmp/chefspec
• Functional testing: http://www.cucumber-chef.org/
• Integration testing:
https://github.com/calavera/minitest-chef-handler

58. @RanjibDey
http://ranjib.posterous.com
http://www.dzone.com/users/ranjibd
dey.ranjib@gmail.com