This document discusses achieving continuous delivery with Puppet. It notes that currently, development cycles are long, integration is painful, and deployments are difficult. It proposes that continuous integration, continuous delivery, DevOps practices, and an agile infrastructure using automation can help address these issues. Puppet is presented as a tool that can be used to help achieve an agile infrastructure and automate application deployments, though some challenges with its use are also discussed. The document advocates for moving away from a strict separation of roles between development and operations teams toward more shared responsibilities.

1. Achieving Continuous Delivery with puppet
Laurent Bernaille, D2SI
#PuppetCampParis
@D2SI@D2SI
#PuppetCampParis

2. Past (current?) situation
Why do we need continuous delivery ?

3. Applications do not answer business needs well
Long development cycle
Difficult to get clear specifications
Market can shift quickly
June

4. Applications do not answer business needs well
Long development cycle
Difficult to get clear specifications
Market can shift quickly
June
What the end user
say he needs in January
January

5. What the dev team
delivers in June
June
Applications do not answer business needs well
Long development cycle
Difficult to get clear specifications
Market can shift quickly
What the end user
say he needs in January
January

6. What the dev team
delivers in June
June
What the end user
needs in June
Applications do not answer business needs well
Long development cycle
Difficult to get clear specifications
Market can shift quickly
June
What the end user
say he needs in January
January

7. Development processes are inefficient
Bug are detected too late
Code TESTOther work Fix

8. Development processes are inefficient
Bug are detected too late
Integration Hell
Code TESTOther work Fix
Start integration

9. Development processes are inefficient
Bug are detected too late
Integration Hell
A lot of wasted time
Code TESTOther work Fix
Request
Approve
&
Prioritize
Technical
Assessment
Code &
Test
Verify &
Fix
Deploy
20 min 2 min 15 min 2 h 4 h 3 min
½ week 2 weeks 2 weeks 1week ½ week
Processing Time = 6 h 40 min
Waiting Time = 6 weeks
Adapted from Implementing Lean Software Development: From Concept to Cash, Mary & Tom Poppendieck.
Start integration

10. OPS view
Relationship with OPS can be “difficult”
Application deployment is a nightmare

11. OPS view
Relationship with OPS can be “difficult”
Performance is not only related to hardware
« Make my website faster in Asia»
Application deployment is a nightmare

12. OPS view
?
Relationship with OPS can be “difficult”
Performance is not only related to hardware
« Make my website faster in Asia»
Application deployment is a nightmare
« Our application is too slow because of your servers»

13. OPS view
?
Relationship with OPS can be “difficult”
DEV view
Performance is not only related to hardware
« Make my website faster in Asia»
Application deployment is a nightmare
« Our application is too slow because of your servers»

14. OPS view
?
Relationship with OPS can be “difficult”
DEV view
Performance is not only related to hardware
« Make my website faster in Asia»
Application deployment is a nightmare
« Our application is too slow because of your servers»
Identical servers are always “slightly” different

15. OPS view
?
Relationship with OPS can be “difficult”
DEV view
Performance is not only related to hardware
« Make my website faster in Asia»
Application deployment is a nightmare
« Our application is too slow because of your servers»
Identical servers are always “slightly” different
OPS always say “no”
Standards do not evolve
10 deploys per day, Dev & ops cooperation at Flickr
John Allspaw & Paul Hammond (Velocity 2009)

16. 6+ months to
setup a new
environment
Infrastructure is not very agile
Server
“hoarding”

17. Infrastructure is not very agile
Server
“hoarding”
Resources
are heavily
shared
Most
environments
are
underutilized
6+ months to
setup a new
environment

18. Infrastructure is not very agile
Server
“hoarding”
Resources
are heavily
shared
Most
environments
are
underutilized
6+ months to
setup a new
environment

19. Production
Infra setup
Deploy Deploy Deploy Deploy
Utilization : 100%
Infrastructure is not very agile
Server
“hoarding”
Resources
are heavily
shared
Most
environments
are
underutilized
6+ months to
setup a new
environment

20. Production
Infra setup
Deploy Deploy Deploy Deploy
Utilization : 100%
Preproduction Utilization : 10%
Infrastructure is not very agile
Server
“hoarding”
Resources
are heavily
shared
Most
environments
are
underutilized
6+ months to
setup a new
environment

21. Production
Infra setup
Deploy Deploy Deploy Deploy
Utilization : 100%
Preproduction Utilization : 10%
Test Utilization : 40%
Infrastructure is not very agile
Server
“hoarding”
Resources
are heavily
shared
Most
environments
are
underutilized
6+ months to
setup a new
environment

22. Business
Development
Operations
• Applications do not answer business needs well
• Too long to get new features
• Integration and bug fixing is painful
• A lot of wasted time
• Deployments are very painful
• A lot of misunderstanding
• Environment setup is too slow
• No on-demand resources
Summary of the issues
WHAT?

23. IT should be easier
Continuous Delivery

24. Agile Manifesto, 2001
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Agile Development
Business
Development
Operations
A
G
I
L
E

25. Test
Discover
Design
Develop
Waterfall method
Developing Incrementally and Iteratively
Long cycles
High-risk
Adaptation is very difficult
6-12 month

26. 2-4 weeks
Test
Discover
Design
Develop
Discover
Design
Develop
Test
Discover
Design
Develop
Test
Discover
Design
Develop
Test
Waterfall method
Developing Incrementally and Iteratively
Agile method
Long cycles
High-risk
Adaptation is very difficult
6-12 month
Low-risk
Many short cycles
Adaptation is much easier
VS

27. Maintain a single source repository
Automate the build
Make your build self-testing
Every commit should build on an integration machine
Keep the build fast
Test in a clone of the production environment
Make it easy for anyone to get the latest executable
Everyone can see what’s happening
Detect problems early and solve them quickly
Continuous Integration
C
I
Business
Development
Operations

28. Continuous Integration
Discover
Design
Develop
Test
Repository
Version
Control
System
Source
Code
Continuous
Integration
Feedback
Source
Code
Build
Analyze
Code
Test
Continuous
Integration
Agile

29. DevOps : bring the wall down
Business
Development
Operations
D
E
V
O
P
S
Measure, Analyze & Describe
Constraints (from DEV and OPS)
Best practices, methods
Automation
Measure
Share
Culture Align objectives on business needs
Innovate
Automate application delivery

30. Continuous Delivery
Discover
Design
Develop
Test
Repository
Version
Control
System
Source
Code
Continuous
Integration
Feedback
Source
Code
Build
Analyze
Code
Test
Deploy
Env
Measure &
analyze
Continuous
Integration
Agile
DevOps

31. Agile
infra
Control resources
Configure
resources
Deploy
applications
Create, Delete, start, stop
Servers (physical, virtual, in the cloud)
Storage volumes, networks
Define system states, when possible
Verify system states
Reconfigure systems when necessary
Provide a service to deploy applications
Automated, with rollbacks
Agile Infrastructure
Business
Development
Operations

32. Agile
infra
Control resources
Configure
resources
Deploy
applications
Create, Delete, start, stop
Servers (physical, virtual, in the cloud)
Storage volumes, networks
Define system states, when possible
Verify system states
Reconfigure systems when necessary
Provide a service to deploy applications
Automated, with rollbacks
Automation
API
API
API
Agile Infrastructure
Business
Development
Operations

33. Continuous Delivery
Discover
Design
Develop
Test
Repository
Version
Control
System
Source
Code
Continuous
Integration
Feedback
Source
Code
Build
Analyze
Code
Test
Deploy
Env
Measure &
analyze
Continuous
Integration
Agile
DevOps
ConfigureProvision
Agile
infrastructure

34. What about puppet ?

35. Discover
Design
Develop
Test
Repository
Version
Control
System
Source
Code
Continuous
Integration
Feedback
Source
Code
Build
Analyze
Code
Test
Deploy
Env
Measure &
analyze
Continuous
Integration
Agile
DevOps
ConfigureProvision
Agile
infrastructure
Puppet use cases

36. Puppet Master
• Resolvers, time servers, standard packages
• Authentication, security
• Monitoring,…
Use case 1 : core OS configuration
Hiera
Modules
Console
ENC / API
1:Create modules
Define variables
Assign classes to nodes
Server team uses puppet as a configuration tool

37. Puppet Master
• Resolvers, time servers, standard packages
• Authentication, security
• Monitoring,…
Use case 1 : core OS configuration
Hiera
Modules
Console
ENC / API
1:Create modules
Define variables
Assign classes to nodes
4: apply/test catalog
Puppet agentPuppet agentPuppet agent
Server team uses puppet as a configuration tool

38. Puppet Master
• Resolvers, time servers, standard packages
• Authentication, security
• Monitoring,…
Use case 1 : core OS configuration
Hiera
Modules
Console
ENC / API
1:Create modules
Define variables
Assign classes to nodes
4: apply/test catalog
McollectiveGet info on nodes
Run agent on subset of nodes
Puppet agentPuppet agentPuppet agent
Server team uses puppet as a configuration tool

39. Puppet Master
• Resolvers, time servers, standard packages
• Authentication, security
• Monitoring,…
Use case 1 : core OS configuration
Hiera
Modules
Console
ENC / API
1:Create modules
Define variables
Assign classes to nodes
4: apply/test catalog
McollectiveGet info on nodes
Run agent on subset of nodes
Puppet agentPuppet agentPuppet agent
 Most common puppet
usage.
 Proven for large scale
deployment.
 Very “Infra oriented”: not
opened to applications.
Server team uses puppet as a configuration tool

40. Binaries of the application
• Puppet manifests and modules describing deployments
Repository
Binaries
Manifest
Modules
1: put
Use case 2 : deploy applications
Developers supply :

41. Binaries of the application
• Puppet manifests and modules describing deployments
Dev
servers
Test servers
Dev
serversOther env
Repository
Binaries
Manifest
Modules
1: put 2: get
4: run
Use case 2 : deploy applications
• Chooses servers (env) where the deployment should happen
• Runs puppet and gathers reports; if run fails, forwards to DEV
Developers supply :
OPS team :

42. Binaries of the application
• Puppet manifests and modules describing deployments
Dev
servers
Test servers
Dev
serversOther env
Repository
Binaries
Manifest
Modules
1: put 2: get
4: run
Use case 2 : deploy applications
• Chooses servers (env) where the deployment should happen
• Runs puppet and gathers reports; if run fails, forwards to DEV
 Much more efficient than written
deployment processes.
 Much easier to understand what
fails.
Developers supply :
OPS team :

43. Configuration as a service
ProfilsLevels
ConfigurationReport
Tools

44. Using a “configuration service”
Env
Configuration
service
Deploy
API
GUI
• Associate “profiles” to nodes, define variables
• Run configuration and get reports
Give application teams the possibility to configure servers1

45. Using a “configuration service”
Env
Configuration
service
Deploy
API
GUI
• Associate “profiles” to nodes, define variables
• Run configuration and get reports
Give application teams the possibility to configure servers1
Different levels of configuration, different responsibilities2
Base OS configuration
Standard middlewares
Application
middlewares
Application

46. Can we do that with puppet?
Sure, but tricky with classic DEV / OPS model

47. Can we do that with puppet?
Sure, but tricky with classic DEV / OPS model
DEV cannot execute anything as root
SURE, but tricky with classic DEV / OPS model

48. Base OS configuration
Standard middlewares
Application
middlewares
Application2
1
Can we do that with puppet?
Some options :
Sure, but tricky with classic DEV / OPS model
• Second puppet master, or puppet apply (non root)
• Other tool
Tool separation1
DEV cannot execute anything as root
SURE, but tricky with classic DEV / OPS model

49. Base OS configuration
Standard middlewares
Application
middlewares
Application2
1
Can we do that with puppet?
Some options :
Sure, but tricky with classic DEV / OPS model
• Second puppet master, or puppet apply (non root)
• Other tool
Tool separation1
Ok to run as root but under full control2
• Custom “profile” facts (facts.d) and hiera
• Run with mcollective (limit to some tags)
• Read-only console access
DEV cannot execute anything as root
SURE, but tricky with classic DEV / OPS model

50. Base OS configuration
Standard middlewares
Application
middlewares
Application2
1
Can we do that with puppet?
Some options :
Sure, but tricky with classic DEV / OPS model
• Second puppet master, or puppet apply (non root)
• Other tool
Tool separation1
Ok to run as root but under full control2
• Custom “profile” facts (facts.d) and hiera
• Run with mcollective (limit to some tags)
• Read-only console access
Many other ways3
DEV cannot execute anything as root
SURE, but tricky with classic DEV / OPS model

51. Approach 1 : OPS write all modules
Version
Control
System
I need a mongodb
module CI Puppet
master
mongodb
module
What if DEV need custom modules (they will)

52. Approach 1 : OPS write all modules
 Impossible to scale
 Not efficient
Version
Control
System
I need a mongodb
module CI Puppet
master
mongodb
module
What if DEV need custom modules (they will)

53. Approach 1 : OPS write all modules
 Impossible to scale
 Not efficient
Version
Control
System
I need a mongodb
module CI Puppet
master
mongodb
module
What if DEV need custom modules (they will)
Approach 2 : Pull request
mongodb
module
Version
Control
System
CI
Puppet
master
Pull request
Version
Control
System
validated
module

54. Approach 1 : OPS write all modules
 Impossible to scale
 Not efficient
Version
Control
System
I need a mongodb
module CI Puppet
master
mongodb
module
 Very limited scalabity
What if DEV need custom modules (they will)
Approach 2 : Pull request
mongodb
module
Version
Control
System
CI
Puppet
master
Pull request
Version
Control
System
validated
module

55. Approach 1 : OPS write all modules
 Impossible to scale
 Not efficient
Version
Control
System
I need a mongodb
module CI Puppet
master
mongodb
module
 Very limited scalabity
What if DEV need custom modules (they will)
Approach 2 : Pull request
Approach 3: DEV can push to some repositories
mongodb
module
Version
Control
System
CI
Puppet
master
mongodb
module
Version
Control
System
CI
Puppet
master
Pull request
Version
Control
System
validated
module

56. Approach 1 : OPS write all modules
 Impossible to scale
 Not efficient
Version
Control
System
I need a mongodb
module CI Puppet
master
mongodb
module
 Very limited scalabity
 Complex permissions
 DEV are still basically root
What if DEV need custom modules (they will)
Approach 2 : Pull request
Approach 3: DEV can push to some repositories
mongodb
module
Version
Control
System
CI
Puppet
master
mongodb
module
Version
Control
System
CI
Puppet
master
Pull request
Version
Control
System
validated
module

57. Sure, we can tweak puppet

58. Sure, we can tweak puppet
Is this this the way??

59. Sure, we can tweak puppet
Maybe we should adapt processes and not just tools
Is this this the way??

60. A NEW relationship between DEV & OPS

61. • Provide application
• Ask for env
From separation and control to shared responsiblities

62. Storage / Network
Servers
• Provide application
• Ask for env
• Provide env
• Run production
From separation and control to shared responsiblities

63. Strict separation of roles
Storage / Network
Servers
• Provide application
• Ask for env
• Provide env
• Run production
From separation and control to shared responsiblities

64. Strict separation of roles
Storage / Network
Servers
• Provide application
• Ask for env
• Provide env
• Run production
• Provide programmable resources
• Provide advice
• Delegate some Prod responsability
Storage
Servers
Servers
Network
From separation and control to shared responsiblities

65. Strict separation of roles
Storage / Network
Servers
• Provide application
• Ask for env
• Provide env
• Run production
• Provide programmable resources
• Provide advice
• Delegate some Prod responsability
Storage
Servers
Servers
Network
• Provide application
• Consume environments
• Share responsibility
From separation and control to shared responsiblities

66. Strict separation of roles
Storage / Network
Servers
• Provide application
• Ask for env
• Provide env
• Run production
• Provide programmable resources
• Provide advice
• Delegate some Prod responsability
Storage
Servers
Servers
Network
• Provide application
• Consume environments
• Share responsibility
Shared responsibilities
From separation and control to shared responsiblities

67. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
What it could look like with the profile/role pattern

68. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
ssh ntp
dns ldap
Modules
OPS provide core OS modules
What it could look like with the profile/role pattern

69. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
ssh ntp
dns ldap
Modules
mysql
apache
OPS provide core OS modules
OPS provide middleware modules
What it could look like with the profile/role pattern

70. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
ssh ntp
dns ldap
Modules
mysql
apache
OS BaseProfiles
OPS provide core OS modules
OPS provide middleware modules
OPS provide Base profile
What it could look like with the profile/role pattern

71. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
ssh ntp
dns ldap
Modules
mysql
apache
OS BaseProfiles Wordpress
OPS provide core OS modules
OPS provide middleware modules
OPS provide Base profile
DEV create profiles using modules
What it could look like with the profile/role pattern

72. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
word
press
ssh ntp
dns ldap
Modules
mysql
apache
OS BaseProfiles Wordpress
OPS provide core OS modules
OPS provide middleware modules
OPS provide Base profile
DEV create profiles using modules
DEV create some custom modules
What it could look like with the profile/role pattern

73. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
word
press
ssh ntp
dns ldap
Modules
mysql
apache
OS BaseProfiles Wordpress
Roles Roles
Wordpress-server OPS provide core OS modules
OPS provide middleware modules
OPS provide Base profile
DEV create profiles using modules
DEV create some custom modules
DEV & OPS define roles
What it could look like with the profile/role pattern

74. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
word
press
ssh ntp
dns ldap
Modules
mysql
apache
OS BaseProfiles Wordpress
Roles Roles
Wordpress-server OPS provide core OS modules
OPS provide middleware modules
OPS provide Base profile
DEV create profiles using modules
DEV create some custom modules
DEV & OPS define roles
DEV & OPS define variables
Hiera
What it could look like with the profile/role pattern

75. “Designing Puppet: Roles / Profiles Design Pattern
Puppet Camp Stockholm, Feb 2013 (Craig Dunn Puppet Labs)
Resources
word
press
ssh ntp
dns ldap
Modules
mysql
apache
OS BaseProfiles Wordpress
Roles Roles
Wordpress-server OPS provide core OS modules
OPS provide middleware modules
OPS provide Base profile
DEV create profiles using modules
DEV create some custom modules
DEV & OPS define roles
DEV & OPS associate roles to nodes
DEV & OPS define variables
Hiera
Classifier
What it could look like with the profile/role pattern

76. Final words

77. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Our feedback on puppet
Puppet is an amazing tool

78. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Our feedback on puppet
Puppet is an amazing tool

79. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
Our feedback on puppet
Puppet is an amazing tool

80. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
Our feedback on puppet
Puppet is an amazing tool

81. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
What the hell are:
* Modules (and classes)
* Hiera
* erb
* spaceships??
Our feedback on puppet
Puppet is an amazing tool

82. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
What the hell are:
* Modules (and classes)
* Hiera
* erb
* spaceships??
First modules
Our feedback on puppet
Puppet is an amazing tool

83. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
What the hell are:
* Modules (and classes)
* Hiera
* erb
* spaceships??
First modules
Wow this is big
Our feedback on puppet
Puppet is an amazing tool

84. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
What the hell are:
* Modules (and classes)
* Hiera
* erb
* spaceships??
First modules
Wow this is big
Ok not that simple
Our feedback on puppet
Puppet is an amazing tool

85. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
What the hell are:
* Modules (and classes)
* Hiera
* erb
* spaceships??
First modules
Wow this is big
Ok not that simple
Too big! We are lost
* Variables?
* Classification
* Module conflicts
Our feedback on puppet
Puppet is an amazing tool

86. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
What the hell are:
* Modules (and classes)
* Hiera
* erb
* spaceships??
First modules
Wow this is big
Ok not that simple
Too big! We are lost
* Variables?
* Classification
* Module conflicts
Best practices
* Roles / Profiles
* Variable location
Our feedback on puppet
Puppet is an amazing tool

87. • Automate configuration
• Declare state, keep configuration on track
• Puppet syntax is very expressive
• Variable management with hiera is very efficient
Time
Expectations
Puppet???
OK, looks interesting
First puppet apply
What the hell are:
* Modules (and classes)
* Hiera
* erb
* spaceships??
First modules
Wow this is big
Ok not that simple
Too big! We are lost
* Variables?
* Classification
* Module conflicts
Best practices
* Roles / Profiles
* Variable location
Our feedback on puppet
• Setups can be complex
• Many solutions to a problem
• Use it for what it does best
Try adapting processes first
• Look for best practices
Puppet is an amazing tool
You can do (almost) anything with puppet, but

88. The pace of innovation in IT is accelerating
New time-to-market challenges will require continuous delivery
We will not get continuous delivery without DEVOPS
Puppet is an amazing DEVOPS tool and will help you
Conclusion

89. The pace of innovation in IT is accelerating
New time-to-market challenges will require continuous delivery
We will not get continuous delivery without DEVOPS
Puppet is an amazing DEVOPS tool and will help you
Conclusion
But tools cannot do everything: puppet is not a magic solution

90. The pace of innovation in IT is accelerating
New time-to-market challenges will require continuous delivery
We will not get continuous delivery without DEVOPS
Puppet is an amazing DEVOPS tool and will help you
Conclusion
• Finding the best way to use puppet for you will take time
• Providing a configuration service will be a challenge
• Processes will need to change
DEV and OPS roles are evolving and Organizations will need to adapt
But tools cannot do everything: puppet is not a magic solution

91. Thank you
Laurent Bernaille, D2SI
@lbernail
@D2SI
#PuppetCampParis