This talk gives you advices on how to successfully ship your products together with the puppet code to deploy them. It also shows you the advantages of that method and give some general advices. This talk targets products that are shipped outside of your company, in an environment you do not manage entirely.
Talk given at FLOSS UK DevOps Spring 2015.
PLAI - Acceleration Program for Generative A.I. Startups
Puppet and Software Delivery
1. Puppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software deliveryPuppet and Software delivery
Shipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet codeShipping your product with Puppet code
Julien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien Pivotto
FLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps SpringFLOSS UK DevOps Spring
March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015March 25, 2015
2. $::user$::user$::user$::user$::user$::user$::user$::user$::user$::user$::user$::user$::user$::user$::user$::user$::user
Julien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien Pivotto
• Open-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.euOpen-Source consultant at inuits.eu
• Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011Puppet User since 2011
• Speaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at PuppetcampsSpeaker/attendee at Puppetcamps
• Puppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributorPuppet core contributor
• Puppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributorPuppet(labs) modules contributor
• @roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie@roidelapluie on irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/githubon irc/twitter/github
5. Providing a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a serviceProviding a service
• YYYYYYYYYYYYYYYYYou know the environment
• YYYYYYYYYYYYYYYYYou do the maintenance
• YYYYYYYYYYYYYYYYYou do the upgrades
6. Shipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a softwareShipping a software
• WWWWWWWWWWWWWWWWWhat is the environment?
• HHHHHHHHHHHHHHHHHow was it setup?
• DDDDDDDDDDDDDDDDDid you change someting?
• WWWWWWWWWWWWWWWWWill someone maintain it? Monitor it?
7. Two different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worldsTwo different worlds
• MMMMMMMMMMMMMMMMManaging everything vs delegating
• KKKKKKKKKKKKKKKKKnown environment vs Fixed environment
• RRRRRRRRRRRRRRRRRolling updates vs Updates at will
• MMMMMMMMMMMMMMMMMonitoring vs end-users complains
10. Challenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distributionChallenges of SW distribution
• AAAAAAAAAAAAAAAAArtifacts
• CCCCCCCCCCCCCCCCConfiguration
• SSSSSSSSSSSSSSSSSecurity
• HHHHHHHHHHHHHHHHHW requirements
• SSSSSSSSSSSSSSSSSW requirements
• UUUUUUUUUUUUUUUUUpgrades, Maintenance
• MMMMMMMMMMMMMMMMMonitoring
14. DependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependenciesDependencies
• Packaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged alsoPackaged also
• In versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositoriesIn versioned repositories
• Test your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependenciesTest your dependencies
• Distribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packagesDistribution, Upstream packages
• Mirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream reposMirror and cherry pick from upstream repos
• Limit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependenciesLimit the number of dependencies
16. Cultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changesCultural changes
• NNNNNNNNNNNNNNNNNo more manual work
• UUUUUUUUUUUUUUUUUse an appropriate toolchain
• EEEEEEEEEEEEEEEEEach manual action has risks and cost
• WWWWWWWWWWWWWWWWWhat did I change 3 years ago?
17. Impact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentationImpact on documentation
• NNNNNNNNNNNNNNNNNo more complex install guides
• PPPPPPPPPPPPPPPPPuppet training needed
• VVVVVVVVVVVVVVVVValid for several products
• IIIIIIIIIIIIIIIIInspect the catalogs
18. Licensed under a Creative Commons Attribution 2.0 License
https://www.flickr.com/photos/jimmcd/4859841581
20. What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?What to automate?
Application
Reverse Proxy / Databases
Monitoring
Operation System
Platform
21. To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?To manage or not?
You have to be able to chose which
part you will setup with Puppet.
Customers have different culture or want to get control of
parts of your application.
22. Use Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modulesUse Puppet modules
Use separate meta-modules for OS,
Dependencies, Monitoring,
Application… And include them only
of needed.
23. Yes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noopYes, no or noop
• IIIIIIIIIIIIIIIIInclude or not each class
• YYYYYYYYYYYYYYYYYou can put a whole class in no-op
• include myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_osinclude myapp_os
• class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}class {'myapp_os': noop => true,}
• Usecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are metUsecase: check that preconditions are met
25. External modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modulesExternal modules
• PPPPPPPPPPPPPPPPPick the best ones
• PPPPPPPPPPPPPPPPPuppetlabs modules
• UUUUUUUUUUUUUUUUUpstream modules
• MMMMMMMMMMMMMMMMModules active on Github
• FFFFFFFFFFFFFFFFForge rating
• TTTTTTTTTTTTTTTTTesting, documentation
• MMMMMMMMMMMMMMMMModules that fit your usecase
26. Review the modules you plan to include.
You do not want bad code in your
application, why would you want it in the
code that deploys it?
27. Contribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute backContribute back
• GGGGGGGGGGGGGGGGGet feedback (peer review)
• EEEEEEEEEEEEEEEEEasier to maintain in long term
• FFFFFFFFFFFFFFFFForces you to write tests
• HHHHHHHHHHHHHHHHHelp other people
• PPPPPPPPPPPPPPPPPuppet is not your core business
28. Your own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modulesYour own modules
• EEEEEEEEEEEEEEEEEveryone has write access (Devs, Ops, …)
• FFFFFFFFFFFFFFFFFollow Puppet code standards (style guide)
• BBBBBBBBBBBBBBBBBe future-proof (follow best practices)
• SSSSSSSSSSSSSSSSSeparate code and data (hiera)
• IIIIIIIIIIIIIIIIIf not you core business: Publish it!
29. Your puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet treeYour puppet tree
• YYYYYYYYYYYYYYYYYour tree is next to your product code
• SSSSSSSSSSSSSSSSSubmodules of your main repository
• GGGGGGGGGGGGGGGGGets the same version number
• PPPPPPPPPPPPPPPPParameters matches your product
parameters
30. Distribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your treeDistribute your tree
• PPPPPPPPPPPPPPPPPackage the whole tree in a package
• UUUUUUUUUUUUUUUUUse package dependencies to pull puppet
• MMMMMMMMMMMMMMMMMaybe add a helper script for the first run
32. Data separationData separationData separationData separationData separationData separationData separationData separationData separationData separationData separationData separationData separationData separationData separationData separationData separation
Your code should be flexible enough:
Hiera code is the only thing that
should differ between your customers.
34. Going liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing liveGoing live
Licensed under a Creative Commons Attribution-ShareAlike 2.0 License
https://www.flickr.com/photos/colink/15096002421
35. Puppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterlessPuppet Agent or masterless
• PPPPPPPPPPPPPPPPPuppet has 2 modes
• AAAAAAAAAAAAAAAAAgent mode: pull your catalog
• MMMMMMMMMMMMMMMMMasterless mode: apply it from files
• BBBBBBBBBBBBBBBBBoth have advantages
• DDDDDDDDDDDDDDDDDepends on what you want
36. Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)Masterless mode (puppet apply)
• OOOOOOOOOOOOOOOOOne-time run
• RRRRRRRRRRRRRRRRRequires hiera data on every host
▶ HHHHHHHHHHHHHHHHHiera data should be consistent accross hosts
▶ NNNNNNNNNNNNNNNNNo single source of truth
• NNNNNNNNNNNNNNNNNo daemon running
• NNNNNNNNNNNNNNNNNo need for a Puppet master
• NNNNNNNNNNNNNNNNNo exported resources
• EEEEEEEEEEEEEEEEExtra work needed for PuppetDB
37. Agent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent modeAgent mode
• RRRRRRRRRRRRRRRRRun every 30 minutes (to be tuned)
• RRRRRRRRRRRRRRRRRequires one puppet master
• PPPPPPPPPPPPPPPPPuppet Agent daemon running (as root)
• RRRRRRRRRRRRRRRRReports sent to the master
• CCCCCCCCCCCCCCCCConsistency check over time
39. Deploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet treeDeploy your Puppet tree
• IIIIIIIIIIIIIIIIInstall your puppet tree package
• IIIIIIIIIIIIIIIIInstall the hiera files (versioned?)
• OOOOOOOOOOOOOOOOOne puppet apply to deploy a basic server
• TTTTTTTTTTTTTTTTThen the first agent run to deploy PuppetDB
and the rest
• TTTTTTTTTTTTTTTTThere is no puppetlabs-puppet module
• TTTTTTTTTTTTTTTTTools can help you with that (e.g kafo)
45. Shipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet codeShipping with puppet code
• EEEEEEEEEEEEEEEEEveryone gets benefits
• FFFFFFFFFFFFFFFFFrom devs to QA to customers
• NNNNNNNNNNNNNNNNNeed to review how you release
• NNNNNNNNNNNNNNNNNeed to review how you deploy your OS
• PPPPPPPPPPPPPPPPPuppet code is part of your product
47. There is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is workThere is work
• BBBBBBBBBBBBBBBBBig cultural changes
▶ RRRRRRRRRRRRRRRRRoot access?
▶ TTTTTTTTTTTTTTTTThe shell script works…
▶ IIIIIIIIIIIIIIIII can't do X anymore…
• PPPPPPPPPPPPPPPPPurge old artifacts on updates
• DDDDDDDDDDDDDDDDDeal with your data
• KKKKKKKKKKKKKKKKKeep that infra up to date
48. Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!Open the pandora box!
• MMMMMMMMMMMMMMMMMonitoring
• BBBBBBBBBBBBBBBBBest practices enforcement
• RRRRRRRRRRRRRRRRRepositories management
• BBBBBBBBBBBBBBBBBring your own tools