As Puppet pushes into it’s second decade of reign, there are several organisations out there that have been using Puppet for a long time. Sometimes, even since the beginning!
With the EOL announcement Puppet 3.X release, we’ve had a number of customers approach us to help with their upgrade. Normally the upgrade itself is fairly, it’s the code base that gives the biggest challenge, especially those with over 3 years of organic growth.
So let’s spread the word of common anti-patterns and issues that can come back to bite you
We’ll be talking about how Hiera is both the best and worst thing to happen to Puppet, marvel at how people were happily running 0.2 Puppet in production and what hacky solutions that seemed good at the time will come back to bite you!
By the end of this, you’ll hopefully have learnt how to make sure that your Puppet code is defensively coded to to make sure your Puppet code base is healthy for the next decade!
Knee deep in the undef - Tales from refactoring old Puppet codebases
1. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Knee deep in the undef:
Tales from refactoring old
Puppet codebases
Peter Souter
Senior Professional Services Engineer | Puppet
@petersouter
2. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
2
Who
am I?
@petersouter
Senior Professional
Services Engineer
5 years using Puppet
2 years @ Puppet Inc
Help customers deploy
Puppet
Teach Puppet classes
Contribute to the
community and
open-source
petems
IRC/Slack/GitHub
3. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Warning: I speak quickly
And I have a different accent...
3
4. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
“Regardless of what we discover, we understand and truly
believe that everyone did the best job they could, given
what they knew at the time, their skills and abilities, the
resources available, and the situation at hand.”
- http://www.retrospectives.com/pages/retroPrimeDirective.html
4
5. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Show of hands in the room
What version of Puppet are you on now?
5
6. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Show of hands in the room
How old is your Puppet codebase? When was your first
commit?
6
7. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
7
The King is Dead… Long live the king!
Puppet 3.X is now EOL
8. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Some customers we hadn’t talked to in a
while...
This is a good time for an upgrade or refactor of code
8
9. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Noticed a few repeating anti-patterns
and issues with a lot of these customers
And similar issues with people asking for community help
in Slack and IRC
9
10. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
10
commit bbdecbeb1c199b7132f94c33ee44a66e265fa456
Author: Jane Doe <jane.doe@example.com>
Date: Tue Jun 4 10:46:13 2013 +0100
Initial commit
11. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
11
commit ac4b7c3312d75c3584ef069c272f7ad4ff610eb4
Author: John Doe <john.doe@example.com>
Date: Thu Mar 3 15:46:59 2011 +0000
Import
git-svn-id: https://svn.example.com/svn/puppet/trunk@1
3d7401f0-959d-0410-a61e-fbe730d1da08
12. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
I’m not going to talk about the Puppet 4
upgrade stuff
Lots of talks on that… but it’s a common time for people to
look over their codebases when upgrading
12
13. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
13
1. Hiera overload/Too much data
2. Lack of validation/CI
3. Reinventing the wheel
4. Lack of VCS best practises
5. Make the newbie experience better
5 Key Areas that came up over again
14. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Hiera overload/Too much data
Hiera is simultaneously the best and worst thing to happen
to Puppet
14
15. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
YAML management is the biggest pain
point in the DevOps world...
Not just limited to Puppet...
15
16. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Why is this a problem?
Two main issues...
16
17. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
First: Context switching is the mind killer
Humans suck at multitasking
17
18. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
18
“The trick here is that when you manage programmers, specifically,
task switches take a really, really, really long time. That's because
programming is the kind of task where you have to keep a lot of things
in your head at once. The more things you remember at once, the
more productive you are at programming. A programmer coding at
full throttle is keeping zillions of things in their head at once: everything
from names of variables, data structures, important APIs, the names of
utility functions that they wrote and call a lot, even the name of the
subdirectory where they store their source code.”
Human Task Switches Considered Harmful - Joel Spolsky
http://www.joelonsoftware.com/articles/fog0000000022.html
19. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
19
● Get the people who are maintaining the code to look at
what's written
● How long does it take them to find out how something’s
done in your code?
● Ask them if they can easily understand what’s
happening and why
To fix it: Get an outsider's perspective
20. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Second: Maintaining this data is costly
Both technically and mentally
20
21. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
21
$ find . -type f | wc -l
587
Hiera can become bloated over time
22. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
22
● Minimise node specific data: it’s the hardest to maintain
and easiest to get stale
● Abstract information into a hierarchy to DRY it up
● Purge irrelevant data
● Remember, it’ll be in the git history still!
Clean up your hiera!
23. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
23
PR I’ve been meaning to resurrect:
maybe at the contributor summit?
- https://github.com/voxpupuli/puppet-syntax/pull/57
24. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Lack of validation/CI
The earlier you can catch errors, the cheaper it is to fix
them.
24
25. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
25
Gareth’s talk about the Future of Puppet testing
https://speakerdeck.com/garethr/the-future-o
f-testing-puppet-code
26. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
26
● Puppet-syntax checks puppet, yaml, erb and epp files
● When you get that under control, use puppet-lint for
style checks
● Then rspec-puppet for unit tests
● Then beaker/testkitchen for acceptance tests
● If you don’t have CI, fix that! Or at least use git pre/post
commit hooks as a basic gate
At its most basic, a syntax error should not be
deployable
27. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
27
Testing with containers makes tests fast and
repeatable
http://cfgmgmtcamp.eu/schedule/testing/andy-henriod.html
28. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Lack of VCS best practises
Monolithic repos make it hard to upgrade modules and for
multiple people to work on the module at one time
28
29. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Make the newbie experience better
People change teams, new people join. Make sure they
can contribute as soon as possible
29
30. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
● If you want people to follow specific workflows or
requirements make it easy as possible
● Prevent risky newbie mistakes with CI and tooling
(puppet-lint, rubocop, rake tasks etc)
● Pair-programming is not just about the code: it’s about
learning the processes and improving the experience
for both new and old users
30
The bar should be low enough to trip over
31. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
313131
● Comments can be a crutch
● The best comment is a good name for a method or
variable
● Sometimes your hand is forced: make sure you
comment clearly and concisely
● Give the full context in the commit message
Code should be self-documenting
32. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
323232
https://blog.codinghorror.com/code-smells/
“There's a fine line between comments that illuminate and comments
that obscure. Are the comments necessary? Do they explain ‘why’ and
not ‘what’? Can you refactor the code so the comments aren't required?
And remember, you're writing comments for people, not
machines.”
33. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
333333
● Separate subject from body with a blank line
● Limit the subject line to 50 characters
● Capitalize the subject line
● Do not end the subject line with a period
● Use the imperative mood in the subject line
● Wrap the body at 72 characters
● Use the body to explain what and why vs. how
Git Commit best practices
34. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
343434
http://chris.beams.io/posts/git-commit/
“Re-establishing the context of a piece of code is wasteful. We can't
avoid it completely, so our efforts should go to reducing it [as much] as
possible. Commit messages can do exactly that and as a result, a
commit message shows whether a developer is a good
collaborator.”
35. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
35
Commit Often, Perfect Later, Publish Once
Learn how rebasing and amending works
36. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
363636
An example of good git commits at work
http://www.philandstuff.com/2014/02/09/git-pickaxe.html
37. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
373737
The Git Pickaxe
The reason I care about commit messages is because I'm an avid user of the git
pickaxe. If I'm ever confused about a line of code, and I want to know what
was going through the mind of the developer when they were writing it, the
pickaxe is the first tool I'll reach for. For example, let's say I was looking at this line
from our puppet-graphite module:
exec <%= @root_dir %>/bin/carbon-cache.py --debug start
That --debug option looks suspect. I might think to myself: "Why are we running
carbon-cache in --debug mode? Isn't that wasteful? Do we capture the output?
Why was it added in the first place?" In order to answer these questions, I'd like to
find the commit that added the switch.
38. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
383838
commit 5288d5804a3fc20dae4f3b2deeaa7f687595aff1
Author: Philip Potter <philip.g.potter@gmail.com>
Date: Tue Dec 17 09:33:59 2013 +0000
Re-add --debug option (reverts #11)
The --debug option is somewhat badly named -- it *both* adds debug
output, *and* causes carbon-cache to run in the foreground. Removing the
option in #11 caused the upstart script to lose track of the process as
carbon-cache started unexpectedly daemonizing.
Ideally we want to have a way of running through upstart without the
debug output, but this will fix the immediate problem.
Ta-dah, mystery solved!
39. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Reinventing the wheel
There are existing modules, use them!
39
40. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
404040
The more existing boilerplate and modules you can
leverage:
● The less work you have to do (Yay for you!)
● The more maintainable it will be (Yay for your team!)
● The more supportable it will be (Yay for everyone!)
Start off on the right foot
41. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
4141414141
Start with the puppet control repo
41https://github.com/puppetlabs/control-repo
42. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
424242424242
Use as supported modules as your number 1 choice
https://forge.puppet.com/modules?endorsements=supported
43. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
4343
Use approved modules as your number 2 choice
43https://forge.puppet.com/modules?endorsements=approved
44. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
4444
If there’s no supported or approved, use your best judgement...
44https://forge.puppet.com/modules?utf-8=%E2%9C%93&sort=downloads&q=limits
45. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
When writing new modules, use a custom module skeleton
https://github.com/petems/petems-puppet-module-skeleton 45
46. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
46
Write as little as possible
Code is like a puppy: anything you write you are responsible
for
47. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
47
Summary
What have we learnt?
48. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
48
Get your hiera under control
Clean up old files, validate and try and abstract
49. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
49
Get validation and CI on your codebase
syntax, lint, spec, acceptance
50. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
50
Don’t reinvent the wheel
Try and reuse as much as possible,
51. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
51
Make sure you’re VCS is top notch
Git is the primary place to show your intentions and the
history of the code
52. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
52
Ensure your newbie experience is great
Make the bar low enough to fall over
53. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
● PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob
Nelson, AT&T 71 views
http://www.slideshare.net/PuppetLabs/puppetconf-2016-enjoying-the-journey-
from-puppet-3x-to-4x-rob-nelson-att
● Does Your Configuration Code Smell? - Tushar Sharma, Marios
Fragkoulis and Diomidis Spinellis - Dept of Management Science and
Technology
http://www.tusharma.in/wp-content/uploads/2016/03/ConfigurationSmells_pre
print.pdf
● Roles and profiles: A complete example
https://docs.puppet.com/pe/2016.5/r_n_p_full_example.html
● Puppet Design Patterns - David Danzilio, Kovarus
http://www.slideshare.net/DavidDanzilio/puppet-design-patterns-puppetconf
53
Want to know more?
54. Knee deep in the undef
Tales from refactoring old Puppet codebases
@petersouter
Q&A
54