Managing a single Puppet module isn't easy, especially if you want to stay up-to-date with current best practices, modern testing, and the Puppet-DSL guidelines. This becomes even more difficult when maintaining multiple modules. Modulesync is the open source tool to change this! Learn from Vox Pupuli how we manage over 130 modules with no overhead and how we lowered the bar for newcomers in the open source world to more easily contribute.
3. What? Maintain many many Puppet modules
Be helpful and polite to each other
Be open-minded to people and technology
Provide a home for orphaned Puppet modules
Unite lonely module and tooling authors to a collective
Ensure continued development for our code
Vox Pupuli
@bastelsblog for @voxpupuliorg
3 / 21
What isWhat is Vox PupuliVox Pupuli
2 / 212 / 21
4. What?
Who?
Amount of Vox Pupuli members with merge permissions img
Vox Pupuli
@bastelsblog for @voxpupuliorg
4 / 21
What? Maintain many many Puppet modules
Be helpful and polite to each other
Be open-minded to people and technology
Provide a home for orphaned Puppet modules
Unite lonely module and tooling authors to a collective
Ensure continued development for our code
Vox Pupuli
@bastelsblog for @voxpupuliorg
3 / 21
5. What?
Who?
Many git newbies
Rebasing a feature branch against upstream master is hard
Many newcomers to test frameworks
Dealing with rspec/rubocop/beaker is hard as well
Many administrators of legacy infrastructures
Stuff older than Puppet 5 without proper hiera5/datatype support
Vox Pupuli
@bastelsblog for @voxpupuliorg
5 / 21
What?
Who?
Amount of Vox Pupuli members with merge permissions
2016-11 2017-10 2018-02 2018-07 2018-11 2019-01 2019-06 2019-10 2020-01 2020-05
0
20
40
60
80
100
120
140
160
Vox Pupuli
@bastelsblog for @voxpupuliorg
4 / 21
6. What?
Who?
Why?
Maintaining a module on your own is hard
Changing best practices, vacations, losing interest
Proper testing of a module requires a lot of stuff from the ruby ecosystem
Ruby ecosystem and related tools are changing too fast for a single
person
Everything has to work on multiple Ruby/Puppet versions
Rubocop
Rubocop
Vox Pupuli
@bastelsblog for @voxpupuliorg
6 / 21
What?
Who?
Many git newbies
Rebasing a feature branch against upstream master is hard
Many newcomers to test frameworks
Dealing with rspec/rubocop/beaker is hard as well
Many administrators of legacy infrastructures
Stuff older than Puppet 5 without proper hiera5/datatype support
Vox Pupuli
@bastelsblog for @voxpupuliorg
5 / 21
7. How? image
Vox Pupuli
7 / 21
What?
Who?
Why?
Maintaining a module on your own is hard
Changing best practices, vacations, losing interest
Proper testing of a module requires a lot of stuff from the ruby ecosystem
Ruby ecosystem and related tools are changing too fast for a single
person
Everything has to work on multiple Ruby/Puppet versions
Rubocop
Rubocop
Vox Pupuli
@bastelsblog for @voxpupuliorg
6 / 21
10. How?
Standards
Unified module file/directory structure
Unified code style
Unified review guidelines and testsuites with CI
Vox Pupuli
@bastelsblog for @voxpupuliorg
8 / 21
How?
Standards
Unified module file/directory structure
Unified code style
Vox Pupuli
@bastelsblog for @voxpupuliorg
8 / 21
11. How?
Standards
Documentation
Review guidelines: voxpupuli.org/docs/reviewing_pr
CONTRIBUTING.md
Explain how the PR creator can test their changes
Explain what happens after a PR was created
Templates for raising issues / creating a PR
Vox Pupuli
@bastelsblog for @voxpupuliorg
9 / 21
How?
Standards
Unified module file/directory structure
Unified code style
Unified review guidelines and testsuites with CI
Vox Pupuli
@bastelsblog for @voxpupuliorg
8 / 21
12. How?
Standards
Documentation
modulesync
We are managing more than 141 modules
113 in July 2019
modulesync was developed by Puppet Inc, moved to Vox Pupuli
It takes a config file + templates, throws it into every modules git repo
Everybody can start it
Vox Pupuli
@bastelsblog for @voxpupuliorg
10 / 21
How?
Standards
Documentation
Review guidelines: voxpupuli.org/docs/reviewing_pr
CONTRIBUTING.md
Explain how the PR creator can test their changes
Explain what happens after a PR was created
Templates for raising issues / creating a PR
Vox Pupuli
@bastelsblog for @voxpupuliorg
9 / 21
13. How?
Standards
Documentation
modulesync
The generic tool: github.com/voxpupuli/modulesync
The Vox Pupuli specific configuration:
github.com/voxpupuli/modulesync_config
Because of the separation, every organisation can use it
With Vox Pupuli configs
Or with own configs
Vox Pupuli
@bastelsblog for @voxpupuliorg
11 / 21
How?
Standards
Documentation
modulesync
We are managing more than 141 modules
113 in July 2019
modulesync was developed by Puppet Inc, moved to Vox Pupuli
It takes a config file + templates, throws it into every modules git repo
Everybody can start it
Vox Pupuli
@bastelsblog for @voxpupuliorg
10 / 21
14. How?
Standards
Documentation
modulesync
modulesync_config docroot
Vox Pupuli
12 / 21
How?
Standards
Documentation
modulesync
The generic tool: github.com/voxpupuli/modulesync
The Vox Pupuli specific configuration:
github.com/voxpupuli/modulesync_config
Because of the separation, every organisation can use it
With Vox Pupuli configs
Or with own configs
Vox Pupuli
@bastelsblog for @voxpupuliorg
11 / 21
16. How?
Standards
Documentation
modulesync
Setup
# checkout latest modulesync_config release
git checkout $(git tag --list | sort -V | tail -n1)
message="modulesync $(git describe)"
Sync all modules and create Pull Requests with a label
bundle exec msync update --message "{$message}"
--pr --pr-labels modulesync --pr-title "${message}"
Apply template to one module, don't commit
bundle exec msync update --filter "${module_name}" --noop
Vox Pupuli
@bastelsblog for @voxpupuliorg
14 / 21
How?
Standards
Documentation
modulesync
Vox Pupuli
13 / 21
17. How?
Standards
Documentation
modulesync
modulesync PR
Vox Pupuli
15 / 21
How?
Standards
Documentation
modulesync
Setup
# checkout latest modulesync_config release
git checkout $(git tag --list | sort -V | tail -n1)
message="modulesync $(git describe)"
Sync all modules and create Pull Requests with a label
bundle exec msync update --message "{$message}"
--pr --pr-labels modulesync --pr-title "${message}"
Apply template to one module, don't commit
bundle exec msync update --filter "${module_name}" --noop
Vox Pupuli
@bastelsblog for @voxpupuliorg
14 / 21
18. How?
Standards
Documentation
modulesync
Travis-CI
Free platform
rpsec-puppet common Puppet*Ruby versions
release modules to the forge
10-60min per module
Soon we will move some workload git GitHub Actions to speed things up
Vox Pupuli
@bastelsblog for @voxpupuliorg
16 / 21
How?
Standards
Documentation
modulesync
Vox Pupuli
15 / 21
19. How?
Standards
Documentation
modulesync
Travis-CI
Beaker
creates a virtual machine or docker instance
starts a complete OS in it
executes puppet in it
checks if everything looks as expected
Vox Pupuli
@bastelsblog for @voxpupuliorg
17 / 21
How?
Standards
Documentation
modulesync
Travis-CI
Free platform
rpsec-puppet common Puppet*Ruby versions
release modules to the forge
10-60min per module
Soon we will move some workload git GitHub Actions to speed things up
Vox Pupuli
@bastelsblog for @voxpupuliorg
16 / 21
20. How?
Standards
Documentation
modulesync
Travis-CI
Beaker
travis CI matrix
Vox Pupuli
18 / 21
How?
Standards
Documentation
modulesync
Travis-CI
Beaker
creates a virtual machine or docker instance
starts a complete OS in it
executes puppet in it
checks if everything looks as expected
Vox Pupuli
@bastelsblog for @voxpupuliorg
17 / 21
21. Summary We have great tooling and automation
We have great people
Vox Pupuli
@bastelsblog for @voxpupuliorg
19 / 21
How?
Standards
Documentation
modulesync
Travis-CI
Beaker
Vox Pupuli
18 / 21
22. Summary We have great tooling and automation
We have great people
You need help with Puppet?
Let us know, we help out!
You have an orphaned module?
Ping us, migrate it to us
You have domain specific knowledge?
Ruby, Python, Rspec, Beaker, $software we automate
Vox Pupuli
@bastelsblog for @voxpupuliorg
20 / 21
Summary We have great tooling and automation
We have great people
Vox Pupuli
@bastelsblog for @voxpupuliorg
19 / 21
23. Summary You want to help out?
We are always looking for new Collaborators and Maintainers
Contact:
IRC #voxpupuli on freenode
Slack: #voxpupuli on slack.puppet.com
Mailinglist: voxpupuli@groups.io
pmc@voxpupuli.org
tim@bastelfreak.de
More talks at github.com/bastelfreak/talks
Thanks for your attention!
Vox Pupuli
21 / 21
Summary We have great tooling and automation
We have great people
You need help with Puppet?
Let us know, we help out!
You have an orphaned module?
Ping us, migrate it to us
You have domain specific knowledge?
Ruby, Python, Rspec, Beaker, $software we automate
Vox Pupuli
@bastelsblog for @voxpupuliorg
20 / 21
24. Summary You want to help out?
We are always looking for new Collaborators and Maintainers
Contact:
IRC #voxpupuli on freenode
Slack: #voxpupuli on slack.puppet.com
Mailinglist: voxpupuli@groups.io
pmc@voxpupuli.org
tim@bastelfreak.de
More talks at github.com/bastelfreak/talks
Thanks for your attention!
Vox Pupuli
21 / 21