This talk will cover the basic life cycle of a Puppet Catalog from compilation request to report processing. It will explore the performance of some of the life cycle steps and show how you might instrument these steps using tools Puppet make available.
Along the way it will provide hints and tips on how to write performant facts and manifests.
2. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Who am I?
• Puppet user since 0.22.x
• Blog at http://devco.net
• Tweets at @ripienaar
• Volcane on IRC
3. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Why?
• Unit and Integration testing
• Cloud based scaling
• Maintenance windows and fast feedback
• Resource usage on the master
7. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Shows per fact timings…but only the setcode part
FREEGEOURL = “http://freegeoip.net/json/"
geoipinfo = JSON.parse(open(FREEGEOURL).read)
geoipinfo.each do |k, v|
next if k == "ip"
Facter.add("geo_#{k.downcase}") { setcode { v } }
end
Timing Facts
geo_latitude: 0.02ms
geo_region_name: 0.04ms
geo_metro_code: 0.04ms
geo_time_zone: 0.03ms
geo_country_code: 0.02ms
geo_country_name: 0.02ms
geo_zip_code: 0.02ms
geo_region_code: 0.04ms
geo_city: 0.04ms
geo_longitude: 0.03ms
8. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Restructuring fact to do the slow work in setcode
geoipinfo = nil
["country_code", “country_name”…].each do |key|
Facter.add(“geo_%s” % key.downcase) do
setcode do
geoipinfo ||= JSON.parse(open(FREEGEOURL).read)
["", nil].include?(geoipinfo[key]) ? "unknown" : geoipinfo[key]
end
end
end
Timing Facts
geo_zipcode: 7174.49ms
geo_metro_code: 0.04ms
geo_country_code: 0.03ms
geo_region_name: 0.04ms
geo_latitude: 0.05ms
geo_area_code: 0.04ms
geo_country_name: 0.03ms
geo_city: 0.04ms
geo_longitude: 0.04ms
geo_region_code: 0.04ms
Fetch once per run
and re-use
9. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Add a local cache, now only the first invoke is slow
def geoip_cached_fetch
store = PStore.new(File.join(Dir.tmpdir, “fgeoip_fact.pstore"))
store.transaction do
store[:freegeoip] ||= JSON.parse(open(FREEGEOURL).read)
end
end
geoipinfo = nil
["country_code", “country_name”…].each do |key|
Facter.add(“geo_%s” % key.downcase) do
setcode do
geoipinfo ||= geoip_cached_fetch
["", nil].include?(geoipinfo[key]) ? "unknown" : geoipinfo[key]
end
end
end
Timing Facts
16. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
class {'epel': } ->
class {'puppetdb': } ->
class {'puppet::master':
storeconfigs => true
}
Compilation
$ puppet apply test.pp 2>&1|grep
"Looking for data source"|wc -l
3562
17. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
# /etc/puppet/modules/apache/init.pp
class apache($version=present) {
validate_string($version)
class{“::apache::package”:
version => $version
}
}
Compilation
ruby code, similar issues as the fact
18. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
# /etc/puppet/modules/apache/init.pp
class apache($version=present) {
validate_string($version)
class{“::apache::package”:
version => $version
}
}
Compilation
also speeds up finding
the class internally
19. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
# /etc/puppet/modules/apache/init.pp
class apache($version=present) {
validate_string($version)
class{“::apache::package”:
version => $version
}
}
Compilation
No hiera lookup,
authoritative value supplied
20. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
class {'epel': } ->
class {'puppetdb': } ->
class {'puppet::master':
storeconfigs => true
}
Profiling Compilation
$ puppet apply test.pp —profile
21. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
importing ‘..epel/manifests/init.pp’
importing ‘..epel/manifests/params.pp’
importing ‘..epel/manifests/rpm_gpg_key.pp’
importing ‘..puppetdb/manifests/init.pp’
importing ‘..puppetdb/manifests/params.pp’
PROFILE [apply] 2.3.1 Called defined: took 0.0001 seconds
PROFILE [apply] 2.3.2 Called downcase: took 0.0000 seconds
PROFILE [apply] 2.3.3 Called validate_re: took 0.0000 seconds
PROFILE [apply] 2.3.4 Called downcase: took 0.0000 seconds
PROFILE [apply] 2.3.5 Called validate_re: took 0.0000 seconds
PROFILE [apply] 2.3.6 Called downcase: took 0.0001 seconds
PROFILE [apply] 2.3.7 Called validate_re: took 0.0000 seconds
importing ‘..puppetdb/manifests/server.pp’ in environment production
PROFILE [apply] 2.3.8 Called downcase: took 0.0000 seconds
PROFILE [apply] 2.3.9 Called validate_re: took 0.0000 seconds
Profiling Compilation
class {'epel': } ->
class {'puppetdb': } ->
class {'puppet::master':
storeconfigs => true
}