Hacking The Data out of Puppet - PuppetConf '12

Hacking
The Data
out of Puppet
Dan Bode| Puppet Labs
dan@puppetlabs.com

Who is this talk for?

•  People who already understand Puppet
•  Developers or people who are dev-curious

# puppetconf # puppetize @ puppetlabs

Shameless plug


What is it about?

•  Deconstructing Puppet to data


Puppet as Data
Dissecting a Puppet Run

Facter, who am I?

Hi! your facts are:

kernel=linux

ipaddress=10.0.0.3

Agent

macaddress=…


facts

Hi Mr. Master,

I need a catalog. Here
Agent

are my facts

http://www.dgcomputers.org/testimonials.php


facts

Thanks for you facts.

Agent

store them in
I’ll just
PuppetDB

PuppetDB


Mr. ENC, is this host
deﬁned as an external Yep, he should be an
node?

apache server. Here is
the deﬁnition

nodes

ENC

Agent

PuppetDB

facts


catalog

Just compiled your
Agent

One sec while
catalog. PuppetDB

I store it in PuppetDB.

facts


Here is your
catalog. Send me
a report and let
me know how it
went!

catalog

Agent

PuppetDB

catalog

facts


I hate to be a
bother, but can
you compute
the md5sums of
a few ﬁles?

catalog

Agent

PuppetDB

catalog

facts


report

Just ﬁnished applying.
Agent

are the results.

Here PuppetDB

catalog

facts


Termini and the indirector
Interacting with Puppet’s Data

facts ﬁnd from
terminus facter

Agent


facts

catalog ﬁnd from
terminus rest

Agent

http://www.dgcomputers.org/testimonials.php


facts

facts save to terminus
Agent

puppetdb

PuppetDB


node ﬁnd from
terminus exec (or
ldap)

nodes

ENC

Agent

PuppetDB

facts


catalog

catalog ﬁnd from
Agent

compiler

terminus PuppetDB

facts


catalog

catalog save to
Agent

puppetdb

terminus PuppetDB

facts


Disecting a Puppet Run

Facter

Facts

ENC

Nodes/
Manifest

Compiler

Reports

Conﬁg

Catalogs


CLI commands

puppet

facts

find

puppet

node

find

puppet

catalog

find


CLI Puppet Facts

# mkdir –p /tmp/yaml/facts

# puppet facts find node_name --render-as yaml
> /tmp/yaml/facts/node_name.yaml


Creating a node (optional):
# puppet node find node_name
--node_terminus=exec
--external_nodes=/etc/puppet/nodes.sh
--facts_terminus=yaml
--clientyamldir=/tmp/yaml/ --render-as=yaml
> /tmp/yaml/nodes/node_name.yaml


Creating a catalog:
# puppet catalog find node_name
--facts_terminus=yaml

# puppet catalog find node_name
--node_terminus=yaml
--manifest=/etc/puppet/manifest/site.pp
--modulepath=/etc/puppet/modules/
--clientyamldir=/tmp/yaml/ > /tmp/catalog.yaml

Fun with IRB

Puppet::Node::Facts.indirection.ﬁnd

facts

Puppet::Node.new

nodes

Puppet::Catalog.indirection.ﬁnd

catalog


IRB Facts

irb:> require ‘puppet/face’
> facts=Puppet::Face[:facts, :current].find('node’)


Access a Fact value (irb):
…
> facts.values['ipaddress']
=> "10.0.2.15"


Creating a node (from irb):
…
> node=Puppet::Node.new('node_name',
{:classes => {:foo => {:bar => :baz}}})
>node.merge(facts.values)


Creating a catalog:
…
irb> catalog=Puppet::Resource::Catalog.indirection.
find('node_name’, :node => node)


Use Cases
Interacting with Puppet’s Data

Inspecting the catalog:

•  What types are in the catalog?
irb> catalog.resources.collect do |r| r.type end.uniq

•  Gimme a resource:
irb>catalog.resource(‘Package[httpd]’)


Rspec Puppet:
let :facts do
{:operatingsystem => ‘Redhat’}
end
let :params do
{:bind_address => ‘0.0.0.0’
end
it { should contain_file(‘/tmp/foo.conf’) }


Thundering Herd

Pre-compile catalogs for faster auto-scaling


Applying pre-compiled
catalogs:

puppet apply --catalog /tmp/catalog.json –server
puppet-fileserver


DMZ

tcp over USB


Hacking reports

Everything in Puppet is a state transition

User[‘dan’] : absent -> present
User[‘dan’][‘shell’] -> ‘/sbin/nologin’ -> /bin/bash


Setting up the agent:

[agent]
report=true


Archive reports in your
yamldir

[master]
reports = store


Puppet reports

$ cd `puppet config print reportdir`
$ ls
node1 node2 node3
$ ls node1


Every report from every run
ever

$ ls node1
201206060256.yaml 201206060303.yaml
201206060519.yaml 201206122349.yaml
201206122354.yaml 201206130002.yaml


Lets crack one open!

Irb > require ‘yaml’
>reports=YAML.load_file('201206130002.yaml')


Have a look

>(reports.methods - Object.methods).sort

Notice the following methods:


High level data
> reports.exit_status
⇒ 0
> reports.status
=> "unchanged"
> reports.host
⇒ ”node1”


metrics

> reports.metrics.keys
⇒ ["resources", "events", "changes", "time"]
> reports.metrics['resources']
⇒ [‘failed’, 0],[ ‘changed’, ‘7’]


And the awesome sauce

> reports.resource_statuses.keys
=> ["Package[xinetd]", "File[/srv/node/1]",
"Package[swift]", "Exec[compile fragments]",
"Package[swift-container]", "File[/var/opt/lib/pe-
puppet/concat/_etc_swift_object-server.conf]",
"File[/etc/rsync.d/frag-account]”]



> status = reports.resource_statuses
> status.keys
=> ["Package[xinetd]", "File[/srv/node/1]",
"Package[swift]", "Exec[compile fragments]",
"Package[swift-container]", "File[/var/opt/lib/pe-
puppet/concat/_etc_swift_object-server.conf]",
"File[/etc/rsync.d/frag-account]”]


>events = status["File[/etc/swift/swift.conf]"].events

> events.first.status

⇒ "success”
> events.first.desired_value

⇒ :present
> events.first.previous_value

=> :absent


Thank You
Dan Bode| Puppet Labs
dan@puppetlabs.com

Hacking The Data out of Puppet - PuppetConf '12

Hacking The Data out of Puppet - PuppetConf '12

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (11)

Similaire à Hacking The Data out of Puppet - PuppetConf '12

Similaire à Hacking The Data out of Puppet - PuppetConf '12 (7)

Plus de Puppet

Plus de Puppet (20)

Dernier

Dernier (20)

Hacking The Data out of Puppet - PuppetConf '12