1. Puppet for
Dummies
ZendCon - October 2011
Santa Clara - United States
http://joind.in/3781

2. Who am I?
Joshua Thijssen
Senior Software Engineer @ Enrise (Netherlands)
Development in PHP, Python, Perl, C, Java,
and system & DB admin.
Blog: http://www.adayinthelifeof.nl
Email: joshua@enrise.com
Twitter: @jaytaph
http://www.flickr.com/photos/akrabat/5422369749/in/photostream/

3. The question of the day

4. The question of the day
What is puppet and why should I care?

5. Why use puppet?
“People are finally figuring out puppet
and how it gets you to the pub by 4pm.
Note that I’ve been at this pub since
2pm.”
- Jorge Castro

6. Why use puppet?

7. What is puppet?
Puppet is a (not necessarily the)
solution for the following problem:
How do we setup, manage, synchronize,
and upgrade our internal and external
infrastructure?

8. What is puppet?
But isn’t that a sysadmin problem??

9. What is puppet?
Short answer:

10. What is puppet?
Short answer:
NO

11. How do we manage our infrastructure?

12. How do we manage our infrastructure?
‣ Solution 1: We don’t,

13. How do we manage our infrastructure?
‣ Solution 1: We don’t,
‣ Solution 2: We outsource,

14. How do we manage our infrastructure?
‣ Solution 1: We don’t,
‣ Solution 2: We outsource,
‣ Solution 3: We automate the process.

15. How do we manage our infrastructure? (1)

16. How do we manage our infrastructure? (1)
‣ It’s not funny: you find it more often
than not. Especially inside small
development companies.

17. How do we manage our infrastructure? (1)
‣ It’s not funny: you find it more often
than not. Especially inside small
development companies.
‣ Internal sysadmin, but he’s too busy
with development to do sysadmin.

18. How do we manage our infrastructure? (1)
‣ It’s not funny: you find it more often
than not. Especially inside small
development companies.
‣ Internal sysadmin, but he’s too busy
with development to do sysadmin.
‣ We only act on escalation

19. How do we manage our infrastructure? (1)
‣ It’s not funny: you find it more often
than not. Especially inside small
development companies.
‣ Internal sysadmin, but he’s too busy
with development to do sysadmin.
‣ We only act on escalation
‣ reactive, not proactive

20. How do we manage our infrastructure? (2)

21. How do we manage our infrastructure? (2)
‣ Expensive $LA’s.

22. How do we manage our infrastructure? (2)
‣ Expensive $LA’s.
‣ What about INTERNAL servers like
your development systems and
infrastructure?

23. How do we manage our infrastructure? (2)
‣ Expensive $LA’s.
‣ What about INTERNAL servers like
your development systems and
infrastructure?
‣ Fight between stability and agility.

24. How do we manage our infrastructure? (2)
‣ Expensive $LA’s.
‣ What about INTERNAL servers like
your development systems and
infrastructure?
‣ Fight between stability and agility.
‣ Does your hosting company decide
on whether you can use PHP5.3???

25. How do we manage our infrastructure? (3)

26. How do we manage our infrastructure? (3)
‣ We are in charge.

27. How do we manage our infrastructure? (3)
‣ We are in charge.
‣ Dedicated package repositories,
tools, etc,..

28. How do we manage our infrastructure? (3)
‣ We are in charge.
‣ Dedicated package repositories,
tools, etc,..
‣ Use: cfEngine, chef, puppet.

29. How do we manage our infrastructure? (3)
‣ We are in charge.
‣ Dedicated package repositories,
tools, etc,..
‣ Use: cfEngine, chef, puppet.
‣ It’s actually not that hard.

30. What is puppet?
‣ Open source configuration
management tool.
‣ Written in Ruby
‣ Open source
https://github.com/puppetlabs
‣ Commercial version available
(puppet enterprise)

31. What is puppet?
¹
‣ Don’t tell HOW to do stuff.
‣ Tell WHAT to do.
¹ It’s not actually true, but good enough for now...

32. Architectural overview

33. Architectural overview
Puppet

34. Architectural overview
Puppet
Puppet CA
Master
https
Puppet
Agent

35. Architectural overview
Puppet
Puppet CA
Master
https
Puppet Puppet Puppet
Agent Agent Agent

36. Puppet structure
‣ Puppet master (puppetmasterd)
‣ Puppet cert (puppetca)
‣ Puppet agent (puppetd)
‣ Facter

37. Puppet master (puppetmasterd)
‣ Central server
‣ File & configuration server
‣ REST over HTTPS interface

38. Puppet cert (puppet CA)
‣ Certificate signing server
‣ Creates, signs, checks x509 certificates
‣ So you don’t have to worry about it

39. Puppet cert (puppet CA)
Check all systems that have connected to our CA server
root@puppetmaster:~# puppet cert --list --all
+ puppetmaster.noxlogic.local
(74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C)
puppetnode1.noxlogic.local
(09:9D:1E:01:D0:A7:BA:FB:8C:F4:2D:96:78:34:54:44)

40. Puppet cert (puppet CA)
Let’s sign our first node
root@puppetmaster:~# puppet cert --sign puppetnode1.noxlogic.local
....
root@puppetmaster:~# puppet cert --list --all
+ puppetmaster.noxlogic.local
(74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C)
+ puppetnode1.noxlogic.local
(CC:50:49:98:1D:F9:06:36:0E:6E:31:F5:27:D8:50:D8)

41. Puppet agent (puppetd)
‣ Runs on every node that will be
managed by puppet.
‣ Calls the puppet master every 30
minutes with system information.
‣ Receives and executes a catalog.

42. Facter
‣ Runs on nodes to gather system
information.
‣ Returns $variables to be used in
configuration.

43. Facter (1)
[root@puppetnode1 ~]# facter --puppet
architecture => x86_64
fqdn => puppetnode1.noxlogic.local
interfaces => eth1,eth2,lo
ipaddress_eth1 => 192.168.1.114
ipaddress_eth2 => 192.168.56.200
kernel => Linux
kernelmajversion => 2.6
operatingsystem => CentOS
operatingsystemrelease => 6.0
processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz
puppetversion => 2.6.9
‣ A simple list with info (also useable in your own tools)

44. Facter (2)
‣ You can add your own facts:
‣ project name
‣ master / slave database server
‣ zend server
‣ directadmin / plesk
‣ Very simple to add new facts (in ruby, that is)

45. Facter (3)
zendstudio.rb:
Facter.add(“Zendserver”) do
confine :kernel => :linux
setcode do
if FileTest.exists?(“/usr/local/zend/bin”)
“true”
else
“false”
end
end
end
‣ Crude, but effective enough for us

46. How does it work
Check cert
Master Return facts Client
Returns catalog

47. Puppet manifests
‣ Manifests are puppet definitions
‣ <filename>.pp
‣ Puppet DSL
‣ De-cla-ra-tive language
‣ Version your manifests! (git/svn)

48. Puppet manifests
package { “mc” :
ensure => present,
}
file { “/home/jaytaph/secret-ingredient.txt” :
ensure => present,
mode => 0600,
user => ‘jaytaph’,
group => ‘noxlogic’,
source => “puppet:///secret.txt”,
}

49. Puppet manifests
package { “httpd” :
ensure => present,
}
service { “httpd”:
running => true,
enable => true,
require => Package[“httpd”],
}
‣ Spot the problem....

50. Puppet manifests
Centos / Redhat
service: httpd
package: httpd
config: /etc/httpd/conf/httpd.conf
vhosts: /etc/httpd/conf.d/*.conf
Debian / Ubuntu
service: apache2
package: apache2
config: /etc/apache2/httpd.conf
vhosts: /etc/apache2/sites-available
‣ Different distributions, different names

51. Puppet manifests
service { “apache”:
case $operatingsystem {
centos, redhat { $apache = “httpd” }
debian, ubuntu { $apache = “apache2” }
default : { fail(‘I don’t know this OS/distro’) }
}
name => $apache,
running => true,
}
‣ $operatingsystem is a FACT

52. Puppet manifests
/etc/puppet/manifests/site.pp:
node default {
$def_packages = [ “mc”, “strace”, “sysstat” ]
package { $def_packages :
ensure => latest,
}
}
‣ “Main” manifest

53. Puppet manifests
node ‘web.noxlogic.local’ {
package { “httpd” :
ensure => latest,
}
}
node ‘db.noxlogic.local’ {
package { “mysql-server” :
ensure => installed,
}
}
‣ Defining nodes

54. Puppet manifests
node basenode {
user { “jaytaph” :
ensure => present,
gid => 1000,
uid => 1000,
home => “/home/jaytaph”,
shell => “/bin/sh”,
password => “supersecrethashedpassword”,
}
}
node *.noxlogic.local inherits basenode {
...
}

55. Puppet manifests

56. Puppet manifests
class webserver {
service { “apache”:
ensure => running,
require => Package[“httpd”],
}
package { “apache” :
ensure => installed,
}
}

57. Puppet manifests
class webserver {
service { “apache”:
ensure => running,
require => Package[“httpd”],
}
package { “apache” :
ensure => installed,
}
file { “vhost_$hostname” :
path => “/etc/httpd/conf/10-vhost.conf”,
content => template(“vhost.template.erb”),
notify => Service[“httpd”],
}
}

58. Puppet manifests
vhost.template.erb
<virtualHost <%= ipaddress %>:80>
ServerName <%= webserver_name %>
ServerAlias <%= webserver_alias %>
DocumentRoot <%= webserver_docroot %>
</virtualHost>
‣ ERB Templates can use custom variables and facts

59. Puppet manifests
node “web01.noxlogic.local” inherits base {
$webserver_name = “web01.noxlogic.local”
$webserver_alias = “www.noxlogic.local”
$webserver_docroot = “/var/www/web01”
import webserver
}
node “web02.noxlogic.local” inherits base {
$webserver_name = “web02.noxlogic.local”
$webserver_alias = “crm.noxlogic.local”
$webserver_docroot = “/var/www/web02”
import webserver
}

60. What can puppet manage
‣ Almost everything.
‣ standard 48 different resource types
‣ Ranging from “file” to “cron” to
“ssh_key” to “user” to “selinux”.
‣ Can control your Cisco routers and
windows machines too (sortakinda)
‣ http://docs.puppetlabs.com/references/stable/type.html

61. Puppet modules
‣ A puppet module is a collection of
resources, classes, templates.
‣ Used for easy distribution and
code-reuse.
‣ Self-contained, run out-of-the-box

62. Puppet modules
‣ puppetforge / github
‣ Create your own (and share!).
‣ Use the ones from puppet
enterprise edition.
‣ Use the standard layout / best
practices

63. Puppet modules
MODULE_PATH/
└──downcased_module_name/
├──files/
├──manifests/
│ ├──init.pp
│ └──foo.pp
├──lib/
│ ├──puppet/
│ │ ├──parser/
│ │ │ └──functions/
│ │ ├──provider/
│ │ └──type/
│ └──facter/
├──templates/
├──tests
│ ├──init.pp
│ └──foo.pp
└──README

64. Puppet modules
class ntp::install {
package{"ntpd":
ensure => latest
}
}
class ntp::config {
File{
require => Class["ntp::install"],
notify => Class["ntp::service"],
owner => "root",
group => "root",
mode => 644
}
file{"/etc/ntp.conf":
source => "puppet:///ntp/ntp.conf";
"/etc/ntp/step-tickers":
source => "puppet:///ntp/step-tickers";
}
}
class ntp::service {
service{"ntp":
ensure => running,
enable => true,
require => Class["ntp::config"],
}
}
class ntp {
include ntp::install, ntp::config, ntp::service
}

65. Test your modules
‣ (Unit)test your modules
‣ Test them with:
puppet apply --noop
‣ More advanced testing: cucumber /
cucumber-puppet (BDD)

66. External Node Configuration (1)
‣ Split modules and nodes
‣ Nodes should be classes - params
only (best case scenario?)
‣ Nodes can be configured through
YAML

67. External Node Configuration (2)
node1.enrise.local.yaml
---
classes:
- base
parameters:
puppetserver: puppet.enrise.local

68. External Node Configuration (2)
node1.enrise.local.yaml
---
classes:
- base
parameters:
puppetserver: puppet.enrise.local
node node1.enrise.local {
$puppetserver = ‘puppet.enrise.local’
include base
}

69. External Node Configuration (3)
Puppet doesn’t care how you create YAML files.
‣ Ruby, PHP, Python, Perl, Pony,
shellscript.
‣ REST, SOAP, XMLRPC.
‣ Use a database backend.
‣ Or use LDAP instead of YAML.

70. Confusing puppet things

71. Confusing puppet things
‣ Puppet went from v0.25 to v2.6.
‣ REST interface since 2.6. XMLRPC
before that.
‣ One binary to rule them all (puppet).
‣ Puppet v2.7 switched from GPLv2 to
apache2.0 license.

72. Confusing puppet things
‣ --test does not mean dry-run!
(--noop does).
‣ It’s not object oriented. (puppet
class != php class)
‣ It’s a declarative language.

73. Puppet dashboard
http://media.techtarget.com/digitalguide/images/Misc/puppetDashboard.gif

74. Live demo | MCollective?

75. MCollective
‣ Puppet agent “calls” the master every 30
minutes.
‣ But what about realtime command & control?
‣ “Puppet kick”... (meh)
‣ MCollective (Marionette Collective)

76. MCollective
‣ Which systems running a database
and have 16GB or less?
‣ Which systems are using <50% of
available memory?
‣ Restart all apache services in
timezone GMT+5.
‣ How do we handle large number of nodes?

77. MCollective
Client Middleware Node
MCollective
Server
MCollective
Client ACTIVEMQ
Server
MCollective
Server
Collective
‣ Middleware takes care of distribution,
‣ queued, broadcast etc..

78. MCollective
‣ The collective
http://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html

79. MCollective
$ mc-facts operatingsystem
Report for fact: operatingsystem
CentOS found 3 times
Debian found 14 times
Solaris found 4 times
$ mc-facts -W operatingsystem=Centos operatingsystemrelease
Report for fact: operatingsystemrelease
6.0 found 1 times
5.6 found 2 times
‣ Filter out nodes based on facts

80. MCollective - cool stuff
‣ Display all running processes
‣ Run or deploy software
‣ Restart services
‣ Start puppet agent
‣ Upgrade your systems

81. Recap
-ETOOMUCHINFO
Let’s recap

82. Recap (1)
‣ Configuration management tool.
‣ Focusses on “what” instead of “how”.
‣ Scales from 1 to 100K+ systems.
‣ Uses descriptive manifests.
‣ Can use external node configurations.

83. Recap (2)
‣ Useful for sysadmins and developers.
‣ Keeps your infrastructure in sync.
‣ Keeps your infrastructure versioned.
‣ MCollective controls your hosts
based on facts, not names.

84. Any questions?
http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg

85. ‣ THANK YOU FOR YOUR ATTENTION