Infrastructure as Code with Chef Puppet

Infrastructure as Code
with Chef / Puppet
Edmund Haselwanter (e.haselwanter@cloudbau.de)

Samstag, 16. November 13

Introduction

Edmund Haselwanter

●Co-Founder and CSO cloudbau GmbH
●Dell Professional Services (Openstack/Crowbar/
Automation)

●Software Development Background
●10 Years Infrastructure Automation
●@ehaselwanter on twitter/github/irc/...
cloudbau

Business-Agility needs IT-Agility
For most companies, IT functions as the nervous system and provides
an increasing amount of the organizational muscle mass. Most critical
business functions are entirely automated within IT, and 95% of all
capital projects depend on IT to get done. Today, nearly every business
decision will result in at least one IT change.
„IT REVOLUTION MANIFESTO“, http://itrevolution.com

Theory of
Constraints

→

→
E.M. Goldratt, 1984

cloudbau

Gene Kim, Kevin Behr,
George Spafford, 2013

IT-Agility and the „Risk of Change“
Request

Approve

Develop

Test

WaterScrum

Deploy

- Fall

The core, chronic conflict that every IT
leader faces is the need to
simultaneously enable faster time to
market (i.e., make as many changes as
you can), while providing stable, secure
and reliable IT services (i.e., make as
few changes as you can).
Gene Kim

„Lowering risk of change through tools and culture“
John Allspaw, Paul Hammond, 2009

small changes, often
reproducible


cooperative culture

process standards

cloudbau

expect failure
metrics

Tools and Culture
Config-DB

CPUVirt.
Binary-Repo

StorageVirt.
SDN

Orchestration

Prod Env

Programmable Infrastructure

Description in Code

Automatic Build
DevOps

Cross-Functional Teams

Process and Culture

cloudbau

Stage Env

Anti-Fragile Organizations

Continuos Delivery

Description

Dev Env

Configuration Management

cloudbau

Evolving towards Configuration
Management

●Just build it
●Keep notes in server.txt
●Move notes to the wiki
●Custom scripts (in scm?!)
●Snapshot & Clone
cloudbau

Applications

http://www.flickr.com/photos/steffenz/
337700069/
http://www.flickr.com/photos/kky/
704056791/

cloudbau

Infrastructure

cloudbau
http://www.flickr.com/photos/sbh/462754460/

Collection of Resources

•
•
•
•
•

Networking
Files
Directories
Symlinks
Mounts

•
•
•
•
•
•
•
•
•

cloudbau
http://www.flickr.com/photos/philliecasablanca/3354734116/

Routes
Users
Groups
Tasks
Packages
Software
Services
Configuration
Other Stuff

Acting in Concert

cloudbau

http://www.flickr.com/photos/glowjangles/4081048126/

To Provide a Service

cloudbau

http://www.flickr.com/photos/28309157@N08/3743455858/

And it Evolves

cloudbau

http://www.flickr.com/photos/16339684@N00/2681435235/

See Node

Application Server

cloudbau

See Nodes

Application Server

Application Database

cloudbau

See Nodes Grow

Application Server

Application Databases

cloudbau

See Nodes Grow

Application Servers


cloudbau

See Nodes Grow

Load Balancer

Application Servers


cloudbau

See Nodes Grow

Load Balancers

Application Servers


cloudbau

See Nodes Grow

Load Balancers

Application Servers

Application Database Cache


cloudbau

Tied Together with Configuration

Load Balancers

Application Servers

Application Database Cache


cloudbau

So when this

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

cloudbau

Postgres Master

Becomes This

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

cloudbau

Postgres Master

This can happen automagically

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

cloudbau

Postgres Master

Count the Resources

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

•

•
•
•
•
•
•
•
•
•
•
•
•

Load balancer config
Nagios host ping
Nagios host ssh
Nagios host HTTP
Nagios host app health
Graphite CPU
Graphite Memory
Graphite Disk
Graphite SNMP
Memcache firewall
Postgres firewall
Postgres authZ config

12+ resource changes for 1 node addition
cloudbau


Focus Today: Chef and Puppet

Series A: 2,5 M$ (2009)
Series B: 11 M$ (2010)
Series C: 19,5 M$ (2012)

Technologie

cloudbau

Series A: 2 M$ (2009)
Series B: 5 M$ (2010)
Series C: 8,5 M$ (2011)
Series D: 30 M$ (2013)

Technologie

*alle Angaben von www.crunchbase.com

Chef & Puppet

since 2005

since 2009

Declarative DSL

Ruby DSL

Customers:

Customers:

• CERN

•

Facebook

• Zynga

•

SAP

•VMware

•

IBM (Smartcloud)

•


AWS (OpsWorks)

• Twitter

cloudbau

•

Microsoft

What is Chef?
Chef is an IT automation platform for developers & systems engineers to continuously
define, build, and manage infrastructure.
CHEF USES:

Recipes and
Cookbooks
that describe and deliver code.
Chef enables people to easily build &
manage complex & dynamic applications
at massive scale.

•  New model for describing infrastructure that
promotes reuse

•  Programmatically provision and configure
•  Reconstruct business from code repository,
data backup, and bare metal resources

cloudbau

“!

”!

cloudbau

What is Puppet?
Puppet is IT automation software that helps system administrators manage infrastructure
throughout its lifecycle, from provisioning and configuration to orchestration and reporting.

How Puppet Works
Puppet uses a declarative, model-based
approach to IT automation.
1. Define the desired state of the
infrastructure's configuration using
Puppet's declarative configuration
language.
2. Simulate configuration changes before
enforcing them.
3. Enforce the deployed desired state
automatically, correcting any configuration
drift.
4. Report on the differences between actual
and desired states and any changes made
enforcing the desired state.

cloudbau


Concepts

cloudbau

Declarative Language

Package A
Template 1

File 1

Package B

cloudbau

Service I

Convergence to desired State

State X

State Y
“run”

Package A

Package A

Template 1

File 1

File 1
Package B

cloudbau

Service I

Idempotence

State Y

State Y
“run”

Package A

Template 1

File 1

Package B

cloudbau

Service I

Package A

Template 1

File 1

Package B

Service I

The Implementations
Puppet and Chef

cloudbau

Terminology

Java

Puppet

Chef

*
.java

manifests *
.pp

recipes *
.rb

package

module

cookbook

cloudbau

Terminology

Java

Puppet

Chef

singleton

parametrized
classes

recipes with
attributes

defines, classes

definitions,
libraries

class
cloudbau

Syntax

Puppet

Chef

Limited DSL (by design)

DSL + full ruby

Extended by Definitions

Custom Resource providers

ERB & Custom Libraries

cloudbau

Definitions, Resource/Providers, Light
Weight Resource
Definitions, Resource/Providers, Light
Weight Resource
Providers, Libraries

Syntax Examples

Puppet

class myapache {
package “apache2”
service “apache2”:
ensure => “running”,
require => Package[“apache2”]
}

cloudbau

Chef

package “apache2”
service “apache2 do
action [:enable , :start]
end

Variables

Puppet

Chef

#special syntax

#native ruby

$bla = “aa”

bla = “aa”

#string interpolation

#string interpolation

$bla_string = “${bla}”

bla_string = #{bla}

cloudbau

Loops

Puppet

Chef

pass array of elements
erb template for loops
create_resource (:type, hash)

cloudbau

do while , repeat etc. available
through native ruby

Conditionals

Puppet (since 2.7)

Chef

if
case (execute block)
selectors (get value)

cloudbau

if, unless, case etc. available
through native ruby

Project Structure

Puppet Modules

Chef
attributes/
definitions/

files/
lib/

files/
libraries/

README.rdoc

metadata.rb

manifests/

providers/

templates/

README.rdoc
recipes/
resources/

cloudbau

templates/

Execution Model

Puppet

Chef

Compile catalog (directed graph model)
Then Execute

Top Down execution

only code for state transition is visited

every line of code is visited all the time

cloudbau

Execution Model

Puppet

Chef

standalone

standalone

client - server

cloudbau

client - server
hosted

System Information

Puppet/Factor => Flat Key/Value

Chef/Ohai => Nested Hash
"hostname": "server-1",
"fqdn": "server-1.example.com",
"domain": "example.com",
"network": {
"interfaces": {
"eth0": {
"type": "eth",
"number": "0",
"encapsulation": "Ethernet",
"addresses": {
"00:0C:29:43:26:C5": {
"family": "lladdr"
},
"192.168.177.138": {
"family": "inet",
"broadcast": "192.168.177.255",
"netmask": "255.255.255.0"
},

cloudbau

System Information

●Selects Package Provider
●Selects Service Provider
●....

cloudbau

System Information (Chef Example)

execute "load sysctl" do
command "/sbin/sysctl -p"
action :nothing
end
bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3,
pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048
# adjust shared memory and semaphores
template "/etc/sysctl.conf" do
source "sysctl.conf.erb"
variables(
:shmmax_in_bytes => bytes,
:shmall_in_pages => pages
)
notifies :run, "execute[load sysctl]", :immediately
end

cloudbau

Templates

# Set up application listeners here.

listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress]
weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>
cloudbau

Lint Tools

http://acrmp.github.io/foodcritic/

http://puppet-lint.com/

cloudbau

Dependency Management
https://github.com/applicationsonline/librarian-chef
http://berkshelf.com

http://librarian-puppet.com

cloudbau

Reports

cloudbau

Web Interface

cloudbau

Tools

●Razor for bare metal provisioning
●mcollectiv for remote execution/
orchestration

●test-kitchen for infrastructure testing
●vagrant for rapid development
● ....

cloudbau

Networking

●Switch Config
●Driven by SDN Adoption
●Lots of Vendors (Arista ..)

cloudbau

Community

cloudbau

Chef Community
Community Overview!
25,000+ Community
Members!
!
1,000+ Community "
Cookbooks!
!
250,000+ Cookbook
Downloads!
!
400+ Public Training
Attendees in the last year!
!
30+ Meetup Groups!
!

cloudbau

Over 200 Corporate Contributors

http://community.opscode.com/cookbooks
approx 1200 co0kbooks
134 maintained by Opscode

cloudbau

http://forge.puppetlabs.com
approx 1700 modules
72 maintained by Puppetlabs

cloudbau

The Price Tag

cloudbau

Puppet: How to Buy
Cumulative # of
Per Node
Per Node Premium
Nodes
Standard Support
Support*
1-10

Download FREE

NA

11-99

$ 99

Contact Sales

100-249

$ 93

Contact Sales

250-499

$ 88

$ 152

500-999

$ 83

$ 119

1000-2499

$ 79

$ 99

2500+

Contact Sales

Contact Sales

cloudbau

https://puppetlabs.com/puppet/how-to-buy

Puppet Enterprise vs. OpenSource

cloudbau

Chef: How to Buy
Free

Launch

Free

$ 120

$ 300

$ 600

Nodes

5

20

50

100

Standard
Support

—

✔

✔

✔

Price per
Month

Standard Premium

http://www.opscode.com/enterprise-chef/#plans-pricing

cloudbau

Enterprise Chef vs. OpenSource Chef

cloudbau

Danke!
cloudbau
Edmund Haselwanter
@ehaselwanter
e.haselwanter@cloudbau.de
+49 30 57701800
www.cloudbau.de


cloudbau GmbH
Körnerstr. 7-10
10785 Berlin

Credits:
• Patrick Debois (http://www.jedi.be/blog/) for some of his Puppet/Chef Slides
• Opscode (http://www.opscode.com) for product info and some slides
• Puppetlabs (http://puppetlabs.com) for product info


Infrastructure as Code with Chef Puppet

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (15)

Similaire à Infrastructure as Code with Chef Puppet

Similaire à Infrastructure as Code with Chef Puppet (20)

Dernier

Dernier (20)

Infrastructure as Code with Chef Puppet