Building Data-Driven Infrastructure with Puppet - PuppetConf 2013

DataDrive
nInfrastructur
e
Friday, August 23, 13

?
http://git.io/UmUdKA

Let me
Tell you
a story

What is a
Cloud

I don’t have to think about IT
With the Cloud...

aaSPlatform
Software
Infrastruct
ure

Where is
our cloud?

James
Fryman

AutomationsJunkie

Are you
Crazy?

Goals?Data?
What

Machine
Parsable

There is SystemOne

Feedback
PuppetDB
Provisioning
Conﬁguring
Destroying
gPanel

Parts
of the
System

provisioner

Home
Grown

Home Grown

controller

Lots O’
Models

file { '/etc/facter/facts.d/quagga_manage_service':
ensure => present,
replace => false,
}

if $::quagga_manage_service {
$service_state[ensure] = running
$service_state[enable] = true
} else {
$service_state[ensure] = undef
$service_state[enable] = undef
}

service { 'quagga':
ensure => $service_state[ensure],
enable => $service_state[enable],
}

orchestrator

Chat
Ops

Feedback
of the
System

Metrics
&Monitoring

begin
JSON.parse(RestClient.get(url)).each do |cache|
data["#{cache['target']}"] = 0
count = 0
cache["datapoints"].each do |point|
unless (point[0].nil?)
data["#{cache['target']}"] += point[0]
count += 1
end
end
data["#{cache['target']}"] /= count
data["total"] += data["#{cache['target']}"]
end
rescue ZeroDivisionError => e
puts "UNKNOWN metric not in graphite!"
exit EXIT_UNKNOWN
rescue => e
puts e.message
exit EXIT_CRITICAL
end

event
log

event
alertlog

event
alertlog error

Must self-correct

Deployable
using text ﬁles

Modularity

haproxy::proxy { $es_proxy_name:
proxy => 'listen',
mode => 'http',
ip => $::ipaddress_lo,
port => '9200',
config => {
balance => 'roundrobin',
},
}

haproxy::proxy::member { $es_proxy_name:
hostname => $::ec2_local_ipv4,
port => '9200',
param => [
'weight 1',
'maxconn 1000',
'check',
],
}

collectd::plugin { [
'cpu',
'load',
'memory',
'swap',
'irq',
'exec',
'entropy',
]:
}

collectd::plugin { [
'df',
'interface',
'protocols',
'disk',
]:
config => true
}

Level 4:
Templates
Level 5:
Data Driven

Self
Authoritati
ve

def self.collect_resources(resource)
query = [
"and",
["=", "type", "File"],
["=", "exported", true],
["=", "tag", "nagios::object::#{resource}"],
["=", ["node", "active"], true]
]
self.search('/v2/resources', query.to_json)
end

# Seed the initial file with false, and enable
# on next run with gh-nagios enable_pager <hostname>
file { '/etc/facter/facts.d/enable_pager.txt':
ensure => file,
owner => 'root',
group => 'root',
mode => '0644',
content => 'enable_pager=false',
replace => false,
}

@@file { "/etc/nagios/objects/${type}.d/${filename}.cfg":
content => template('nagios/etc/nagios/object.erb'),
mode => '0444',
group => 'nagios',
notify => Service['nagios'],
}

Refactoring

least to most
speciﬁc

frymanet.com
mysqlnginx rails
rubycommon admin
package repos

graduate to
params

class ntp::params {
$defaults = {
package => {
version => ‘latest’,
},
config => {
servers => [‘pool.ntp.org’],
},
}
}

externalize

class ntp::params {
$defaults = {
package => {
version =>
hiera(‘ntp_package_version’),
},
config => {
servers => hiera(‘ntp_servers’),
},
}
}

CloudFormation

"Resources": {
"RendererServerGroup": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": { "Fn::GetAZs": "" },
"LaunchConﬁgurationName": { "Ref": "LaunchConﬁg" },
"MinSize": "2",
"MaxSize": "16",
"Tags": [
{ "Key": "Environment", "Value": "Production",
"PropagateAtLaunch": "true" },
{ "Key": "Role", "Value": "renderer", "PropagateAtLaunch": "true" }
]
}
},

Autoloading

# autoloader.pp
class nagios::autoload_helpers {
$helpers = get_nagios_helpers()
case $::puppetversion {
/^3/: { include $helpers }
default: {
nagios::autoload_helpers::import_shim { $helpers: }
}
}
}

module Puppet::Parser::Functions
newfunction(:get_nagios_helpers,
:type => :rvalue,
:doc => "Grab all modules that have nagios helpers for
import") do
module_path = File.expand_path('..',
Puppet::Module.find('nagios',compiler.environment.to_s).path)
helpers = Dir["#{module_path}/**/nagios/helpers.pp"].map do |d|
"#{d.split('/')[-4]}::nagios::helpers"
done
helpers
end
end

Modeling

Be Dynamic

<%- if @comment -%>
### <%= @comment %>
<%- end -%>
<%= @type %> <% if @label %><%=
@label %><% end %> {
<%- @config.sort.reverse.each do |
key,value| -%>
<%- if value.class == Array -%>
<%- value.each do |element| -
%>
<%= key %>(<%= element %>);
<%- end -%>
<%- else -%>
<%= key %>(<%= value %>);
<%- end -%>
<%- end -%>
};

Fencing Resources

if $::enable_pager {
@@file { "/etc/nagios/objects/${type}.d/${filename}.cfg":
content => template('nagios/etc/nagios/object.erb'),
mode => '0444',
group => 'nagios',
notify => Service['nagios'],
}
}

It’s About

Missing?
What’s

Language

Predictive
Analysis

Coming
Home

Systems
Thinking
Recap

There is SystemOne
Machine
Parsable
Must be
a
Know
n
Quantity
Self
Authoritati
ve
Must self-correctFriday, August 23, 13

jfryman
fryman@github.com

what’s next?

Building Data-Driven Infrastructure with Puppet - PuppetConf 2013

Recommandé

Recommandé

Contenu connexe

Plus de Puppet

Plus de Puppet (20)

Dernier

Dernier (20)

Building Data-Driven Infrastructure with Puppet - PuppetConf 2013