Data Driven Infrastructure

Data

Driven
Infrastructure

Friday, August 23, 13

?

http://git.io/UmUdKA

Let me
Tell you
a story

What is a

Cloud


With the Cloud...

I don’t have to think about IT

Software
Platform
Infrastructure

aaS

Where is
our cloud?


James Fryman


Auto
matio
ns
Junk
ie


Are you

Crazy?

What
Data? Goals?

ne
hi
ac
M
le
ab
rs
Pa


There is


One System

gPanel

PuppetDB

Provisioning

Configuring

Destroying

Feedback

Parts
of the
System

provisioner

Home
Grown

Home Grown


controller

Lots O’ Models


file { '/etc/facter/facts.d/quagga_manage_service':
ensure => present,
replace => false,
}

if $::quagga_manage_service {
$service_state[ensure] = running
$service_state[enable] = true
} else {
$service_state[ensure] = undef
$service_state[enable] = undef
}

service { 'quagga':
ensure => $service_state[ensure],
enable => $service_state[enable],
}

orchestrator

Chat
Ops

Feedback
of the
System

&

Metrics
Monitoring

begin
JSON.parse(RestClient.get(url)).each do |cache|
data["#{cache['target']}"] = 0
count = 0
cache["datapoints"].each do |point|
unless (point[0].nil?)
data["#{cache['target']}"] += point[0]
count += 1
end
end
data["#{cache['target']}"] /= count
data["total"] += data["#{cache['target']}"]
end
rescue ZeroDivisionError => e
puts "UNKNOWN metric not in graphite!"
exit EXIT_UNKNOWN
rescue => e
puts e.message
exit EXIT_CRITICAL
end


event
log


event
log


alert

event
log


alert

error

Must self-correct


Deployable
using text files


larity
odu
M


haproxy::proxy
proxy
=>
mode
=>
ip
=>
port
=>
config
=>
balance =>
},
}


{ $es_proxy_name:
'listen',
'http',
$::ipaddress_lo,
'9200',
{
'roundrobin',

haproxy::proxy::member { $es_proxy_name:
hostname => $::ec2_local_ipv4,
port
=> '9200',
param
=> [
'weight 1',
'maxconn 1000',
'check',
],
}


collectd::plugin { [
'cpu',
'load',
'memory',
'swap',
'irq',
'exec',
'entropy',
]:
}


collectd::plugin { [
'df',
'interface',
'protocols',
'disk',
]:
config => true
}

Level 4:
Templates


Level 5:
Data Driven

Self
Authoritative


def self.collect_resources(resource)
query = [
"and",
["=", "type", "File"],
["=", "exported", true],
["=", "tag", "nagios::object::#{resource}"],
["=", ["node", "active"], true]
]
self.search('/v2/resources', query.to_json)
end


# Seed the initial file with false, and enable
# on next run with gh-nagios enable_pager <hostname>
file { '/etc/facter/facts.d/enable_pager.txt':
ensure => file,
owner
=> 'root',
group
=> 'root',
mode
=> '0644',
content => 'enable_pager=false',
replace => false,
}


}

@@file {
content
mode
group
notify


"/etc/nagios/objects/${type}.d/${filename}.cfg":
=> template('nagios/etc/nagios/object.erb'),
=> '0444',
=> 'nagios',
=> Service['nagios'],

ng
ri
to

ac
ef
R


least to most

specific

frymanet.com

nginx

common

rails

ruby

mysql

admin

package repos

graduate to

params

class ntp::params {
$defaults = {
package => {
version => ‘latest’,
},
config => {
servers => [‘pool.ntp.org’],
},
}
}

externalize

class ntp::params {
$defaults = {
package => {
version =>
hiera(‘ntp_package_version’),
},
config => {
servers => hiera(‘ntp_servers’),
},
}
}


Cloud
Forma

tion


"Resources": {
"RendererServerGroup": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": { "Fn::GetAZs": "" },
"LaunchConfigurationName": { "Ref": "LaunchConfig" },
"MinSize": "2",
"MaxSize": "16",
"Tags": [
{ "Key": "Environment", "Value": "Production",
"PropagateAtLaunch": "true" },
{ "Key": "Role", "Value": "renderer",
"PropagateAtLaunch": "true" }
]
}
},

ding
toloa
Au


# autoloader.pp
class nagios::autoload_helpers {
$helpers = get_nagios_helpers()
case $::puppetversion {
/^3/: { include $helpers }
default: {
nagios::autoload_helpers::import_shim { $helpers: }

}
}
}


module Puppet::Parser::Functions
newfunction(:get_nagios_helpers,
:type => :rvalue,
:doc => "Grab all modules that have nagios helpers for
import") do
module_path = File.expand_path('..',
Puppet::Module.find('nagios',compiler.environment.to_s).path)
helpers = Dir["#{module_path}/**/nagios/helpers.pp"].map do |d|
"#{d.split('/')[-4]}::nagios::helpers"
done
helpers
end
end

Modeling


Be Dynamic

<%- if @comment -%>
### <%= @comment %>
<%- end -%>
<%= @type %> <% if @label %><%=
@label %><% end %> {
<%- @config.sort.reverse.each do |
key,value| -%>
<%- if value.class == Array -%>
<%- value.each do |element| %>
<%= key %>(<%= element %>);
<%- end -%>
<%- else -%>
<%= key %>(<%= value %>);
<%- end -%>
<%- end -%>
};

ing
nc
e

F


so
Re

es
rc
u

if $::enable_pager {
@@file { "/etc/nagios/objects/${type}.d/${filename}.cfg":
content => template('nagios/etc/nagios/object.erb'),
mode
=> '0444',
group
=> 'nagios',
notify => Service['nagios'],
}
}


Buy it?


It’s About


What’s
Missing?


Language

Predictive
Analysis

g
in
m
o e
C
m
o
H


p
a
c

e
R

Systems

Thinking


There is
ne
hi
ac
M
le
ab
rs
Pa

One System
Must be a

Self
Authoritative

Known
Quantity

Must self-correct

jfryman
fryman@github.com

what’s next?


Data Driven Infrastructure

Recommandé

Recommandé

Contenu connexe

Plus de Puppet

Plus de Puppet (20)

Dernier

Dernier (20)

Data Driven Infrastructure