5. 3 . 24
CHEF VS PUPPET/ANSIBLE
PUPPET
Language : DSL Ruby/Json
Approach : Execution by dependency and
chained action
Stored data : YAML
Agent : Yes
ANSIBLE
Language : Python
Approach : Hierarchical Execution
Stored data : YAML
Agent : No
CHEF
Language : Full Ruby & DSL
Approach : Hierarchical Execution
Stored data : JSON
Agent : Yes
11. 7
CHEF VS PUPPET/ANSIBLE
Advantages Disadvantages
Chef Fast & powerful
Dev Oriented => Flexible
Full Ruby
Search
Crypted data
Flexibility -> Complexity
Puppet Mature
Large Community
Lot of tools
Slow
Complex Language
Complex execution order
Ansible Agentless
« No code »
Multilingual plugins
Immaturity: Small community and few tools
Data stored in les
12. 8 . 1
CHEF SERVER
Free/Basic Version
Free & no node limit
In your Infrastructure
› No access to "premium" features
14. 8 . 3
CHEF AUTOMATE
Deliver a continuous deployment pipeline for infrastructure and applications.
Gain insight into operational, compliance, and work ow events.
Identify compliance issues, security risks, and outdated software with customizable
reports
15. 9
NOT ONLY CHEF SERVER...
Chef Solo
No Server
Deploy & Run recipes directly on the node
› No search & dedicated attributes on a node
› Need to push cookbooks on each nodes
› Not compatible with version constraint
Chef Zero
Chef Server instance in memory
Chef Local mode (-z)
embedded chef-zero (faster)
17. 11
KNIFE – CHEF'S SWISS KNIFE
Management of chef environment
Search, SSH (Executing commands in parallel)
Plugins (VMware, solo, spork...)
18. 12
CLIENTS AND NODES
A client is a registered machine with the server
A Node is a client that executes one or more recipe
→ A node is a client but a client is not necessarily a node
20. 14
THE COOKBOOKS – WHAT WE WILL COOK TODAY ?
Create a cookbook
Structure
Metadata
Recipes
Attributes
Files & Templates
Resources & Providers
LWRP & HWRP
De nitions
Library
21. 15
THE COOKBOOKS - CREATE
Three options to create a cookbook:
Create your own
Get a cookbook from Chef Supermarket
Get a cookbook from Github (or other)
knife cookbook create nginx
chef generate cookbook nginx
knife cookbook site install nginx
26. 19 . 2
THE COOKBOOKS - ATTRIBUTES
case node['platform']
when "debian"
case
when node['platform_version'].to_f < 6.0 # All 5.X
default['postgresql']['version'] = "8.3"
when node['platform_version'].to_f < 7.0 # All 6.X
default['postgresql']['version'] = "8.4"
else
default['postgresql']['version'] = "9.1"
end
default['postgresql']['client']['packages'] = ["postgresql-client-#{node['postgresql']['version']}","libpq-dev"]
default['postgresql']['server']['packages'] = ["postgresql-#{node['postgresql']['version']}"]
default['postgresql']['contrib']['packages'] = ["postgresql-contrib-#{node['postgresql']['version']}"]
when "fedora"
...
30. 20 . 2
THE COOKBOOKS - TEMPLATES
› memcached/templates/default/memcached.conf.erb
# Run memcached as a daemon. This command is implied, and is not needed for the
# daemon to run. See the README.Debian that comes with this package for more
# information.
-d
# Log memcached's output to /var/log/memcached
logfile /var/log/<%= @logfilename %>
# Be verbose
#-v
# Be even more verbose (print client commands as well)
# -vv
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m <%= @memory %>
# Default connection port is 11211
-p <%= @port %>
#-U <%= @udp_port %>
# Run the daemon as root. The start-memcached will default to running as root if no
# -u command is present in this config file
-u <%= @user %>
# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
31. 21 . 1
THE COOKBOOKS - FILES
› cookbook/app/ les/default/application-5.8.3.pm
› cookbook/app/ les/default/application-5.20.1.pm
cookbook_file "application.pm" do
path case node['platform']
when "centos","redhat"
"/usr/lib/version/1.2.3/dir/application.pm"
when "arch"
"/usr/share/version/core_version/dir/application.pm"
else
"/etc/version/dir/application.pm"
end
source "application-#{node['languages']['perl']['version']}.pm"
owner 'root'
group 'root'
mode '0644'
end
33. 22
THE COOKBOOKS – RESOURCES & PROVIDERS (LWRP / HWRP)
$ tree cookbooks/mdm/
cookbooks/mdm/
├── CHANGELOG.md
├── README.md
├── attributes
├── definitions
├── files
│ └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│ └── default.rb
├── resources
└── templates
└── default
Resources : Used to de ne a set of actions and attributes
Providers : Used to say to chef-client what to do foreach
de ned actions
user 'random' do
supports :manage_home => true
comment 'Random User'
uid 1234
gid 'users'
home '/home/random'
shell '/bin/bash'
password '$1$JJsvHslV$szsCjVEroftprNn4JHtDi'
end
38. 24
THE COOKBOOKS – RESOURCES & PROVIDERS
(HWRP)
Create a resource with pure Ruby
Bypass the limits of Chef DSL
http://tech.yipit.com/2013/05/09/advanced-chef-writing-
heavy-weight-resource-providers-hwrp/
39. 25 . 1
THE COOKBOOKS - DEFINITIONS
Used to de ne a set of actions with or without parameters
(function)
You can call your de nition many times in one or more
recipes
This is the same as a resource (LWRP) except that you may
not notify (trigger) other resources
40. 25 . 2
THE COOKBOOKS - DEFINITIONS
define :host_porter, :port => 4000, :hostname => nil do
params[:hostname] ||= params[:name]
directory "/etc/#{params[:hostname]}" do
recursive true
end
file "/etc/#{params[:hostname]}/#{params[:port]}" do
content "some content"
end
end
host_porter node['hostname'] do
port 4000
end
host_porter "www1" do
port 4001
end
42. 26 . 2
CUSTOM RESOURCES
From Chef version 12.5
De nitions are useless. Advise you to use a resources instead.
Custom resources is a provider redesigned to be simpler.
Located only in "resources" directory
43. 26 . 3
CUSTOM RESOURCES
exampleco/resources/site.rb
property :homepage, String, default: '<h1>Hello world!</h1>'
load_current_value do
if ::File.exist?('/var/www/html/index.html')
homepage IO.read('/var/www/html/index.html')
end
end
action :create do
package 'httpd'
service 'httpd' do
action [:enable, :start]
end
file '/var/www/html/index.html' do
content homepage
end
end
action :delete do
package 'httpd' do
action :delete
end
end
exampleco_site 'httpd' do
homepage '<h1>Welcome to the Example Co. website!</h1>'
action :create
end
44. 27 . 1
THE COOKBOOKS - LIBRARIES
Allows to extend the Chef's classes or create your own Ruby lib
Do what Chef does not already do
Do external data processing to use in Chef
49. POLICYFILE
Policies are a new feature of Chef that combine the very best
parts of Roles, Environments and cookbook dependency
resolvers (Berkshelf) into a single easy to use work ow.
It is associated with a group of nodes, cookbooks, and
settings. When these nodes run, they run the recipes
speci ed in the Policy le run-list
Resolves real-world problems of team work ow
50. 30 . 130 . 2
POLICYFILE
De ne in Policy le:
› Run list
› Cookbook dependencies with version constraint and source
› Attributes overriding
name "jenkins-master"
run_list "java", "jenkins::master", "recipe[policyfile_demo]"
default_source :supermarket, "https://mysupermarket.example"
cookbook "policyfile_demo", path: "cookbooks/policyfile_demo"
cookbook "jenkins", "~> 2.1"
cookbook "mysql", github: "chef-cookbooks/mysql", branch: "master"
default['java']['version'] = '8'
51. 30 . 3
POLICYFILE
Create your policy le into your cookbook
$ cd cookbooks/app
$ chef generate policyfile
$ ls -l
-rw-r--r-- 1 mlopez wheel 596 12 oct 21:57 Policyfile.rb
Create your policy le into policies directory
$ ls -l
total 24
-rw-r--r-- 1 mlopez wheel 70 12 oct 22:16 LICENSE
-rw-r--r-- 1 mlopez wheel 1546 12 oct 22:16 README.md
-rw-r--r-- 1 mlopez wheel 1067 12 oct 22:16 chefignore
drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 cookbooks
drwxr-xr-x 4 mlopez wheel 136 12 oct 22:16 data_bags
drwxr-xr-x 4 mlopez wheel 136 12 oct 23:13 policies
$ chef generate policyfile policies/web
52. 30 . 4
Create the lock le
$ chef install [path/to/policyfile.rb]
Update lock le after modi cation
$ chef update [path/to/policyfile.rb]
Upload cookbook and policy le
$ chef push POLICY_GROUP PATH/TO/POLICYFILE.rb
→ If POLICY_GROUP doesn't exists it will be created
Assign policy name and policy group into client.rb's node or in
node con guration
53. 31 . 1
DATA BAGS
Stock & share data
Read the desired data from a recipe or from your desktop (knife)
Write data collected during a run
$ knife data bag create DATA_BAG_NAME [ITEM]
knife data bag create users toto
knife data bag from file users data_bags/users/toto.json
54. 31 . 2
DATA BAGS
› knife data bag create users myUser
{
"id": "myUser",
"ssh_keys": [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDakMx4mjWYMko2r085yq/vq0Ey2DiVWXeJ
],
"groups": [
"mdm_qa"
],
"uid": 4001,
"shell": "/bin/bash",
"comment": "My User",
"action": "remove"
}
› knife data bag show users myUser –Fj > data_bags/users/myUser.json
55. 31 . 3
DATA BAGS
admins = []
search(:admins, "*:*").each do |admin|
login = admin[‘id’]
admins << login
home = "/home/#{login}"
user login do
uid admin['uid']
gid admin['gid']
shell admin['shell']
home home
comment admin['comment’]
supports :manage_home => true
end
end
56. 31 . 4
Write in a data bag
DATA BAGS
sam = {
"id" => "sam",
"Full Name" => "Sammy",
"shell" => "/bin/zsh"
}
databag_item = Chef::DataBagItem.new
databag_item.data_bag("users")
databag_item.raw_data = sam
databag_item.save
sam = data_bag_item("users", "sam")
sam["Full Name"] = "Samantha"
sam.save
57. 31 . 5
What should I do if I have sensitive data ?
DATA BAGS
→ Encrypt !!
$ knife data bag create passwords postgresql –secret-file <path>/encrypted_data_bag_secret</path>
$ knife data bag show passwords postgresql -Fj
{
"id": "postgresql",
"password": {
"encrypted_data": "nu0GFIaJuzefK1iCgmYxWbRO64tvEezZJA/7iOUT87NLg=n",
"iv": "LWK$u1omaWHHNfzfDcYN45g==n",
"version": 1,
"cipher": "aes-256-cbc"
},
"databases": {
"encrypted_data": "lG4EULs9UQKKwjfzef8/WrccoGilQO2m7O6JNnIeMu199jGIT2l+/MvR+bX6dnk2U/_dwn
"iv": "UmtXyR9m0ornADWbiayPyw==n",
"version": 1,
"cipher": "aes-256-cbc"
}
}
58. 32 . 1
CHEF VAULT
Whithout Chef Vault, nodes share a shared secret key le
→ Not good for security
With Chef Vault, nodes and workstation use their own
keypair to decrypt data bag. Chef administrators de ne
which node or admin can access to the encrypted data
59. 32 . 2
or
CHEF VAULT
gem install chef-vault
Create a Vault
Allow mlopez, vaubert and nodes with web role assigned to decrypt content
$ knife vault create credentials database -A mlopez,vaubert -M client -S ‘roles:web'
-J '{“db_password”:”some_password”}'
$ knife encrypt create credentials database --json '{“db_password”:”some_password”}'
--search 'role:web'
--admins mlopez, vaubert --mode client
60. 32 . 3
or
CHEF VAULT
From an Unauthorized admin's workstation
$ knife vault show credentials database
db_password:
cipher: aes-256-cbc
encrypted_data: dsiBtADAV8Sbis89yKuYBvbdNXPpu8bQfJrS20op7zoysfR8roFlzpVHyoaG2
4yb3
iv: +0siNLzFHHqEkP07k6JhYw==
version: 1
id: database
Authorized users see the decrypted content
$ knife vault show credentials database
$ knife decrypt credentials database --mode client
db_password: some_password
id: database
61. 32 . 4
CHEF VAULT
Content of database_keys
$ knife data bag show credentials database_keys
admins:
mlopez
vaubert
clients:
web-01
web-02
id: database_test_keys
mlopez: SOME KEY
vaubert: SOME KEY
web-01: SOME KEY
web-02: SOME KEY
62. 32 . 5
CHEF VAULT
Add a new admin workstation
$ knife vault update credentials database -A mhue
Rotate all keys
$ knife vault rotate all keys
63. 32 . 6
CHEF VAULT
Use Vault in recipe
chef_gem 'chef-vault' do
compile_time true if respond_to?(:compile_time)
end
require 'chef-vault'
# Or just include chef_vault coobkook
include_recipe 'chef-vault'
case ChefVault::Item.data_bag_item_type('credentials', 'database')
when :normal
...
when :encrypted
...
when :vault
...
end
64. 33 . 1
OHAI
Ohai is a tool that is used to detect attributes on a node, inventory the
system (platform, cpu, memory...), and then provide automatically these
attributes to the chef-client at the start of every chef-client run.
Check Automatic attributes previously seen. You can list node attributes
by running "ohai" command
$ ohai
{
"hostname": "node-01",
"machinename": "node-01",
"fqdn": "node-01.my.local",
"domain": "my.local",
"network": {
"interfaces": {
"lo": {
"encapsulation": "Loopback",
"addresses": {
"127.0.0.1": {
"family": "inet",
"prefixlen": "8",
"netmask": "255.0.0.0",
...
65. 33 . 2
OHAI - CUSTOM PLUGIN
You can create your own Ohai plugin to collect data before
the run and set them into Ohai as an attribute
Use Ohai cookbook and put your plugin into ' les' directory
› cookbooks/ohai/ les/default/plugins/haproxy.rb
# Encoding: utf-8
# Get current version of Haproxy
Ohai.plugin(:Haproxy) do
provides 'haproxy'
collect_data(:linux) do
haproxy Mash.new
[['dpkg-query -W haproxy | awk '{print $2}' | sed 's/(^[1-9].[1-9]).*/1/'',
:installed_version]].each do |cmd, property|
so = shell_out(cmd)
haproxy[property] = so.stdout.delete("n")
end
end
end
66. 34
TEAM WORKING
Git -> Create branches
Test locally before uploading his cookbook
Vagrant / Virtualbox, Kitchen CI...etc.
Chef solo/zero/local
Bump version of your cookbook and upload it into environment
Freeze uploaded version and apply version constraint into environnements
Knife Spork – Plugin allow you to bump, upload and promote a cookbook more easily.
Noti cation plugin, auto git add …
$ knife cookbook upload nginx [--freeze] [--force] [-E <environment>]
$ knife spork omni nginx –l minor –e qa_group</environment>
67. 35 . 1
TEST HIS JOB
Syntax
Logical tests with Foodcritic
Obsolescence of resources used, invalid search queries, syntax best-
practice…
Unit tests with ChefSpec
$ knife cookbook test nginx
package 'foo'
require 'chefspec'
describe 'example::default' do
let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }
it 'installs foo' do
expect(chef_run).to install_package('foo')
end
end
68. 35 . 2
TEST HIS JOB
Integration tests with Kitchen CI
Check if a run runs without errors (converge)
Include unit tests (Rspec, Bats…)
Run many tests suite in one time (client / server) on many platforms
Check which nodes use your cookbook
Simulate an execution of a run
knife preflight web::ws
knife search node -i "recipes:web::ws »
chef-client --why-run
knife ssh ‘name:srv-01.pp’ ‘sudo chef-client –W’
69. 36
DEBUG – WHY IT DOESN'T WORK ?
chef-client –l debug
Generate logs
Chef::Log.debug(« Doesn’t work »)
puts myVariable
Raise Exceptions
Chef::Log.fatal!('Deployment failure...')
raise
70. 37 . 1
ADVANCED
Override a run-list (-o "recipe[]")
Override a community Cookbook
knife ssh 'name:srv-01.dev' 'sudo chef-client –o "recipe[firefox]"'
include_recipe 'nginx'
resources("template[/etc/nginx/nginx.conf]").cookbook 'myNginx')
