Building a Cauldron for Chef to Cook In

Building a Cauldron for Chef to
Cook In
Jonathan Altman
@async_io
For DevOpsDC August 2013
Wednesday, August 14, 2013

Background
• 12 years architecting at a leading SaaS software provider
• Started 2 companies in August 2012
• One is a SaaS/product based company
• One is a services ﬁrm
• Customers have access to their source
• No time to waste: automation good

What is a Cauldron?
• A complete (as you want) code/artifact management and build/deployment system
• Chef server
• Git server
• Automated build system e.g. Jenkins (eventually)
• JIRA server?
• Group chat server? (IRC or XMPP)
• With a real cert

But SaaS is Awesome!
• Yes. Yes it is.
• Github
• Hosted Chef
• ShiningPanda, CloudBees, JenkinsHosting, travis-ci.com
• Jira OnDemand
• HipChat (or https://grove.io/ for hosted IRC, or Google+ hangouts)

The 3 R’s
• Redundancy
• Resiliency
• Revelation

Redundancy
• Does a good backup strategy ever keep just one copy of something?
• Disaster Recovery: how do you recover with your backups?
• At least for git, chef, and CI you can use multiple servers
• git “just works” in this model
• The knife plugin has backup/restore capability for example
• Not sure how to integrate with Opscode-hosted chef

Resiliency
• Tradeoff between your ability to deliver uptime and their motivation
to address their issues that affect your uptime
• Honeypot: tradeoff again. github and hosted chef server are well
protected, but a big, attractive attack vector. Your server, not so
much; but is it secure?
• Ability to assign the amount of compute resources you need to
deliver the performance, uptime, and redundancy you want

Revelation
• You will have the tools and ability to help yourself
• Git != Github, for example

A Cauldron: workstation
• Client workstation with
• omnibus installer (http://www.opscode.com/chef/install/)
• knife-server plugin (http://fnichol.github.io/knife-server/)
• git client software installed on it
• You could do this on your workstation’s host OS, but using aVM
simpliﬁes/isolates having multiple cauldrons

Create Cauldron Workstation
Cocytus:vagrant_servers jonathan$ mkdir cauldron_ws
Cocytus:vagrant_servers jonathan$ cd cauldron_ws
Cocytus:cauldron_ws jonathan$ vagrant init
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
Cocytus:cauldron_ws jonathan$

Bootstrap theVagrant
Cocytus:cauldron_ws jonathan$ vi Vagrantfile # Set up your config here
Cocytus:cauldron_ws jonathan$ vagrant up
[default] Box base was not found. Fetching box from specified URL...
[vagrant] Downloading with Vagrant::Downloaders::HTTP...
[vagrant] Extracting box...
[vagrant] Verifying box...
[vagrant] Cleaning up downloaded box...
[default] Importing base box 'base'...
{bunch of lines deleted ...}
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
VM must be created before running this command. Run `vagrant up` first.
Cocytus:cauldron_ws jonathan$

Install chef omnibus
Cocytus:cauldron_ws jonathan$ vagrant ssh
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
* Documentation: https://help.ubuntu.com/
Welcome to your Vagrant-built virtual machine.
Last login: Fri Sep 14 06:23:18 2012 from 10.0.2.2
vagrant@precise64:~$ sudo apt-get install build-essential curl
[...bunch of output deleted]
vagrant@precise64:~$ curl -L https://www.opscode.com/chef/install.sh | sudo bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6790 100 6790 0 0 22442 0 --:--:-- --:--:-- --:--:-- 36902
Downloading Chef for ubuntu...
Installing Chef
Selecting previously unselected package chef.
(Reading database ... 51127 files and directories currently installed.)
Unpacking chef (from .../tmp.8PmNsIUQ/chef__amd64.deb) ...
Setting up chef (11.6.0-1.ubuntu.12.04) ...
Thank you for installing Chef!
vagrant@precise64:~$

Knife-server plugin
• knife plugin to create chef servers on the command line running on
ubuntu:
• on ec2 (requires the knife-ec2 plugin as well)
• on linode (requires the knife-linode plugin)
• on an already-running instance (“standalone”)

Install knife-server Plugin
vagrant@precise64:~$ sudo /opt/chef/embedded/bin/gem install knife-server
--no-ri --no-rdoc
Building native extensions. This could take a while...
Fetching: ruby-hmac-0.4.0.gem (100%)
Fetching: fog-1.14.0.gem (100%)
Fetching: knife-server-1.1.0.gem (100%)
Successfully installed nokogiri-1.6.0
Successfully installed ruby-hmac-0.4.0
Successfully installed fog-1.14.0
Successfully installed knife-server-1.1.0
4 gems installed
vagrant@precise64:~$

Make a Server (Standalone)
• Standalone there is a bunch of other stuff to get right:
• Better if the server you’re installing on has root with ssh
authorized_keys set up to your workstation’s ssh private key
• Need both private and public part of key on workstation because of
http://tickets.opscode.com/browse/CHEF-4180

Server Bootstrapping...
knife server bootstrap standalone --node-name cauldron.your.domain --host
xxx.yyy.zzz.aaa
[hundreds of lines of output deleted ...]
192.241.179.65 Recipe: chef-server::erchef
192.241.179.65 * service[erchef] action restart
192.241.179.65
192.241.179.65 - restart service service[erchef]
192.241.179.65
192.241.179.65
192.241.179.65 Chef Client finished, 244 resources updated
192.241.179.65 chef-server Reconfigured!
192.241.179.65 Server reconfigured
192.241.179.65 -----> Bootstrapping Chef Server on cauldron.async.io is
complete.
If you want the web UI enabled, include --web-ui-enable

Voila?

uhh...Voila!

IfYou Used --web-ui-enable
BTW, login right now and change the password! Or: turn off web-ui

git server
• Several (sane) choices for hosting your own git:
• Gitolite: http://gitolite.com/gitolite/
• Gitlab: http://gitlab.org/
• Gitblit: https://code.google.com/p/gitblit/
• gitweb: https://git.wiki.kernel.org/index.php/Gitweb (please don’t)
• On windows or OSX, hosted github ($$)
• We are going to use gitolite

Gitolite
• Gitlab is cool, tries to reproduce as much of github other web-based
git hosting as possible. Installation? Several pages of hand-invoking
• Gitblit: same goal as gitlab, but built in java. So: easy install, but big and
not using the ofﬁcial git binaries so compatibility?
• Gitosis is dead if you run across it
• Gitweb: just don’t

Install
• Put the id_rsa.pub key of the user you want to admin gitolite as on the
cauldron server, but name it username.pub where username is the
username you want to be known as on the gitolite server
# get the software
git clone git://github.com/sitaramc/gitolite
# install it
gitolite/install -ln
# setup the initial repos with your key
gitolite setup -pk your-name.pub

Conﬁgure gitolite
• Back on your workstation machine:
git clone git@host:gitolite-admin.git
• Add your git repository setups in the cloned conf/gitolite.conf ﬁle
• Example, jonathan is the owner of the heatNode repository (and there
is a jonathan.pub RSA public key):
repo heatNode
RW+ = jonathan

You have a cauldron
• You can git add remote your cauldron plus any other git servers such as
github to any git repository you have
git remote add origin git@cauldron.your.domain:name_of_your_repo.git
git remote add github git@github.com:name_of_your_repo.git
• The name after git remote add is arbitrary! “origin” is a convention but not
required
• You git push/pull from all external servers so you have redundant copies
• Your cauldron is now a chef server

Put a real cert on the box
• As of omnibus Chef 11, webserver is nginx. Edit the ssl conﬁg to put a
real server on to get rid of the big red untrusted cert warning
• Check out http://chr4.org/blog/2013/08/01/howto-use-chef-with-ssl/
for steps on how to do it with chef
• Probably a good idea, as chef-ctl-reconﬁgure might blast manual
changes

Thank you. Questions?
Also, thanks to @nathenharvey and @devopsdc for letting me present, @devopsdc
and @fnichol (Fletcher Nichol) for the awesome real-time interactive improvements
to my presentation, and @fnichol for the awesome knife-server plugin!

Building a Cauldron for Chef to Cook In

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (7)

Similaire à Building a Cauldron for Chef to Cook In

Similaire à Building a Cauldron for Chef to Cook In (20)

Dernier

Dernier (20)

Building a Cauldron for Chef to Cook In