SlideShare une entreprise Scribd logo
1  sur  27
DevOps Toolchain:
Vagrant/Chef/AWS
Web http://juanvicenteherrera.eu
Twitter https://twitter.com/jvicenteherrera
LinkedIn http://es.linkedin.com/in/jvherrera
GitHub https://github.com/juanviz
Slideshare
http://www.slideshare.net/juanvicenteherrera
About me
Index
1. Requisites
2. Vagrant Box
3. Vagrantfile
4. Vagrant Settings
5. Vagrant Commands
6. Provisioning with Chef
7. Multi machines
8. Version control
9. Cleanup
10.Conclusions
11.References
Why a common toolchain?
• Consistency
• Efficiency
• Safety
• Automation
Requisites
Knife
https://learnchef.opscode.com/quickstart/workstation-setup/
Vagrant
$ gem install vagrant (ruby needed)
Vagrant plugins
https://github.com/mitchellh/vagrant-aws
$ vagrant plugin install vagrant-aws
$ vagrant plugin install vagrant-omnibus
$ vagrant plugin install vagrant-butcher
AWS credentials
Chef server running and chef client certificates
Quick view
Chef Server
AMI base
Dev/Ops workstation
(Vagrant+knife)
VPC Subnet
App instances
VPC Subnet
Virtual Private Cloud
AWS Cloud
1. Create instances from AMI
base and make chef
bootstrap
2. Apply roles
Vagrant Box
Base Box: The base box is simply a saved hard-disk of a Virtual Machine
created with VirtualBox. It can contain anything but it needs at least :
• Ruby
• VirtualBox guest additions
• Puppet or Chef
Vagrant AWS Example Box
Vagrant providers each require a custom provider-specific box format. These
files compose the contents of a box for the AWS provider.
• README.md
• Vagrantfile
• metadata.json
To turn this into a box:
$ tar cvzf aws.box ./metadata.json ./Vagrantfile
Vagrantfile EC2
Vagrant.configure("2") do |config| # Version 2 Vagrant syntax
config.vm.box = "dummy"
config.omnibus.chef_version = :latest
config.vm.provider :aws do |aws, override|
aws.keypair_name = "bq”
aws.access_key_id = ”xxxxxxxxx"
aws.secret_access_key = ”xxxxxxxxxxxxxxxxxx"
aws.ami = "ami-3ad1af53”
override.ssh.username = "ec2-user”
override.ssh.private_key_path = "/Users/juanvi/keypairs/bq.pem"
end
end
Vagrant Settings
Creating Vagrant File
mkdir [project name]
cd [project name]
Vagrant init -> creates Vagrantfile
Vagrant ec2 test 2 ec2 without chef management
$ vagrant plugin install vagrant-aws
$ vagrant box add dummy https://github.com/mitchellh/vagrant-
aws/raw/master/dummy.box
$ vagrant up --provider=aws
This box works by using Vagrant's built-in Vagrantfile merging to setup
defaults for AWS. These defaults can easily be overwritten by higher-
level Vagrantfiles (such as project root Vagrantfiles).
Vagrant Commands
vagrant box # Commands to manage system boxes
vagrant destroy # Destroy the environment, deleting the created virtual
machines
vagrant halt # Halt the running VMs in the environment
vagrant help [TASK] # Describe available tasks or one specific task
vagrant init [box_name] [box_url] # Initializes the current folder for Vagrant usage
vagrant package # Package a Vagrant environment for distribution
vagrant provision # Rerun the provisioning scripts on a running VM
vagrant reload # Reload the environment, halting it then restarting it.
vagrant resume # Resume a suspended Vagrant environment.
vagrant ssh # SSH into the currently running Vagrant environment.
vagrant ssh_config # outputs .ssh/config valid syntax for connecting to this
environment via ssh
vagrant status # Shows the status of the current Vagrant environment.
vagrant suspend # Suspend a running Vagrant environment.
vagrant up # Creates the Vagrant environment
vagrant version # Prints the Vagrant version information
Provisioning with Chef
web.vm.provision :chef_client do |chef|
chef.chef_server_url =
"https://api.opscode.com/organizations/juanvi"
chef.validation_key_path = "/Users/juanvi/chef-
repo/.chef/juanvi-validator-new.pem"
chef.validation_client_name = "juanvi-validator"
# Provision with the database role
chef.add_role("webserver")
# Set the environment for the chef server
chef.environment = ”prod"
Multi Machines
Vagrant.configure("2") do |config|
config.vm.define :web do |web|
web.vm.box = "apache"
end
web.vm.provision :chef_client do |chef|
chef.chef_server_url = chef_server_url
chef.validation_key_path = validation_key_path
chef.validation_client_name = validation_client_name
…….
end
config.vm.define :db do |db|
db.vm.box = "mysql"
end
db.vm.provision :chef_client do |chef|
chef.chef_server_url = chef_server_url
chef.validation_key_path = validation_key_path
chef.validation_client_name = validation_client_name
…
end
end
Provisioning with Chef
Provisioning only one kind of server executing
chef_client
$ vagrant provision web --provision-with
chef_client
Provisioning the whole platform executing chef_client
$ vagrant provision --provision-with
chef_client
After editing the Vagrantfile you need to 'reboot' the
machine to take this settings
$ vagrant reload
Version control with git
Version control
Now is a good time to version control our awesome-
vagrant project
$ cd
$ git init
$ git add Vagrantfile
$ git commit -m "This was just my first commit"
Cleanup
$ vagrant plugin install vagrant-butcher
then you can terminate a instance and deregister in the Chef
server executing:
$ vagrant destroy -f
[Butcher] knife.rb location set to
'/path/to/knife.rb'
[Butcher] Chef node 'node_name' successfully
butchered from the server...
[Butcher] Chef client 'node_name' successfully
butchered from the server...
[default] Forcing shutdown of VM...
[default] Destroying VM and associated drives...
• The task of writing recipes is not picked up by all team
members, and seem to stay the main job of the system
oriented people on the team.
• Reading Chef Cookbooks help people understand
what is needed and makes it easy to point out what
needs to be changed
Conclusions
Conclusions
• Before committing cookbooks-> destroy 'development'
box and re-provision a new box to see if this works.
(immutable servers)
• The longer the provision takes, the less frequent
people do it. It's important to keep that process as
fast as possible.
Conclusions
• Installation problems would get noticed far
sooner in the process.
• People would only do a full rebuild in the
morning when getting their coffee.
Conclusions
• It clearly helps everybody to have a consistent
environment to develop against, the latest version is just
one git pull away.
• The central approach drives people to
• do frequent commits
• do stable commits.
Conclusions
• Vagrant is an essential part of the DevOps process:
it is the solution to developing, testing and deploying
in the same environment. It thus ensures a smoother
transition of your project from the dev team to the ops
team.
• Chef has to be the only way to manage the
configuration; NO MORE MANUAL CHANGES
Hands On
A typical situation in a company that makes software is the
arrival of a new developer that has the tough task to install
a lot of tools in his workstation that in the best case takes
days.
We are going to create a whole developer stack in AWS
not for this new developer but for a complete new
developer team!
After make magic with Vagrant and Chef the unique task
for all of the members of the team to begin to work with the
environment is clone the git repository, receive the chef
and EC2 certificates/keypairs from their project leader.
Hands On
1. Download project (private repository)
2. Configure ~/.chef/aws_credentials.rb with your AWS security credentials
3. Check that you have in ~/.chef the chef and aws credentials and keypairs.
4. Well you are ready to create the whole demo environment!
5. $ gem install vagrant
6. $ vagrant plugin install vagrant-aws
7. $ vagrant plugin install vagrant-omnibus
8. $ environment=prod vagrant up --provider=aws
9. Tweak in the /etc/hosts url wiki.juanvicenteherrera.es pointing
to elastic ip assigned to ws.(only for test, in prod env change
dns entry or proxywebserver virtualhost)
10. Check the url http://wiki.juanvicenteherrera.es
11. Congrats! IT’S MAGIC! You have got your first complete environment created with
Vagrant and Chef!
Notes: if you don’t want to import a initial db dump…
1. This step can be avoid changing node attribute db_load to 0 in
environments/prod.json.
Create a new demo environment called demo
1. $ cp environments/prod.json environments/demo.json
2. Change id env and ips
3. $ knife environment from file environments/demo.json
4. Change values for all of the attributes with the appropiate demo values
5. $ vi Vagrantfile -> add the environment in the array of the allowed
environments
6. if (environment_id =~ /^(prod|qa|dev|demo)$/)
Add demo and app db password in encrypted databags for the new
environment
7. $ knife data bag edit --secret-file ~/.chef/encrypted_data_bag_secret secrets
php_app
8. $ knife data bag edit --secret-file ~/.chef/encrypted_data_bag_secret secrets
postgresql
9. $ environment=demo vagrant up --provider=aws
10.Tweak in the /etc/hosts url wiki.juanvicenteherrera.es
pointing to elastic ip assigned to ws.(only for test, in
prod env change dns entry or proxywebserver virtualhost)
11. Check the url http://wiki.juanvicenteherrera.es
Hands On
Video with the execution of the hands on available
at:
https://vimeo.com/76336356
For more information about this hands on check
http://juanvicenteherrera.eu/2013/09/04/immutable-
servers-with-vagrant-and-chef-chapter-i/
Hands On
References
• Video of this presentation recorded in Madrid DevOps’ September meeting
• https://vimeo.com/76334136
• https://vimeo.com/76334134
• Vagrant: Up and Running, Mitchell Hashimoto
http://www.amazon.com/Vagrant-Up-Running-Mitchell-
Hashimoto/dp/1449335837/ref=sr_1_1?ie=UTF8&qid=1375433466&sr=8-
1&keywords=vagrant+up
• http://juanvicenteherrera.eu/2013/09/04/immutable-servers-with-vagrant-
and-chef-chapter-i/
• https://github.com/mitchellh/vagrant-aws
• http://docs.vagrantup.com/v2/cli/index.html
• http://red-badger.com/blog/2013/02/21/automating-your-infrastructure-with-
vagrant-chef-from-development-to-the-cloud/
• http://docs-v1.vagrantup.com/v1/docs/provisioners/chef_server.html
Immutable servers provided by Vagrant/Chef/AWS
irc channel #madrid-devops.
Web https://sites.google.com/site/madridevops/
Twitter https://twitter.com/MadridDevops
Grupo de Google
https://groups.google.com/d/forum/madriddevops
GitHub https://github.com/madrid-devops
Group information

Contenu connexe

Plus de Juan Vicente Herrera Ruiz de Alejo

AWS migration: getting to Data Center heaven with AWS and Chef
AWS migration: getting to Data Center heaven with AWS and ChefAWS migration: getting to Data Center heaven with AWS and Chef
AWS migration: getting to Data Center heaven with AWS and ChefJuan Vicente Herrera Ruiz de Alejo
 

Plus de Juan Vicente Herrera Ruiz de Alejo (20)

OpenShift Multicluster
OpenShift MulticlusterOpenShift Multicluster
OpenShift Multicluster
 
Deploying Minecraft with Ansible
Deploying Minecraft with AnsibleDeploying Minecraft with Ansible
Deploying Minecraft with Ansible
 
Tell me how you provision and I'll tell you how you are
Tell me how you provision and I'll tell you how you areTell me how you provision and I'll tell you how you are
Tell me how you provision and I'll tell you how you are
 
Santander DevopsandCloudDays 2021 - Hardening containers.pdf
Santander DevopsandCloudDays 2021 - Hardening containers.pdfSantander DevopsandCloudDays 2021 - Hardening containers.pdf
Santander DevopsandCloudDays 2021 - Hardening containers.pdf
 
X by orange; una telco en la nube
X by orange;   una telco en la nubeX by orange;   una telco en la nube
X by orange; una telco en la nube
 
Dorsal carrera de la mujer ROSAE 2017
Dorsal carrera de la mujer ROSAE 2017 Dorsal carrera de la mujer ROSAE 2017
Dorsal carrera de la mujer ROSAE 2017
 
Cartel carrera de la mujer ROSAE 2017
Cartel carrera de la mujer  ROSAE 2017Cartel carrera de la mujer  ROSAE 2017
Cartel carrera de la mujer ROSAE 2017
 
Volkswagen Prague Marathon 2017
Volkswagen Prague Marathon 2017Volkswagen Prague Marathon 2017
Volkswagen Prague Marathon 2017
 
Plan de entrenamiento Maratón de Madrid Mes 3
Plan de entrenamiento Maratón de Madrid Mes 3Plan de entrenamiento Maratón de Madrid Mes 3
Plan de entrenamiento Maratón de Madrid Mes 3
 
Plan de entrenamiento Maratón de Madrid Mes 2
Plan de entrenamiento Maratón de Madrid Mes 2Plan de entrenamiento Maratón de Madrid Mes 2
Plan de entrenamiento Maratón de Madrid Mes 2
 
Plan de entrenamiento Maratón de Madrid Mes 1
Plan de entrenamiento Maratón de Madrid Mes 1Plan de entrenamiento Maratón de Madrid Mes 1
Plan de entrenamiento Maratón de Madrid Mes 1
 
Cartel carrera de la mujer ROSAE 2014
Cartel carrera de la mujer ROSAE 2014Cartel carrera de la mujer ROSAE 2014
Cartel carrera de la mujer ROSAE 2014
 
AWS migration: getting to Data Center heaven with AWS and Chef
AWS migration: getting to Data Center heaven with AWS and ChefAWS migration: getting to Data Center heaven with AWS and Chef
AWS migration: getting to Data Center heaven with AWS and Chef
 
Devops madrid: successful case in AWS
Devops madrid: successful case in AWSDevops madrid: successful case in AWS
Devops madrid: successful case in AWS
 
Devops Madrid Marzo - Caso de uso en AWS
Devops Madrid Marzo - Caso de uso en AWSDevops Madrid Marzo - Caso de uso en AWS
Devops Madrid Marzo - Caso de uso en AWS
 
Configuration management with Chef
Configuration management with ChefConfiguration management with Chef
Configuration management with Chef
 
DevOps and Chef improve your life
DevOps and Chef improve your life DevOps and Chef improve your life
DevOps and Chef improve your life
 
MongoDB Devops Madrid February 2012
MongoDB Devops Madrid February 2012MongoDB Devops Madrid February 2012
MongoDB Devops Madrid February 2012
 
Amazon EC2: What is this and what can I do with it?
Amazon EC2: What is this and what can I do with it?Amazon EC2: What is this and what can I do with it?
Amazon EC2: What is this and what can I do with it?
 
MongoDB - Madrid Devops Febrero
MongoDB - Madrid Devops FebreroMongoDB - Madrid Devops Febrero
MongoDB - Madrid Devops Febrero
 

Dernier

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 

Dernier (20)

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 

Immutable servers provided by Vagrant/Chef/AWS

  • 2. Web http://juanvicenteherrera.eu Twitter https://twitter.com/jvicenteherrera LinkedIn http://es.linkedin.com/in/jvherrera GitHub https://github.com/juanviz Slideshare http://www.slideshare.net/juanvicenteherrera About me
  • 3. Index 1. Requisites 2. Vagrant Box 3. Vagrantfile 4. Vagrant Settings 5. Vagrant Commands 6. Provisioning with Chef 7. Multi machines 8. Version control 9. Cleanup 10.Conclusions 11.References
  • 4. Why a common toolchain? • Consistency • Efficiency • Safety • Automation
  • 5. Requisites Knife https://learnchef.opscode.com/quickstart/workstation-setup/ Vagrant $ gem install vagrant (ruby needed) Vagrant plugins https://github.com/mitchellh/vagrant-aws $ vagrant plugin install vagrant-aws $ vagrant plugin install vagrant-omnibus $ vagrant plugin install vagrant-butcher AWS credentials Chef server running and chef client certificates
  • 6. Quick view Chef Server AMI base Dev/Ops workstation (Vagrant+knife) VPC Subnet App instances VPC Subnet Virtual Private Cloud AWS Cloud 1. Create instances from AMI base and make chef bootstrap 2. Apply roles
  • 7. Vagrant Box Base Box: The base box is simply a saved hard-disk of a Virtual Machine created with VirtualBox. It can contain anything but it needs at least : • Ruby • VirtualBox guest additions • Puppet or Chef Vagrant AWS Example Box Vagrant providers each require a custom provider-specific box format. These files compose the contents of a box for the AWS provider. • README.md • Vagrantfile • metadata.json To turn this into a box: $ tar cvzf aws.box ./metadata.json ./Vagrantfile
  • 8. Vagrantfile EC2 Vagrant.configure("2") do |config| # Version 2 Vagrant syntax config.vm.box = "dummy" config.omnibus.chef_version = :latest config.vm.provider :aws do |aws, override| aws.keypair_name = "bq” aws.access_key_id = ”xxxxxxxxx" aws.secret_access_key = ”xxxxxxxxxxxxxxxxxx" aws.ami = "ami-3ad1af53” override.ssh.username = "ec2-user” override.ssh.private_key_path = "/Users/juanvi/keypairs/bq.pem" end end
  • 9. Vagrant Settings Creating Vagrant File mkdir [project name] cd [project name] Vagrant init -> creates Vagrantfile Vagrant ec2 test 2 ec2 without chef management $ vagrant plugin install vagrant-aws $ vagrant box add dummy https://github.com/mitchellh/vagrant- aws/raw/master/dummy.box $ vagrant up --provider=aws This box works by using Vagrant's built-in Vagrantfile merging to setup defaults for AWS. These defaults can easily be overwritten by higher- level Vagrantfiles (such as project root Vagrantfiles).
  • 10. Vagrant Commands vagrant box # Commands to manage system boxes vagrant destroy # Destroy the environment, deleting the created virtual machines vagrant halt # Halt the running VMs in the environment vagrant help [TASK] # Describe available tasks or one specific task vagrant init [box_name] [box_url] # Initializes the current folder for Vagrant usage vagrant package # Package a Vagrant environment for distribution vagrant provision # Rerun the provisioning scripts on a running VM vagrant reload # Reload the environment, halting it then restarting it. vagrant resume # Resume a suspended Vagrant environment. vagrant ssh # SSH into the currently running Vagrant environment. vagrant ssh_config # outputs .ssh/config valid syntax for connecting to this environment via ssh vagrant status # Shows the status of the current Vagrant environment. vagrant suspend # Suspend a running Vagrant environment. vagrant up # Creates the Vagrant environment vagrant version # Prints the Vagrant version information
  • 11. Provisioning with Chef web.vm.provision :chef_client do |chef| chef.chef_server_url = "https://api.opscode.com/organizations/juanvi" chef.validation_key_path = "/Users/juanvi/chef- repo/.chef/juanvi-validator-new.pem" chef.validation_client_name = "juanvi-validator" # Provision with the database role chef.add_role("webserver") # Set the environment for the chef server chef.environment = ”prod"
  • 12. Multi Machines Vagrant.configure("2") do |config| config.vm.define :web do |web| web.vm.box = "apache" end web.vm.provision :chef_client do |chef| chef.chef_server_url = chef_server_url chef.validation_key_path = validation_key_path chef.validation_client_name = validation_client_name ……. end config.vm.define :db do |db| db.vm.box = "mysql" end db.vm.provision :chef_client do |chef| chef.chef_server_url = chef_server_url chef.validation_key_path = validation_key_path chef.validation_client_name = validation_client_name … end end
  • 13. Provisioning with Chef Provisioning only one kind of server executing chef_client $ vagrant provision web --provision-with chef_client Provisioning the whole platform executing chef_client $ vagrant provision --provision-with chef_client After editing the Vagrantfile you need to 'reboot' the machine to take this settings $ vagrant reload
  • 14. Version control with git Version control Now is a good time to version control our awesome- vagrant project $ cd $ git init $ git add Vagrantfile $ git commit -m "This was just my first commit"
  • 15. Cleanup $ vagrant plugin install vagrant-butcher then you can terminate a instance and deregister in the Chef server executing: $ vagrant destroy -f [Butcher] knife.rb location set to '/path/to/knife.rb' [Butcher] Chef node 'node_name' successfully butchered from the server... [Butcher] Chef client 'node_name' successfully butchered from the server... [default] Forcing shutdown of VM... [default] Destroying VM and associated drives...
  • 16. • The task of writing recipes is not picked up by all team members, and seem to stay the main job of the system oriented people on the team. • Reading Chef Cookbooks help people understand what is needed and makes it easy to point out what needs to be changed Conclusions
  • 17. Conclusions • Before committing cookbooks-> destroy 'development' box and re-provision a new box to see if this works. (immutable servers) • The longer the provision takes, the less frequent people do it. It's important to keep that process as fast as possible.
  • 18. Conclusions • Installation problems would get noticed far sooner in the process. • People would only do a full rebuild in the morning when getting their coffee.
  • 19. Conclusions • It clearly helps everybody to have a consistent environment to develop against, the latest version is just one git pull away. • The central approach drives people to • do frequent commits • do stable commits.
  • 20. Conclusions • Vagrant is an essential part of the DevOps process: it is the solution to developing, testing and deploying in the same environment. It thus ensures a smoother transition of your project from the dev team to the ops team. • Chef has to be the only way to manage the configuration; NO MORE MANUAL CHANGES
  • 21. Hands On A typical situation in a company that makes software is the arrival of a new developer that has the tough task to install a lot of tools in his workstation that in the best case takes days. We are going to create a whole developer stack in AWS not for this new developer but for a complete new developer team! After make magic with Vagrant and Chef the unique task for all of the members of the team to begin to work with the environment is clone the git repository, receive the chef and EC2 certificates/keypairs from their project leader.
  • 22. Hands On 1. Download project (private repository) 2. Configure ~/.chef/aws_credentials.rb with your AWS security credentials 3. Check that you have in ~/.chef the chef and aws credentials and keypairs. 4. Well you are ready to create the whole demo environment! 5. $ gem install vagrant 6. $ vagrant plugin install vagrant-aws 7. $ vagrant plugin install vagrant-omnibus 8. $ environment=prod vagrant up --provider=aws 9. Tweak in the /etc/hosts url wiki.juanvicenteherrera.es pointing to elastic ip assigned to ws.(only for test, in prod env change dns entry or proxywebserver virtualhost) 10. Check the url http://wiki.juanvicenteherrera.es 11. Congrats! IT’S MAGIC! You have got your first complete environment created with Vagrant and Chef! Notes: if you don’t want to import a initial db dump… 1. This step can be avoid changing node attribute db_load to 0 in environments/prod.json.
  • 23. Create a new demo environment called demo 1. $ cp environments/prod.json environments/demo.json 2. Change id env and ips 3. $ knife environment from file environments/demo.json 4. Change values for all of the attributes with the appropiate demo values 5. $ vi Vagrantfile -> add the environment in the array of the allowed environments 6. if (environment_id =~ /^(prod|qa|dev|demo)$/) Add demo and app db password in encrypted databags for the new environment 7. $ knife data bag edit --secret-file ~/.chef/encrypted_data_bag_secret secrets php_app 8. $ knife data bag edit --secret-file ~/.chef/encrypted_data_bag_secret secrets postgresql 9. $ environment=demo vagrant up --provider=aws 10.Tweak in the /etc/hosts url wiki.juanvicenteherrera.es pointing to elastic ip assigned to ws.(only for test, in prod env change dns entry or proxywebserver virtualhost) 11. Check the url http://wiki.juanvicenteherrera.es Hands On
  • 24. Video with the execution of the hands on available at: https://vimeo.com/76336356 For more information about this hands on check http://juanvicenteherrera.eu/2013/09/04/immutable- servers-with-vagrant-and-chef-chapter-i/ Hands On
  • 25. References • Video of this presentation recorded in Madrid DevOps’ September meeting • https://vimeo.com/76334136 • https://vimeo.com/76334134 • Vagrant: Up and Running, Mitchell Hashimoto http://www.amazon.com/Vagrant-Up-Running-Mitchell- Hashimoto/dp/1449335837/ref=sr_1_1?ie=UTF8&qid=1375433466&sr=8- 1&keywords=vagrant+up • http://juanvicenteherrera.eu/2013/09/04/immutable-servers-with-vagrant- and-chef-chapter-i/ • https://github.com/mitchellh/vagrant-aws • http://docs.vagrantup.com/v2/cli/index.html • http://red-badger.com/blog/2013/02/21/automating-your-infrastructure-with- vagrant-chef-from-development-to-the-cloud/ • http://docs-v1.vagrantup.com/v1/docs/provisioners/chef_server.html
  • 27. irc channel #madrid-devops. Web https://sites.google.com/site/madridevops/ Twitter https://twitter.com/MadridDevops Grupo de Google https://groups.google.com/d/forum/madriddevops GitHub https://github.com/madrid-devops Group information