Publicité
Publicité

Contenu connexe

Publicité
Publicité

Deployment Tactics

  1. LOYM ENT DEP ICS TACT Managing code from development to production Ian Barber - ian.barber@gmail.com twitter.com/ianbarber | phpir.com
  2. Image: http://flickr.com/photos/denisdervisevic/4527695803
  3. - Table of Contents - 1.... Change Control 2.... Environments 3.... Version Control 4.... The Deploy Process 5.... Scripts 6.... Continuous Integration 7.... Remote Releases 8.... Packaged Releases 9.... Package Management 10.. Managing Hotfixes 11.. Managing Database Changes 12.. Rollbacks 13.. Tactical Deployment
  4. Change control plan execute change change identify verify need close deliver
  5. export copy to code server require compare release md5 report restart back apache
  6. Change Request Form Requested By: J. Teamlead Authorised By: S. Manager Submit Date: 2011-01-27 Change Date: 2011-02-04 Reason For Change: Resolve JIRA-1602 - Listen for new .com variants on vhost Change Request: Release tag 1.1.3 via normal process mv /etc/httpd/conf.d/fooweb.conf /etc/httpd/conf.d/fooweb.old mv ~releases/1.1.3/conf/fooweb.conf /etc/httpd/conf.d/ fooweb.conf Verification: http://foweb.com shows the same page as http://fooweb.com Rollback: Re-release 1.1.2 mv /etc/httpd/conf.d/fooweb.old /etc/httpd/conf.d/fooweb.conf
  7. Environments Production Development static verbose robust dynamic reliable unstable optimised experimental
  8. on Envir onment The Producti Image: http://flickr.com/photos/lejoe/3763218501
  9. The Staging Environment Image: http://flickr.com/photos/simononly/4454401446
  10. n Enviro nment The Int egratio Image: http://flickr.com/photos/unfoldedorigami/2374016430
  11. The Developme nt Environmen t Image: http://flickr.com/photos/drewnew/511936681
  12. VERSION CONTROL Image: http://flickr.com/photos/robbie73/4346732208
  13. /branches/newpage /branches/... /branches/search /trunk /branches/1.1.2 /tags/1.1.2
  14. Development /branches/newpage /branches/... /branches/search /trunk Integration /branches/1.1.2 /tags/1.1.2 Staging Production
  15. master release1.1.1 devel search feature long feature
  16. Production master release1.1.1 Staging devel search feature Integration long feature Development
  17. The DEPLoy PROCESS
  18. The DEPLoy PROCESS transparent flexible easy scalable graceful reliable
  19. support SMTP process config apache vhost app update code config file perms packages commands libpng update cache restart service service
  20. code config package repository repository repository ta da server commands deployment server server controller
  21. code config package repository repository repository da ta server commands deployment server + data controller server
  22. BUILD SCRIPTS #!/bin/bash # Deployment script for FooWeb Project git archive --format=tar --remote=git://repo.com/myrepo/myrepo.git HEAD -o fooweb.tar tar -xf fooweb.tar /var/www service httpd restart
  23. #!/bin/bash # Deployment script for FooWeb Project svn export svn://localhost/fooweb-service/ trunk release cd release && mkdir build cp -r web/* build/ javac -cp /usr/share/java/servlet-api-2.5.jar -d build/WEB-INF/classes src/com/fooweb/service/*.java cd build && jar cvf ../fooweb.war * && cd ../ # assumes autoDeploy is true cp fooweb.war /var/lib/tomcat6/webapps
  24. BUILDS TOOLS tests release code build docs test assets results
  25. build tools
  26. <?xml version="1.0" encoding="UTF-8"?> <project name="FooWeb"> <property name="install" location="/var/lib/ tomcat6/webapps" /> <property name="svn.repo" value="svn:// localhost/fooweb-service/trunk" /> <!--A "clean" target to delete compiled files--> <target name="clean"> <delete dir="build" /> <delete dir="release" /> <delete file="fooweb.war" /> </target>
  27. <!-- Checkout, mkdir and compile--> <target name="build"> <exec executable="svn"> <arg line="export ${svn.repo} release" /> </exec> <mkdir dir="build"/> <copy todir="build"> <fileset dir="release/web" /> </copy> <javac srcdir="release/src" destdir="build/WEB-INF/classes/"> <classpath> <pathelement path="/usr/share/ java/servlet-api-2.5.jar"/> </classpath> </javac> </target>
  28. <!-- Build our WAR file --> <target name="war" depends="build"> <war destfile="fooweb.war" webxml="build/ WEB-INF/web.xml"> <fileset dir="build"/> <classes dir="build/WEB-INF/classes"/> </war> </target> <!-- Copy our file --> <target name="deploy" depends="war"> <copy file="fooweb.war" todir="$ {install}" /> </target> </project>
  29. $ sudo ant deploy Buildfile: build.xml build: [exec] Exported revision 8. [mkdir] Created dir: /tmp/build [copy] Copying 2 files to /tmp/build [copy] Copied 3 empty directories to 1 empty directory under /tmp/build [javac] Compiling 1 source file to /tmp/build/WEB-INF/classes war: [war] Building war: /tmp/fooweb.war deploy: [copy] Copying 1 file to /var/lib/tomcat6/ webapps BUILD SUCCESSFUL Total time: 2 seconds
  30. CONTINUOUS INTEGRATION Look at the Hud son Wiki at dson-ci.org http://wiki.hu
  31. <project name="Fooweb" default="build"> <target name="build" depends="phpunit" /> <target name="init"> <mkdir dir="${basedir}/build/logs" /> </target> <target name="phpunit" depends="init"> <exec executable="phpunit" dir="${basedir}/tests" failonerror="on"> <arg line=" --log-junit '${basedir}/ build/logs/phpunit.xml' --coverage-clover '$ {basedir}/build/logs/clover.xml' --coverage- html '${basedir}/build/logs/coverage'" /> </exec> </target>
  32. <target name="phpcpd" depends="init"> <exec executable="phpcpd" dir="${basedir}/application" failonerror="on"> <arg line=" --log-pmd '${basedir}/build/logs/php-cpd.xml' ." /> </exec> </target> </project>
  33. REMOTE RELEASES Image: http://flickr.com/photos/scragz/309353618
  34. server server authorized_keys authorized_keys cp ss /s h /s h ss cp deployment controller id_rsa.pub user “deploy” ssh-keygen -t rsa
  35. Fabric from fabric.api import * http://fabfile .org # Development environment def dev(): env.user = 'deployer' env.roledefs = { "web" : ['localhost'], "db" : ['localhost'], } # Production environment def production(): env.user = 'deployer' env.roledefs = { "web" : ['primary.fooweb.com', 'secondary.fooweb.com'], "db" : ['backend.fooweb.com'], }
  36. # Package up release - run local def prepare_deploy(): local('svn export svn://localhost/fooweb/ trunk release') with cd('release'): local('tar cvzf ../fooweb.tar.gz .') local('rm -rf release') # Restart web server @roles('web') def restart_webserver(): sudo('/etc/init.d/apache2 restart')
  37. # Deploy to remote servers @roles('web') def deploy(): prepare_deploy() # in case of already existing with settings(warn_only=True): run('mkdir /tmp/release') run('rm -rf /tmp/release/*') put("fooweb.tar.gz", '/tmp/release') with cd('/tmp/release'): run("tar xvzf fooweb.tar.gz") run("rm -rf fooweb.tar.gz") run("mv * /tmp/test") restart_webserver(); local("rm -rf fooweb.tar.gz");
  38. $ fab dev deploy [localhost] run: svn export svn://localhost/ fooweb/trunk release [localhost] run: tar cvzf ../fooweb.tar.gz . [localhost] run: rm -rf release [localhost] run: mkdir /tmp/release [localhost] err: mkdir: cannot create directory `/tmp/release': File exists Warning: run() encountered an error (return code 1) while executing 'mkdir /tmp/release' [localhost] run: rm -rf /tmp/release/* [localhost] put: fooweb.tar.gz -> /tmp/ release/fooweb.tar.gz
  39. [localhost] run: tar xvzf fooweb.tar.gz [localhost] run: rm -rf fooweb.tar.gz [localhost] run: mv * /tmp/test [localhost] sudo: /etc/init.d/apache2 restart Password for ianbarber@localhost: [localhost] out: * Restarting web server apache2 [localhost] out: ... waiting ...done. [localhost] run: rm -rf fooweb.tar.gz Done. Disconnecting from localhost... done.
  40. $ fab production deploy [localhost] run: tar cvzf ../fooweb.tar.gz . .... [primary.fooweb.com] run: rm -rf /tmp/release/ [localhost] run: tar cvzf ../fooweb.tar.gz . .... [secondary.fooweb.com] run: mkdir /tmp/release .... Disconnecting from secondary.fooweb.com...done Disconnecting from primary.fooweb.com... done
  41. $ mkdir config && cd config && capify . [add] writing './Capfile' [add] making directory './config' [add] writing './config/deploy.rb' [done] capified! set :application, "set your application name " set :repository, "set your repository" set :scm, :subversion role :web, "your web-server here" role :app, "your app-server here" role :db, "your primary db-server here", :primary => true Capi strano role :db, "slave db" .com/ https:/ /github capis trano/
  42. set :application, "fooweb" set :repository,"svn://localhost/fooweb/trunk" set :scm, :subversion set :scm_username, "deployment" set :scm_password, "s3kkr3tp4a55" set :scm_checkout, "export" set :keep_releases, 4 set :normalize_asset_timestamps, false set :deploy_to, "/usr/local/#{application}" role :web, "primary.fooweb.com" role :web, "secondary.fooweb.com" role :db, "backend.fooweb.com"
  43. namespace :deploy do task :migrate do # nothing end task :restart do sudo "/etc/init.d/apache2 restart" end end namespace :fooweb do task :perms do sudo "chmod -R a+w #{deploy_to}" end end after "deploy:setup", "fooweb:perms"
  44. $ cap deploy:setup * executing `deploy:setup' * executing "sudo mkdir -p /usr/local/fooweb [...]" servers: ["primary","secondary", "backend"] [backend] executing command [...] command finished triggering after callbacks for deploy:setup * executing `fooweb:perms' * executing "sudo chmod -R a+w /usr/local/ fooweb" servers: ["primary","secondary","backend"] [primary] executing command [...] command finished
  45. $ cap deploy * executing `deploy' * executing `deploy:update' ** transaction: start * executing `deploy:update_code' executing locally: "svn info svn:// localhost/fooweb/trunk -rHEAD" /usr/bin/svn * executing "svn checkout -q -r17 svn:// localhost/fooweb/trunk /usr/local/fooweb/ releases/20110116192456 && (echo 17 > /usr/ local/fooweb/releases/20110116192456/ REVISION)" servers: ["primary.fooweb.com"] [primary.fooweb.com] executing command [....] * executing `deploy:finalize_update' * executing "chmod -R g+w /usr/local/fooweb/
  46. /usr/local/fooweb/ !"" current -> releases/20110116192316 !"" releases #   !"" 20110116190608 #   #   !"" application #   #   !"" log -> /usr/local/fooweb/shared/log #   #   !"" public #   #   !"" REVISION #   #   !"" tmp #   !"" 20110116192316 #   #   !"" application #   #   !"" log -> /usr/local/fooweb/shared/log #   #   !"" public #   #   !"" REVISION #   #   !"" tmp $"" shared
  47. Webistrano https://g ithub.com/ perito r/webistrano
  48. PACKAGED RELEASES Image: http://flickr.com/photos/halfbisqued/2353845688
  49. Fooweb Fooweb Fooweb Mail Service Any SMTP Symfony 1.3 Tomcat 6.0 Server PHP 5.2.12 Java 1.6
  50. !"" application #   !"" controllers #   #   $"" home.php #   $"" library #   $"" Foow #   $"" Router.php !"" fooweb.spec !"" public #   $"" index.php $"" vhosts $"" fooweb.conf
  51. Summary: Fooweb Application Vendor: Fooweb Name: fooweb Version: 1.0 Release: 1 Source0: fooweb-%{version}.tar.gz License: BSD Group: Fooweb BuildArch: noarch BuildRoot: %{_tmppath}/%{name}-%{version}- buildroot Requires: php %description This is the Fooweb web application %prep %setup
  52. %install mkdir -p $RPM_BUILD_ROOT/var/www/fooweb mkdir -p $RPM_BUILD_ROOT/etc/httpd/conf.d/ cp -r application $RPM_BUILD_ROOT/var/www/fooweb cp -r public $RPM_BUILD_ROOT/var/www/fooweb cp vhosts/fooweb.conf $RPM_BUILD_ROOT/etc/httpd/ conf.d/ %clean rm -rf $RPM_BUILD_ROOT %files %dir /var/www %dir /var/www/fooweb %config /etc/httpd/conf.d/fooweb.conf /var/www/fooweb/*
  53. ~$ mkdir buildroot buildroot/tmp ~$ cat .rpmmacro %packager Fooweb Release Manager %_topdir ~/buildroot %_tmppath ~/buildroot/tmp ~$ cd ~/tags ~/tags$ tar cvzf fooweb-1.0.tar.gz fooweb-1.0
  54. ~$ rpmbuild -ta fooweb-1.0.tar.gz ~$ rpm -qip ~/rpmbuild/RPMS/noarch/ fooweb-1.0-1.noarch.rpm Name : fooweb Relocations: (not relocatable) Version : 1.0 Vendor: Fooweb Release : 1 Build Date: Thu 13 Jan 2011 12:26:24 AM PST Install Date: (not installed) Build Host: ubuntu.localdomain Group : Fooweb Source RPM: fooweb-1.0-1.src.rpm Size : 781 License : BSD Signature : (none) Summary : Fooweb Application Description : This is the Fooweb web application
  55. $ mkdir /var/www/repo $ cd /var/www/repo $ mkdir centox/5/fooweb/ {SRPMS,X86_64,i386,noarch} $ cp ~rpmbuild/RPMS/noarch/* centos/5/fooweb/ noarch $ cp ~rpmbuild/SRPMS/* centos/5/fooweb/SRPMS $ createrepo -v centos/5/fooweb/noarch/ centos/5/fooweb/noarch/ !"" fooweb-1.0-1.noarch.rpm $"" repodata !"" filelists.xml.gz !"" other.xml.gz !"" primary.xml.gz $"" repomd.xml
  56. $ cat /etc/yum.repos.d/fooweb.repo [fooweb_noarch] name = Fooweb Private Repository baseurl = http://fooweb.com/repo/centos/5/fooweb/ noarch enabled = 1 gpgcheck = 0 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY- fooweb $ yum update fooweb_noarch 100% |========| 951 B fooweb_noarch/primary 100% |========| 701 B fooweb_noarch 1/1 Setting up Update Process No Packages marked for Update
  57. $ yum info fooweb Available Packages Name : fooweb Arch : noarch Version : 1.0 Release : 1 Size : 3.3 k Repo : fooweb_noarch Summary : Fooweb Application License : BSD Description: This is the Fooweb web application
  58. <target name="buildrpm" depends="init"> <tar destfile="build/rpm/SOURCES/ fooweb.tar.gz" compression="gzip"> <tarfileset dir="${basedir}" prefix="fooweb-1.0"> <include name="*/**" /> <exclude name="build/**" /> </tarfileset> </tar> <copy file="${basedir}/fooweb.spec" tofile="$ {basedir}/build/rpm/SPECS/fooweb.spec" /> <rpm command="-ba" specFile="fooweb.spec" topDir="${basedir}/build/rpm" cleanBuildDir="true" failOnError="true" /> </target>
  59. PACKAGE MANAGEMENT Image: http://flickr.com/photos/southerncalifornian/2129676744
  60. $ sudo aptitude install puppetmaster 0 packages upgraded, 10 newly installed, 0 to remove and 76 not upgraded. Need to get 3,233kB of archives. After unpacking 13.7MB will be used. Puppet http://puppetlabs.com $ sudo aptitude install puppet 0 packages upgraded, 5 newly installed, 0 to remove and 76 not upgraded. Need to get 587kB of archives. After unpacking 1,892kB will be used.
  61. backend fooweb.com primary fooweb.com seconday fooweb.com int fooweb.com staging fooweb.com Puppet master
  62. /etc/puppet !"" auth.conf !"" fileserver.conf !"" manifests #   $"" site.pp !"" modules #   !"" apache2 #   #   $"" manifests #   #   $"" init.pp #   $"" fooweb #   !"" files #   #   $"" fooweb.conf #   $"" manifests #   $"" init.pp !"" puppet.conf $"" templates
  63. class fooweb { package { "fooweb": ensure => latest, } " file { "/etc/apache2/sites-enabled/fooweb.conf": owner => root, group => root, mode => 0444, source => "puppet:///files/fooweb/files/fooweb.conf", notify => Service["apache2"] } } mod ules/fooweb manif ests/init.pp
  64. node "ubuntu.localdomain" { manifests/site.pp include fooweb include apache2 } class apache2 { service { apache2: ensure => running } modules/apache2 manifests/init.pp
  65. # puppet agent -o -v --no-daemonize info: Caching catalog for ubuntu.localdomain info: Applying configuration version '1295514488' notice: /Stage[main]/Fooweb/Package[fooweb]/ ensure: ensure changed 'purged' to 'latest' notice: /Stage[main]/Fooweb/File[/etc/apache2/ sites-enabled/fooweb.conf]/ensure: defined content as '{md5} d41d8cd98f00b204e9800998ecf8427e' info: /Stage[main]/Fooweb/File[/etc/apache2/ sites-enabled/fooweb.conf]: Scheduling refresh of Service[apache2] notice: /Stage[main]/Apache2/Service[apache2]: Triggered 'refresh' from 1 events notice: Finished catalog run in 3.33 seconds # ls /etc/httpd/conf.d/fooweb.conf /etc/httpd/conf.d/fooweb.conf # ls /var/www/fooweb/ application public
  66. MANAGING HOTFIXES Image: http://flickr.com/photos/moogan/8206134
  67. /branches/1.1.3 /trunk /tags/1.1.2 /tags/1.1.3
  68. package copy code run db backup run db changes make code active
  69. package copy code run db backup run db changes make code active
  70. MANAGING Database ChanGES Image: http://flickr.com/photos/theplanetdotcom/4878814847
  71. CREATE TABLE `blogpost` ( `id` int(11) auto_increment NOT NULL PRIMARY KEY, `title` VARCHAR(255), `timestamp` DATETIME, `content` TEXT ); --//@UNDO DBDeploy tp://dbdeploy.com DROP TABLE `blogpost`; ht
  72. ALTER TABLE `blogpost` ADD `author` varchar(255) NULL; --//@UNDO ALTER TABLE `blogpost` DROP `author`;
  73. $ wget http://dbdeploy.googlecode.com/files/ dbdeploy-dist-3.0M2-distribution.zip CREATE TABLE changelog ( change_number BIGINT NOT NULL, delta_set VARCHAR(10) NOT NULL, start_dt TIMESTAMP NOT NULL, complete_dt TIMESTAMP NULL, applied_by VARCHAR(100) NOT NULL, description VARCHAR(500) NOT NULL, PRIMARY KEY(change_number, delta_set) );
  74. $ java -cp mysql-connector-java.jar:dbdeploy- cli-3.0M2.jar com.dbdeploy.CommandLineTarget - D com.mysql.jdbc.Driver -d mysql -o delta.sql -u jdbc:mysql://localhost/foowebdb -U root -P ****** dbdeploy 3.0M2 Reading change scripts from directory dbdeploy. Changes currently applied to database: (none) Scripts available: 1, 2 To be applied: 1, 2
  75. -- START CHANGE SCRIPT #1: 1-create-blogposts.sql CREATE TABLE `blogpost` ( `id` int(11) auto_increment NOT NULL PRIMARY KEY, `title` VARCHAR(255), `timestamp` DATETIME, `content` TEXT ); INSERT INTO changelog (change_number, complete_dt, applied_by, description) VALUES (1, CURRENT_TIMESTAMP, USER(), '1-create- blogposts.sql'); COMMIT;
  76. -- END CHANGE SCRIPT #1: 1-create-blogposts.sql -- START CHANGE SCRIPT #2: 2-add-author.sql ALTER TABLE `blogpost` ADD `author` varchar(255) NULL; INSERT INTO changelog (change_number, complete_dt, applied_by, description) VALUES (2, CURRENT_TIMESTAMP, USER(), '2-add- author.sql'); COMMIT; -- END CHANGE SCRIPT #2: 2-add-author.sql
  77. <?xml version="1.0" encoding="UTF-8" standalone="no"?> <databaseChangeLog [....]> <changeSet author="ianbarber" id="1"> <createTable tableName="blogposts"> <column autoIncrement="true" name="id" type="int(11)"> <constraints nullable="false" primaryKey="true" /> </column> <column name="title" type="varchar(255)" /> <column name="body" type="text" /> <column name="author" type="varchar(255)"/> <column name="date" type="timestamp" /> </createTable> </changeSet> Liquibase ht tp://liquibase.org
  78. <?xml version="1.0" encoding="UTF-8" standalone="no"?> <databaseChangeLog [....] > <include file="v000/master.xml" /> </databaseChangeLog> update.xml <?xml version="1.0" encoding="UTF-8" standalone="no"?> <databaseChangeLog [....]> <include file="v000/create-blog- posts-1.xml" /> </databaseChangeLog> v000/master.xml
  79. $ wget http://downloads.sourceforge.net/ project/liquibase/Liquibase%20Core/2.0.0/ liquibase-2.0.0-bin.zip # Liquibase properties driver: com.mysql.jdbc.Driver url: jdbc:mysql://localhost/fooweb classpath: /usr/share/java/mysql-connector- java.jar username: root password: ******* roperties liquibase.p
  80. $ liquibase --changeLogFile=update.xml update Liquibase Home: /opt/liquibase INFO 1/18/11 1:32 PM:liquibase: Successfully acquired change log lock INFO 1/18/11 1:32 PM:liquibase: Reading from `DATABASECHANGELOG` INFO 1/18/11 1:32 PM:liquibase: Reading from `DATABASECHANGELOG` INFO 1/18/11 1:32 PM:liquibase: ChangeSet v000/create-blog-posts-1.xml::1::ianbarber ran successfully in 101ms INFO 1/18/11 1:32 PM:liquibase: Successfully released change log lock Liquibase Update Successful
  81. class CreateProjects < ActiveRecord::Migration def self.up create_table :projects do |t| t.column :name, :string t.column :description, :text t.column :template, :string t.column :created_at, :datetime t.column :updated_at, :datetime end end def self.down drop_table :projects end end
  82. ROLLING BACK Image: http://flickr.com/photos/roolrool/4758613588
  83. <?xml version="1.0" encoding="UTF-8" standalone="no"?> <databaseChangeLog [....]> <changeSet author="ianbarber" id="2"> <addColumn tableName="blogposts"> <column name="commenter" type="varchar (255)" /> </addColumn> </changeSet> </databaseChangeLog> $ liquibase --changeLogFile=update.xml update Liquibase Home: /opt/liquibase INFO 1/18/11 2:38 PM:liquibase: ChangeSet v000/ add_commenter-2.xml::2::ianbarber ran successfully in 136ms Liquibase Update Successful
  84. $ liquibase --changeLogFile=update.xml rollbackCount 1 Liquibase Home: /opt/liquibase INFO 1/18/11 2:39 PM:liquibase: Successfully acquired change log lock INFO 1/18/11 2:39 PM:liquibase: Reading from `DATABASECHANGELOG` INFO 1/18/11 2:39 PM:liquibase: Rolling Back Changeset:v000/add_commenter-2.xml:: 2::ianbarber::(Checksum: 3:cc45ae1014b26f8b35cb70a5fc39a1ae) INFO 1/18/11 2:39 PM:liquibase: Successfully released change log lock Liquibase Rollback Successful
  85. TACTICAL DEPLOYMENTS Image: http://flickr.com/photos/romainguy/230416692
  86. E.G. mk_ Primary slave_de http://bi lay DB t.ly/hDW DFi Replication read read slave slave 30 Minute Backup Delay
  87. namespace :deploy do namespace :web do task :disable, :roles => :web do on_rollback { rm "#{shared_path}/system/maintenance.html" } require 'erb' deadline, reason = ENV['DATE'], ENV['WHY'] maintenance = ERB.new( File.read("./templates/maintenance.erb" )).result(binding) put maintenance, "#{shared_path}/system/maintenance.html", :mode => 0644 end end
  88. # DATE="16:00 MST" WHY="a database upgrade" cap deploy:web:disable if (-f $document_root/system/maintenance.html) { rewrite ^(.*)$ /system/maintenance.html last; break; }
  89. warm redirect caches & sessions links proxies migrate
  90. developers devops sys QA admins
  91. See : Continuous Deplo yment In 5 Easy Steps http://o reil.ly/13EPgd Image: http://flickr.com/photos/jurvetson/3961794276
  92. Feature F la gs Image: http://flickr.com/photos/rossharmes/4153769740
  93. Gradual Ramp With Feature Without Feature 100% 75% 50% 25% 0% Day 1 Day 2 Day 3 Day 4
  94. Dark Lau nches
  95. THanks! Deployment Tactics: Managing code from development to production Ian Barber - ian.barber@gmail.com twitter.com/ianbarber | phpir.com
Publicité