Session 2, Presentation 3 from the AWS Sydney Summit

1. Simon Elisha | Principal Solutions Architect
CONTINOUS DEPLOYMENT
WITH AMAZON WEB SERVICES

2. CONTINUOUS
INTEGRATION

3. DEVELOPER

4. SOURCE CODE
REPOSITORY

5. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER

6. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER

7. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
PICK
TASKS

8. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
SUBMIT
CODE

9. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
SCHEDULE
BUILD

10. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
RECURENT
BUILDS

11. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
CODE
FETCH

12. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
CODE QUALITY
TESTS
TEST
RESULTS

13. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
BUILD OUTPUT

14. SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
DOCS
BINARIES /
PACKAGES

16. SOURCE CODE
REPOSITORY
DNS
CONTINUOUS
INTEGRATION SERVER
PROJECT
MANAGEMENT SERVER
BUILDS

18. KEY = ITERATION

19. ITERATIVELY MODIFY THE
SYSTEM TO BETTER MEET
THE EXPECTATIONS OF
YOUR USERS

20. ON-DEMAND
PAY AS YOU GO
ELASTIC

21. =
PROGRAMMABLE
PLATFORM

25. AWS
CLOUDFORMATION
STACK-BASED DEPLOYMENT
SERVICE

26. CLOUDFORMATION
TEMPLATE

28. {
"Description" : "Create RDS with username and password",
"Resources" : {
"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "500",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : "MyName",
"MasterUserPassword" : "MyPassword"
}
}
}
}

29. "AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"mysql" : [],
"mysql-server" : [],
"httpd" : [],
"php" : [],
"php-mysql" : []
}
},
"sources" : {
"/var/www/html" :
"https://s3.amazonaws.com/my-builds/build-v4.zip"
}
}
}

30. {
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2
KeyPair to enable SSH
access to the instance",
"Type" : "String"
}
},
}

31. CLOUDFORMATION
TEMPLATE
PROCEDURAL
DEFINITION
Create it programmatically
KNOWN
CONFIGURATION
Store stack configuration in
source control
PARAMETER
DRIVEN
Dynamic and user-driven
templates
COLLABORATION
Share templates with ease
as just files

34. Template ELBs to front
secondary cache
~100 Nginx
secondary cache
servers
2-3 Nginx mid-
tier cache
servers
Stack
CLOUDFORMATION
TEMPLATE
VIDEO CACHING
INFRASTRUCTURE

35. APPLICATION
VERSIONS
&
INFRASTRUCTURE
VERSIONS

36. CLOUDFORMATION
TEMPLATE

37. TEST ENVIRONMENTS

39. 30,000 REQUESTS / SECOND
1 TB TRAFFIC / DAY

40. “…AWS seemed to be the best solution
available to allow a small, independent
company to rapidly develop and test a
completely new infrastructure, and host it.
We also loved the flexibility that AWS
allowed us, when spinning up smaller test
environments, for beta trials, QA,
localization, and during development. The
low initial cost was also crucial.”
Alex Evans, CTO

41. CONTINUOUS
DEPLOYMENT
SMALL, FREQUENT CHANGES
CONSTANTLY INTEGRATING INTO
PRODUCTION.

42. SOFTWARE DEPLOY
≠
PRODUCT LAUNCH

44. 1.5 BILLION PAGE VIEWS
$83 MILLION IN TRANSACTIONS
4.2 MILLION ITEMS SOLD
OCTOBER 2012

45. 30 DEPLOYS PER DAY
= 1 DEPLOY EVERY 20 MINUTES

46. "Production is truly the only place you
can validate your code."

51. HTTP://CODEASCRAFT.ETSY.COM/
HTTPS://GITHUB.COM/ETSY/DEPLOYINATOR

56. HTTP://SORCERY>SMUGMUG.COM/

57. AWS OPSWORKS
INTEGRATED APPLICATION
MANAGEMENT

58. MODEL, CONTROL AND AUTOMATE
AT ANY SCALE AND COMPLEXITY

59. A stack represents
your application. One
stack might be used
for staging and
another for production.
Tell OpsWorks where
it can find your code
and define any
additional deployment
tasks. OpsWorks will
take care of deploying
your app.
Scale your stack
based on time or
load. Clone your
production stack to a
different region.
Automate workflows
for common tasks.
A layer defines how to
setup and configure a
set of instances and
related resources such
as volumes
and software.

60. A stack represents
your application. One
stack might be used
for staging and
another for production.
Tell OpsWorks where
it can find your code
and define any
additional deployment
tasks. OpsWorks will
take care of deploying
your app.
Scale your stack
based on time or
load. Clone your
production stack to a
different region.
Automate workflows
for common tasks.
A layer defines how to
setup and configure a
set of instances and
related resources such
as volumes
and software.

61. A stack represents
your application. One
stack might be used
for staging and
another for production.
Tell OpsWorks where
it can find your code
and define any
additional deployment
tasks. OpsWorks will
take care of deploying
your app.
Scale your stack
based on time or
load. Clone your
production stack to a
different region.
Automate workflows
for common tasks.
A layer defines how to
setup and configure a
set of instances and
related resources such
as volumes
and software.

62. A stack represents
your application. One
stack might be used
for staging and
another for production.
A layer defines how to
setup and configure a
set of instances and
related resources such
as volumes
and software.
Tell OpsWorks where
it can find your code
and define any
additional deployment
tasks. OpsWorks will
take care of deploying
your app.
Scale your stack
based on time or
load. Clone your
production stack to a
different region.
Automate workflows
for common tasks.

63. FREE OF CHARGE

65. 14 BILLION REQUESTS/MONTH
50 000 DATABASE UPDATES/SEC
NO CACHE

66. “AWS OpsWorks gives us the tools we
need to automate operations.
We can scale Monster World, one of the
largest Facebook games, to millions of
users without ever needing more than two
backend developers”
Jesper Richter-Reichhelm
Head of engineering – Wooga

67. A/B TESTING

69. 24%

70. AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2

71. AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2
CONVENIENCE CONTROL

72. AWS Elastic
Beanstalk
AWS
OpsWorks
AWS
CloudFormation
Amazon EC2
CONVENIENCE CONTROL
HIGH-LEVEL DO IT YOURSELF

73. LOAD TESTING

79. USING AMAZON EC2 TO SIMULATE
2.4 MILLION PLAYERS

80. 11.6s
Mean time between
deployments
(weekday)
1,079
Max number of
deployments in a
single hour
10,000
Mean number of
hosts
simultaneously
receiving a
deployment
30,000
Max number of
hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT
AMAZON.COM

81. CONTINUOUS
INTEGRATION
CONTINUOUS
DEPLOYMENT

82. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
EXPERIMENTATION

83. CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
IMPROVEMENT

84. INNOVATE

85. « Want to increase innovation?
Lower the cost of failure »
Joi Ito

86. SPEED AND AGILITY
Experiment
Often
Fail quickly at
a low cost
More Innovation
Experiment
Infrequently
Failure is
expensive
Less Innovation
“ON-PREMISE”

88. 1x QA
1x DEV
Shared across all teams (100+ people)
Previous Environment

89. Allow anyone to deploy
e2e
env for dev & test
Support Continuous
Delivery
Objectives

90. 400-500 instances per day
20-30 environments used
for dev, QA & CI
Multiple teams doing CD
Current State

91. A Tale of Scarcity

92. Limits
are not
helpful
Does not encourage the right
behaviour

95. Efficient Usage
provide
tools
assign
the cost
no hard
limits

96. metrics

97. tools
enable users to be
more efficient

98. One more thing…

99. ARCHITECTURE CENTER
http://aws.amazon.com/architecture
TECHNICAL ARTICLES
http://aws.amazon.com/articles
PODCAST
http://aws.amazon.com/podcast
BLOG
http://aws.typepad.com

100. CONTINOUS DEPLOYMENT
WITH AMAZON WEB SERVICES