Building and deploying serverless applications introduces new challenges for developers whose development workflows are optimized for traditional VM-based applications. In this session, we discuss a method for automating the deployment of serverless applications running on AWS Lambda. We first cover how you can model and express serverless applications using the open-source AWS Serverless Application Model (AWS SAM). Then, we discuss how you can use CI/CD tooling from AWS CodePipeline and AWS CodeBuild, and how to bootstrap the entire toolset using AWS CodeStar. We will also cover best practices to embed in your deployment workflow specific to serverless applications. You will also hear from iRobot about its approach to serverless deployment. iRobot will share how it achieves coordinated deployments of microservices, maintains long-lived and/or separately-managed resources (like databases), and red/black deployments.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Building CI/CD Pipelines for
Serverless Applications
C h r i s M u n n s – S e r v e r l e s s S e n i o r D e v e l o p e r A d v o c a t e – A W S
B e n K e h o e - C l o u d R o b o t i c s R e s e a r c h S c i e n t i s t – i R o b o t
D e c e m b e r 1 , 2 0 1 7
S R V 3 0 2

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About me:
Chris Munns - munns@amazon.com, @chrismunns
• Senior Developer Advocate - Serverless
• New Yorker
• Previously:
• AWS Business Development Manager – DevOps, July ’15 - Feb ‘17
• AWS Solutions Architect Nov, 2011- Dec 2014
• Formerly on operations teams @Etsy and @Meetup
• Little time at a hedge fund, Xerox and a few other startups
• Rochester Institute of Technology: Applied Networking and Systems
Administration ’05
• Internet infrastructure geek

3. https://secure.flickr.com/photos/mgifford/4525333972
Why are we
here today?

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SERVICES (ANYTHING)
Changes in
data state
Requests to
endpoints
Changes in
resource state
EVENT SOURCE FUNCTION
Node.js
Python
Java
C#
Go (soon)
Serverless applications
NEW!

5. • Integration
tests with
other systems
• Load testing
• UI tests
• Penetration
testing
Release processes have four major phases
Source Build Test Production
• Check-in
source code
such as .java
files.
• Peer review
new code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create
deployable
artifacts
• Deployment
to production
environments

6. • Integration
tests with
other systems
• Load testing
• UI tests
• Penetration
testing
Release processes have four major phases
Source Build Test Production
• Check-in
source code
such as .java
files.
• Peer review
new code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create
deployable
artifacts
• Deployment
to production
environments
Focus for
this talk

7. Release processes levels
Source Build Test Production
Continuous integration
Continuous delivery
Continuous deployment

8. Release processes levels
Source Build Test Production
Continuous integration
Continuous delivery
Continuous deployment
Focus for
this talk

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1. Deploy safely without impacting customers/business
2. Validate/test code in a number of ways:
1. Code itself is free of syntax, regression, and unit test
errors
2. Integration with direct dependencies is working right
3. Entire application stack is operating properly
3. Support multiple environments
1. Development, Staging, Production, etc
Pipeline goals

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1. Deploy safely without impacting customers/business
2. Validate/test code in a number of ways:
1. Code itself is free of syntax, regression, and unit test
errors
2. Integration with direct dependencies is working right
3. Entire application stack is operating properly
3. Support multiple environments
1. Development, Staging, Production, etc
Pipeline goals
Testing tools
Pipelining tools
Deployment tools

11. Deploying your
applications
https://secure.flickr.com/photos/simononly/15386966677

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Serverless Application Model (SAM)
Simplified template driven deployment
model for serverless applications
Supported serverless resource types:
functions, APIs, and tables
Supports anything AWS CloudFormation
supports
Open specification (Apache 2.0)
https://github.com/awslabs/serverless-application-model

13. SAM template
AWSTemplateFormatVersion: '2010-09-09’
Transform: AWS::Serverless-2016-10-31
Resources:
GetHtmlFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: s3://sam-demo-bucket/todo_list.zip
Handler: index.gethtml
Runtime: nodejs4.3
Policies: AmazonDynamoDBReadOnlyAccess
Events:
GetHtml:
Type: Api
Properties:
Path: /{proxy+}
Method: ANY
ListTable:
Type: AWS::Serverless::SimpleTable

14. SAM template
AWSTemplateFormatVersion: '2010-09-09’
Transform: AWS::Serverless-2016-10-31
Resources:
GetHtmlFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: s3://sam-demo-bucket/todo_list.zip
Handler: index.gethtml
Runtime: nodejs4.3
Policies: AmazonDynamoDBReadOnlyAccess
Events:
GetHtml:
Type: Api
Properties:
Path: /{proxy+}
Method: ANY
ListTable:
Type: AWS::Serverless::SimpleTable
Tells CloudFormation this is a SAM
template it needs to “transform”
Creates an AWS Lambda function
with the referenced managed IAM
policy, runtime, code at the
referenced zip location, and handler
as defined. Also creates an Amazon
API Gateway and takes care of all
mapping/permissions necessary
Creates an Amazon DynamoDB
table with 5 Read & Write units

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SAM Template Capabilities
• Can mix in other non-SAM CloudFormation resources
in the same template
• i.e. Amazon S3, Amazon Kinesis, AWS Step
Functions
• Supports use of Parameters, Mappings, Outputs, etc
• Supports Intrinsic Functions
• Can use ImportValue
(exceptions for RestApiId, Policies, StageName
attributes)
• YAML or JSON

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SAM commands – Package & Deploy
Package
•Creates a deployment package (.zip file)
•Uploads deployment package to an Amazon S3 Bucket
•Adds a CodeUri property with S3 URI
Deploy
•Calls CloudFormation ‘CreateChangeSet’ API
•Calls CloudFormation ‘ExecuteChangeSet’ API

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
NEW: Can deploy AWS Lambda!!
Uses AWS SAM to deploy serverless applications
Supports Lambda Alias Traffic Shifting enabling
canaries and blue|green deployments
Can rollback based on Amazon CloudWatch
Metrics/Alarms
Pre/Post-Traffic Triggers can integrate with other
services (or even call Lambda functions)
AWS CodeDeploy NEW!

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Globals:
Function:
Runtime: nodejs4.3
AutoPublishAlias: !Ref ENVIRONMENT
MyLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
DeploymentPreference:
Type: Linear10PercentEvery10Minutes
Alarms:
# A list of alarms that you want to monitor
- !Ref AliasErrorMetricGreaterThanZeroAlarm
- !Ref LatestVersionErrorMetricGreaterThanZeroAlarm
Hooks:
# Validation Lambda functions that are run before & after traffic
shifting
PreTraffic: !Ref PreTrafficLambdaFunction
PostTraffic: !Ref PostTrafficLambdaFunction
SAM Globals + CodeDeploy Support
NEW!

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Globals:
Function:
Runtime: nodejs4.3
AutoPublishAlias: !Ref ENVIRONMENT
MyLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
DeploymentPreference:
Type: Linear10PercentEvery10Minutes
Alarms:
# A list of alarms that you want to monitor
- !Ref AliasErrorMetricGreaterThanZeroAlarm
- !Ref LatestVersionErrorMetricGreaterThanZeroAlarm
Hooks:
# Validation Lambda functions that are run before & after traffic
shifting
PreTraffic: !Ref PreTrafficLambdaFunction
PostTraffic: !Ref PostTrafficLambdaFunction
SAM Globals + CodeDeploy Support
NEW!

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon API Gateway Canary Support
Use canary release deployments to gradually roll out new
APIs in Amazon API Gateway:
• Configure percent of traffic to go to a new stage
deployment
• Can test stage settings and variables
• API gateway will create additional Amazon CloudWatch
Logs group and CloudWatch metrics for the requests
handled by the canary deployment API
• To rollback: delete the deployment or set percent of
traffic to 0
NEW!

21. Build & test your
application
https://secure.flickr.com/photos/spenceyc/7481166880

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Establish our testing/validation model
We want to make sure our code:
• is without syntax issues
• meets company standards for format
• compiles
• is sufficiently tested at the code level via unit tests
We want to make sure our serverless service:
• functions as it is supposed to in relation to other components
• has appropriate mechanisms to handle failures up or down stream
We want to make sure our entire application/infrastructure:
• functions end to end
• follows security best practices
• handles scaling demands

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Node.js & Python
• .zip file consisting of
your code and any
dependencies
• Use npm/pip to install
libraries
• All dependencies must
be at root level
Java
• Either .zip file with all
code/dependencies, or
standalone .jar
• Use Maven / Eclipse
IDE plugins
• Compiled class &
resource files at root
level, required jars in
/lib directory
C# (.NET Core)
• Either .zip file with all
code/dependencies, or
a standalone .dll
• Use NuGet /
VisualStudio plugins
• All assemblies (.dll) at
root level
Building a deployment package

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Testing tools
Code Inspection/Test Coverage:
• Landscape - https://landscape.io/ (only for Python)
• CodeClimate - https://codeclimate.com/
• Coveralls.io - https://coveralls.io/
Mocking/stubbing tools:
• https://github.com/atlassian/localstack - “A fully functional local AWS cloud stack. Develop and test
your cloud apps offline!”
• Includes:
• https://github.com/spulec/moto - boto mock tool
• https://github.com/mhart/dynalite - DynamoDB testing tool
• https://github.com/mhart/kinesalite - Kinesis testing tool
• more!
API Interface/UI testing:
• Runscope - https://www.runscope.com/ - API Monitoring/Testing
• Ghost Inspector - https://ghostinspector.com/ - Web interface testing

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fully managed build service that can compile source
code, runs tests, and produces software packages
Scales continuously and processes multiple builds
concurrently
Can consume environment variables from AWS SSM
Parameter Store
NEW: Can run in your VPC
NEW: Supports dependency caching
AWS CodeBuild
NEW!

26. version: 0.1
environment_variables:
plaintext:
"INPUT_FILE": "saml.yaml”
"S3_BUCKET": ""
phases:
install:
commands:
- npm install
pre_build:
commands:
- eslint *.js
build:
commands:
- npm test
post_build:
commands:
- aws cloudformation package --template $INPUT_FILE --s3-
bucket $S3_BUCKET --output-template post-saml.yaml
artifacts:
type: zip
files:
- post-saml.yaml
- beta.json
buildspec.yml Example

27. version: 0.1
environment_variables:
plaintext:
"INPUT_FILE": "saml.yaml”
"S3_BUCKET": ""
phases:
install:
commands:
- npm install
pre_build:
commands:
- eslint *.js
build:
commands:
- npm test
post_build:
commands:
- aws cloudformation package --template $INPUT_FILE --s3-
bucket $S3_BUCKET --output-template post-saml.yaml
artifacts:
type: zip
files:
- post-saml.yaml
- beta.json
• Variables to be used by phases of
build
• Examples for what you can do in
the phases of a build:
• You can install packages or run
commands to prepare your
environment in ”install”.
• Run syntax checking,
commands in “pre_build”.
• Execute your build
tool/command in “build”
• Test your app further or ship a
container image to a repository
in post_build
• Create and store an artifact in S3
buildspec.yml Example

28. Building your
pipeline
https://www.flickr.com/photos/seattlemunicipalarchives/12504672623/

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Continuous delivery service for fast and
reliable application updates
Model and visualize your software release
process
Builds, tests, and deploys your code every
time there is a code change
Integrates with third-party tools and AWS
AWS CodePipeline

30. Source
Source
GitHub
Build
CodeBuild
AWS CodeBuild
Deploy
MyServerlessApp
AWS CodeDeploy
Pipeline
Stage
Action
Transition
CodePipeline
MyApplication

31. Build
CodeBuild
AWS CodeBuild
NotifyDevelopers
Lambda
Parallel actions
Source
Source
GitHub
CodePipeline
MyApplication
Deploy
MyServerlessApp
AWS CodeDeploy

32. Build
CodeBuild
AWS CodeBuild
NotifyDevelopers
Lambda
TestAPI
Runscope
Sequential actions
Deploy
MyServerlessApp
AWS CodeDeploy
Source
Source
GitHub
CodePipeline
MyApplication

33. Build
CodeBuild
AWS CodeBuild
Staging-Deploy
MyServerlessApp
AWS CodeDeploy
Prod-Deploy
ProdServerlessApp
AWS CodeDeploy
QATeamReview
Manual Approval
Manual Approvals
Review
CodePipeline
MyApplication

34. An example minimal developer’s pipeline:
MyBranch-Source
Source
CodeCommit
MyApplication
Build
test-build-source
CodeBuild
MyDev-Deploy
create-changeset
AWS CloudFormation
execute-changeset
AWS CloudFormation
Run-stubs
AWS Lambda
This pipeline:
• Three Stages
• Builds code artifact
• One Development environment
• Uses SAM/CloudFormation to
deploy artifact and other AWS
resources
• Has Lambda custom actions for
running my own testing functions

35. Source
Source
AWS CodeCommit
MyApplication
An example pipeline that goes to production:
Build
test-build-source
AWS CodeBuild
Deploy Testing
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Run-stubs
AWS Lambda
Deploy Staging
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Run-API-test
Runscope
QA-Sign-off
Manual Approval
Review
Deploy Prod
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Post-Deploy-Slack
AWS Lambda
This CodePipeline pipeline:
• Five Stages
• Builds code artifact w/ CodeBuild
• Three deployed to “Environments”
• Uses SAM/CloudFormation to
deploy artifact and other AWS
resources
• Has Lambda custom actions for
running my own testing functions
• Integrates with a 3rd party
tool/service
• Has a manual approval before
deploying to production

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Where and what to test?
Source
MyApplication
Build
Deploy Testing
Deploy Staging
Deploy Prod
• Code review via Pull
Requests
• (NEW In CodeCommit)
• Lint/syntax check
• Unit tests pass
• Code successfully
compiles
• Application deploys
successfully
• Mocked/stubbed
integration tests
• Application deploys
successfully
• Tests against real services
(potentially against
production dependencies)
• Deploy canaries
• Complete wait period
successfully
• Deploy 100%
1.
2.
3.
4.
5.

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Environments, Stages, Versioning, & Canaries?
A few best practices:
1. Use canaries for production deployments with a rollback as
automated as possible
2. In Lambda Versioning is useful if you need to support multiple
versions to multiple consumers/invocation points
3. In API Gateway Stages work similarly and are useful if you need
to support multiple API versions
4. Try to always have separate “stacks” for Development, Testing,
Staging, Production environments
1. Do not use Stages or Versioning for this
2. Think about having different accounts all together for
this

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SAM Best Practices
• Use Parameters and Mappings when possible to build
dynamic templates based on user inputs and pseudo
parameters such as AWS::Region
• Use the new Globals section to simplify templates
• Use ExportValue & ImportValue to share resource
information across stacks
• Build out multiple environments, such as for
Development, Test, Production and even DR using the
same template, even across accounts
SAM Template
Source
Control
Dev
Test
Prod

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lambda Environment Variables
• Key-value pairs that you can dynamically pass to your
function
• Available via standard environment variable APIs such as
process.env for Node.js or os.environ for Python
• Can optionally be encrypted via AWS Key Management
Service (KMS)
• Allows you to specify in IAM what roles have access to
the keys to decrypt the information
• Useful for creating environments per stage (i.e. dev,
testing, production)

40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API Gateway Stage Variables
• Stage variables act like environment variables
• Use stage variables to store configuration values
• Stage variables are available in the $context object
• Values are accessible from most fields in API Gateway
• Lambda function ARN
• HTTP endpoint
• Custom authorizer function name
• Parameter mappings

41. Lambda and API Gateway Variables + SAM
Parameters:
MyEnvironment:
Type: String
Default: testing
AllowedValues:
- testing
- staging
- prod
Description: Environment of this stack of
resources
SpecialFeature1:
Type: String
Default: false
AllowedValues:
- true
- false
Description: Enable new SpecialFeature1
…
…
#Lambda
MyFunction:
Type: 'AWS::Serverless::Function'
Properties:
…
Environment:
Variables:
ENVIRONMENT: !Ref: MyEnvironment
Spec_Feature1: !Ref: SpecialFeature1
…
#API Gateway
MyApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
…
Variables:
ENVIRONMENT: !Ref: MyEnvironment
SPEC_Feature1: !Ref: SpecialFeature1
…

42. AWS Systems Manager – Parameter Store
Centralized store to manage your
configuration data
• supports hierarchies
• plain-text or encrypted with KMS
• Can send notifications of changes
to Amazon SNS/ AWS Lambda
• Can be secured with IAM
• Calls recorded in CloudTrail
• Can be tagged
• Available via API/SDK
Useful for: centralized environment
variables, secrets control, feature
flags
from __future__ import print_function
import json
import boto3
ssm = boto3.client('ssm', 'us-east-1')
def get_parameters():
response = ssm.get_parameters(
Names=['LambdaSecureString'],WithDe
cryption=True
)
for parameter in
response['Parameters']:
return parameter['Value']
def lambda_handler(event, context):
value = get_parameters()
print("value1 = " + value)
return value # Echo back the first key
value

43. Via referenced parameter file:
CodePipeline + CloudFormation Parameters
Via Parameter Overrides:

44. Start with AWS CodeStar

45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Serverless Development
@ iRobot

46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ben Kehoe
• Cloud Robotics Research
Scientist
at iRobot
• Serverless evangelist
• AWS Community Hero
About me
@ben11kehoe

47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Experience building devices,
not cloud applications
• Fleet already at scale
• Go straight to serverless to
skip the undifferentiated
heavy lifting step
Choosing serverless at iRobot

48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deployment
Source

49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Red/black deployment
Blue/green: update behind
the load balancer
Red/black: entirely
new copy

50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• On-prem Jenkins servers deploying fully
serverless applications
• Migrating to Amazon EC2
• Experimenting with CodePipeline
• Separate AWS account per environment
• At least one per developer
• CI but not CD
• Coordination with robot firmware
iRobot deployment setup

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Single-button deploys
• One artifact through pipeline
• Partial testing after each
commit, full test nightly
• Delivered roughly weekly to
integration testing
environment, monthly to
production (along with app,
OTA firmware)
iRobot deployment process

52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Absolutely critical
• Not worth doing local or
mocked integration testing
• Extensive unit testing for local
validation
• Stubbed SDK calls
• CI for integration testing once
deployed
Serverless integration testing

53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Database migrations handled manually
Application Proxy/manifest Resources
v1
v2
v3
v4
A
1
B
1
C
1
B
2
D
1
A
1
B
1
C
1
B
2
A
1
C
1
D
1
A
1
C
1
B
2

54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Same architecture and
infrastructure in dev accounts
as production
• Orders of magnitude more
churn
• Exercise account limits
• CI can give you warning of
platform issues before they hit
production
DiffOps

55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FIN, ACK
Peer review: Step 1 for most CI/CD processes
Continuous Integration: A Must!
Continuous Delivery: Configure it up through pre-Production
environments, use a ”gate” or manual approval/task to push to production
Multiple Environments: So easy and so low cost with #serverless
“Basic” 5 stage pipeline: Source, Build, Test, Pre-Production, Production
Check out the new features in:
• AWS CodeDeploy
• AWS CodeCommit
• Amazon Cloud9
• (and many more this week)
• AWS Lambda
• Amazon API Gateway
• AWS SAM
• AWS CodeBuild

56. aws.amazon.com/serverless

57. Chris Munns
munns@amazon.com
@chrismunnshttps://www.flickr.com/photos/theredproject/3302110152/

58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THANK YOU!
S R V 3 0 2