Containers gained strong traction since day one for both enterprises and startups. Today AWS customers are launching hundreds of millions of new containers – each week. Join us as we cover the state of containerized application development and deployment trends. This session will dive deep on new container capabilities that help customers deploying and running container-based workloads for web services and batches. Speaker: Steffen Grunwald, Senior Solutions Architect, AWS & Sascha Möllering, Senior Solutions Architect, AWS

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Container-based
Architectures on AWS
Sascha Möllering, Solutions Architect, @sascha242, Amazon Web Services EMEA SARL
Steffen Grunwald, Solutions Architect, @steffeng, Amazon Web Services EMEA SARL

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
We started in 2014

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Customers running Docker on EC2 from the very
beginning...

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
But there were pain points.
Things like scheduling, placing, managing
and deploying containers were difficult.
They wanted something to make those pain
points better.

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
So we built ECS
Highly scalable,
high performance
container
management system
A managed
platform
Amazon Elastic
Container Service
Cluster
management
Container
orchestration
Deep AWS
integration

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
So we built ECS
Highly scalable,
high performance
container
management system
AWS VPC
networking mode
Advanced task
placement
Deep integration
with AWS platform
ECS CLI…{ }
Global footprint
Powerful scheduling
engines
Auto scaling
CloudWatch metrics
Load balancers
Amazon Elastic
Container Service

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon ECS
EC2 INSTANCES
ECS
AGENT
TASK
Containers
TASK
ECS
AGENT
TASK TASK
AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER
MANAGEMENT ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK TASK
Internet LOAD
BALANCER
LOAD
BALANCER
Containers
Containers

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Cluster of hosts
ECS
AGENT
TASK TASK
ECS
AGENT
TASK TASK
AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK TASK
Internet LOAD
BALANCER
LOAD
BALANCER
EC2 INSTANCES
TASK TASK
TASK TASK
AGENT COMMUNICATION
SERVICE
API
CLUSTER
MANAGEMENT ENGINE
KEY/VALUE STORE
TASK TASK
Internet
Containers
Containers
Containers

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Lightweight agent on each host
EC2 INSTANCES
ECS
AGENT
TASK TASK
ECS
AGENT
TASK TASK
AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK TASK
Internet LOAD
BALANCER
LOAD
BALANCER
TASK TASK
TASK TASK
AGENT COMMUNICATION
SERVICE
API
CLUSTER
MANAGEMENT ENGINE
KEY/VALUE STORE
TASK TASK
Internet
Containers
Containers
Containers

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
API for launching containers on the cluster
EC2 INSTANCES
ECS
AGENT
TASK TASK
ECS
AGENT
TASK TASK
AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK TASK
Internet LOAD
BALANCER
LOAD
BALANCER
EC2 INSTANCES
TASK TASK
TASK TASK
AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
TASK TASK
Internet
Containers
Containers
Containers

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
EC2 INSTANCES
ECS
AGENT
TASK TASK
ECS
AGENT
TASK TASK
AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER
MANAGEMENT ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK TASK
Internet LOAD
BALANCER
LOAD
BALANCER
Container task is placed on a host
Containers
Containers
Containers

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Traffic is sent to your host
EC2 INSTANCES
ECS
AGENT
TASK TASK
ECS
AGENT
TASK TASK
AGENT COMMUNICATION
SERVICE
Amazon
ECS
API
CLUSTER
MANAGEMENT ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK TASK
Internet LOAD
BALANCER
LOAD
BALANCER
Containers
Containers
Containers

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
ECS Optimized Amazon Machine Images (AMIs)
• Optimized AMIs available for Linux & Windows
• Bring your own images based on it
• Expects ECS cluster name in user-data
• Update images on SNS update notifications

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Publish metrics
Auto Scaling ECS service
Availability
Zone A
Availability
Zone B
TASK A
Add/Remove
ECS tasks
TASK C
TASK BScaling Policies
Amazon
CloudWatch
Amazon ECS
Application
Load Balancer
Automatic Service Scaling

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
Amazon
DynamoDB
Amazon
S3
IAM Roles For Tasks
Amazon ECS

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
•prod.app1.db-pass
•general.license-code
•prod.app2.user-name
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
System Manager –
Parameter Store
Secrets Management
Amazon ECS

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Elastic Container Registry (Amazon ECR)
• Cloud-based Docker image registry
• Fully managed
• Secure – images encrypted at rest,
integrated with IAM
• Scalable and Highly Available
• Integrated with Amazon ECS and the
Docker CLI

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Fargate
No cluster or
infrastructure to
manage or scale
Everything is
handled at the
container level
Scale seamlessly
on demand
Underlying technology for container
management

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
What does Fargate mean?
Not worrying about scaling, underlying infrastructure, cluster
resources, capacity, setup.
Just give it a task definition or pod (in 2018), set some resource
limits, and away you go.

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Task Definitions Repository on GitHub

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS
CodePipeline
AWS
CodeBuild
Developers
1 2
3 4
7
Continuous Deployment in Amazon ECS
5
6
AWS Fargate Amazon EC2
Amazon ECS
Amazon ECR

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
“Run Kubernetes for me.”

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Elastic Container Service for Kubernetes: EKS
Managed Kubernetes on
AWS
Highly
available
Automated
version
upgrades
Integration
with other
AWS services
Etcd
Master
Managed
Kubernetes
control
plane CloudTrail,
CloudWatch, ELB,
IAM, VPC, PrivateLink

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Elastic Container Service for Kubernetes
• Platform for enterprises to run production grade workloads
• Provides a native and upstream Kubernetes experience
• Not forced to use additional AWS services, but offer seamless integration
• EKS team actively contributes to the Kubernetes project

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Elastic Container Service for Kubernetes

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl
EKS Architecture

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Cross-account Kubernetes
Workers Masters
Customer VPC EKS VPC
Network Load
Balancer
ENI
API Access
Kubectl
Exec/Logs
TLS
Static IPs

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
EKS Cross-Account Networking: Availability Zones
Availability
Zone 1
Master Master
Availability
Zone 2
Availability
Zone 3
Master
Workers Workers Workers
Customer VPC
EKS VPC
ENI ENI ENI

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Jenkins – CI/CD with Kubernetes

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS CodePipeline – CI/CD with Kubernetes
AWS CodePipeline
AWS CodeCommit AWS CodeBuild AWS Lambda
Amazon ECR
1 2 4
3 5
6
Developer

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Batch Processing with Containers

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
What is batch computing?
Run jobs
asynchronously and
automatically across one
or more computers.
Jobs may have
dependencies, making the
sequencing and scheduling
of multiple jobs complex
and challenging.

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Cloud makes Sense for Batch
• Scalable
• Reliable
• Choice:
• Compute resources (GPUs, RAM- or CPU-bound)
• Storage resources (fileshares, performance characteristics)
• Downstream services (e.g. databases, streaming services)
• Pricing models
• Pay as you go (per second)

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Anatomy of a Batch
Event
Changes in
data state
Requests to
endpoints
Services (anything)
Scheduled
triggers
Compute
Execution
Your Code
Auto Scaling
Job Queue

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Options for Batch Workloads on AWS
AWS Lambda
NEW: Amazon SQS as
built-in event source
AWS BatchAmazon Elastic
Container
Service (ECS)

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Containers make Sense for Batch
• Benefits from container development model
• Polyglot
• Do one thing well
• Black Box – and easy to model

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Basic Batch Workflow with ECS
Input is put to
S3 Bucket
Output is put
to S3 Bucket
Event is put to
Amazon SQS
ECS provisions clusters
and schedules tasks
Containerized batch workers process files
Queue depth
is used for
scaling
Workers
poll queue

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Introducing AWS Batch
• Fully managed batch primitives
• Focus on your applications (shell scripts, Linux
executables, Docker images) and their resource
requirements
• We take care of the rest!

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Execution
What you need to take care of...
Auto Scaling
Event
Changes in
data state
Requests to
endpoints
Services (anything)
Scheduled
triggers
Compute
Your Code
Job Queue

45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Typical AWS Batch Job Architecture
Input is put to
S3 Bucket
Output is put
to S3 Bucket
Job Queue with
runnable jobs
AWS Batch Compute Environment
IAM role
Job definition
Application
Image
+ config
Scheduler

46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
„Flip Image“ Job‘s Dockerfile
FROM amazonlinux:latest
RUN yum update -y
RUN yum install ImageMagick aws-cli -y
ADD flip.sh /usr/local/bin/flip.sh
WORKDIR /tmp
USER nobody
ENTRYPOINT ["/usr/local/bin/flip.sh"]

47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
flip.sh
#!/bin/bash
SRCBKT=$1 # SOURCE BUCKET
OBJ=$2 # OBJECT KEY
TRGBKT=$3 # TARGET BUCKET
error_exit() { echo "${1}" >&2; exit 1; }
tmpfile=$(mktemp /tmp/image.XXXXXX)
aws s3 cp "s3://$SRCBKT/$OBJ" "$tmpfile" # DOWNLOAD
|| error_exit "Download failed $SRCBKT/$OBJ"
convert -flip "$tmpfile" "$tmpfile" # FLIP w/ IMAGEMAGICK
|| error_exit "Failed to flip file"
aws s3 cp "$tmpfile" "s3://$TRGBKT/$OBJ" # UPLOAD
|| error_exit "Upload failed $TRGBKT/$OBJ"

48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Important AWS Batch Concepts
• Job definitions
• Jobs
• Job queue
• Compute environments

49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Job Definitions
AWS Batch job definitions specify how jobs are to be run.
Some attributes in a job definition:
• Container Image
• IAM role associated with the job
• vCPU and memory requirements
• Mount points
• Environment variables
• Retry strategy

50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Jobs are the unit of work executed by
AWS Batch.
Set/ overwrite Job Definition
attributes, e.g.:
• Command
• Parameters
• Dependencies
Job command example:
Jobs

51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Easily run massively parallel jobs
Efficient way to run:
• Parametric sweeps
• Monte Carlo simulations
• Processing a large collection of objects
Start up to 10,000 copies of an application with a single call using Array Jobs.
Get
File
List
flip.sh [1]
flip.sh [2]
flip.sh [n]

52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
AWS Batch supports up to 10 attempts per job:
• errors in the AWS Batch job
• termination of the Spot Instance
The AWS_BATCH_JOB_ATTEMPT environment variable is set to the container's
corresponding job attempt number.
Job Retries

53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Managed
AWS scales and configures your
instances for you.
Optional choice:
• On demand/ Spot
• Instance Types/ Mix
• Amazon Machine Image (AMI)
Compute Environments
Unmanaged
You control and manage the
instance configuration,
provisioning, and scaling.
Full control over scaling and
instance provisioning for the ECS
cluster used by AWS Batch.

54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Bring your own AMIs
Customer Provided AMIs let you set the AMI that is launched as part of a
managed compute environment.
Makes it possible to configure Docker settings, mount EBS/ EFS volumes, and
configure drivers for GPU jobs.
AMIs must be Linux-based, HVM and have a working ECS agent installation.

55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Jobs are submitted to Job Queues.
Job queues are assigned to one or more compute environments.
Each job queue has a priority assigned. Jobs in queues with higher priority take
precedence.
Job Queues

56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Use Spot Instances
Get your jobs done faster
or cheaper.
AWS Batch retries jobs on
instance terminations and
selects from multiple
instance types.

57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
No additional charge for AWS Batch or Amazon ECS.
You only pay for the underlying resources that you consume!
Pricing

58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Let’s
build!

59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Questions?
Ask these guys!

60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Thank you!