This presentation is part of the AWS Lunch & Learn series.

1. Running Containerised Applications at
scale on AWS

2. What to Expect From This Session
• Why Containers & how they are used
• Challenges of scaling containerised workloads
• How to scale containerised workloads using Amazon ECS

3. Why Containers?

4. Why Containers?
Environment
Consistency

6. Why Containers?
Environment
Consistency
Developer
Productivity
Version
Control
Operational
Efficiency

7. Why Containers?
Serverless? Containers?

8. Monolithic vs SOA vs Microservices
SOA
Coarse-grained
Microservices
Fine-grained
Monolithic
Single Unit

9. Monolithic Architecture
Order UI User UI Shipping UI
Order
Service
User Service
Shipping
Service
Data Access

10. Monolithic Architecture

11. Monolithic Architecture

12. Microservices Architecture
Order UI User UI
Shipping
UI
Order Service
User
Service
Shipping
Service

13. Scaling Microservices
Order UI User UI UI
Order Service Service
Shipping
Service
Order UI
Order UI
User UI UIShipping
UI
Order Service
Order Service
Service
Service
Service
Service
User
Service
Shipping
Service

14. Challenges of Scaling
Containerised Workloads

15. Going from this…
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

16. ..to this – is hard!
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS

17. Challenges running containers at scale
Resources and State Management
Monitoring
Service Discovery
Deployment

18. Challenges running containers at scale
Resources and State Management
Monitoring
Service Discovery
Deployment

19. Orchestration vs Choreography

20. Amazon ECS Concepts
•Clusters
•Tasks
•Services

21. Amazon ECS Concepts
Regional
Resource Pool
Grouping of
Container Instances
Start Empty,
Scale Dynamically

22. Amazon ECS Concepts - Clusters
EC2
INSTANCES
ECS
AGENT
ECS
AGENT
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
ECS
AGENT

23. Amazon ECS Concepts - Clusters
EC2
INSTANCES
ECS
AGENT
ECS
AGENT
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
ECS
AGENT

24. Amazon ECS Concepts - Tasks
•Unit of work
•Grouping of related containers
•Run on container instances

25. Amazon ECS Concepts - Tasks
EC2
INSTANCES
ECS
AGENT
ECS
AGENT
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
ECS
AGENT
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container

26. Amazon ECS Concepts - Tasks
EC2
INSTANCES
ECS
AGENT
ECS
AGENT
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
ECS
AGENT
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container

27. Amazon ECS Concepts - Services
•Good for long-running applications
•Load balance traffic across containers
•Automatically recover unhealthy containers
•Discover services

28. Amazon ECS Concepts - Services
LOAD
BALANCER
Internet
EC2
INSTANCES
ECS
AGENT
ECS
AGENT
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
ECS
AGENT
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
LOAD
BALANCER

29. Amazon ECS Concepts - Services
Internet
EC2
INSTANCES
ECS
AGENT
ECS
AGENT
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
ECS
AGENT
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
TASK
Container
LOAD
BALANCER
LOAD
BALANCER

30. Application Auto Scaling
AMAZON ECS
AMAZON
CLOUDWATCH
ELASTIC LOAD
BALANCING
Auto Scaling ECS cluster

31. Application Auto Scaling
AMAZON ECS
Publish
metrics
AMAZON
CLOUDWATCH
ELASTIC LOAD
BALANCING
Auto Scaling ECS cluster
Scale in / out
policies

32. Application Auto Scaling
AMAZON ECS
Publish
metrics
AMAZON
CLOUDWATCH
ELASTIC LOAD
BALANCING
Auto Scaling ECS cluster
Add/remove
ECS tasks
Scale in / out
policies

33. Application Auto Scaling
AMAZON ECS
Publish
metrics
AMAZON
CLOUDWATCH
ELASTIC LOAD
BALANCING
Auto Scaling ECS cluster
Add/remove
ECS tasks
Scale in / out
policies

34. Application Auto Scaling
AMAZON ECS
Publish
metrics
AMAZON
CLOUDWATCH
ELASTIC LOAD
BALANCING
Auto Scaling ECS cluster
Add/remove
ECS tasks
Scale in / out
policies

35. Application Auto Scaling
AMAZON ECS
Publish
metrics
AMAZON
CLOUDWATCH
ELASTIC LOAD
BALANCING
Auto Scaling ECS cluster
Add/remove
ECS tasks
Scale in / out
policies

36. Flexible Container Placement
•Applications
•Batch Jobs
•Multiple Schedulers

37. Designed for Use with Other AWS Services
• Elastic Load Balancing (ELB)
• Application Load Balancing (ALB)
• Amazon Elastic Block Store (EBS)
• Amazon Virtual Private Cloud (VPC)
• AWS Identity and Access Management
• AWS CloudTrail

38. http://www.slideshare.net/AmazonWebServices/aws-reinvent-2016-running-batch-jobs-on-amazon-ecs-con310

39. Scalable

40. Demo – app.catsndogs.lol

41. Challenges running containers at scale
Resources and State Management
Monitoring
Service Discovery
Deployment

42. Service Discovery with ALB
TASK A TASK A TASK A
Application Load
Balancer

43. Service Discovery with DNS
https://github.com/awslabs/service-discovery-ecs-dns
TASK A TASK B TASK C
ecssd_agent ecssd_agent
Route53 Private
Hosted Zone

44. Service Discovery with DNS
https://github.com/awslabs/service-discovery-ecs-dns
TASK A TASK B TASK C
ecssd_agent ecssd_agent
Route53 Private
Hosted Zone

45. Service Discovery with Weaveworks
•DNS interface for cross-host container communication
•Gossip protocol to share grouped updates
•Overlay network between hosts

46. Service Discovery with Weaveworks
HTTP Server
Data Producer
Docker
Weave
Router/DNS
Weave Proxy
ECS Agent
ECS Task
HTTP Server
Data Producer
Docker
Weave
Router/DNS
Weave Proxy
ECS Agent
ECS Task
HTTP Server
Data Producer
Docker
Weave
Router/DNS
Weave Proxy
ECS Agent
ECS Task
Amazon ECS Infrastructure

47. Service Discovery with Consul
• Consul Agent – Runs on each node, responsible for checking the
health of the services and of the node itself.
• One or more Consul servers – Store and replicate data, leader
elected using the Raft consensus algorithm
• Registrator agent – Automatically register / deregisters services
based on published ports and metadata from the container
environment variables defined in the ECS task definition

48. Service Discovery with Consul
Back end 1
Back end 2
Registrator
Consul Agent
Front end
Registrator
Consul Agent
Consul-Server
ECSClusterECSCluster

49. Challenges running containers at scale
Resources and State Management
Monitoring
Service Discovery
Deployment

50. Monitoring with Amazon CloudWatch
• Metric data sent to CloudWatch in 1-minute periods and
recorded for a period of two weeks
• Available metrics: CPUReservation, MemoryReservation,
CPUUtilization, MemoryUtilization
• Available dimensions: ClusterName, ServiceName

51. Monitoring with Amazon CloudWatch

52. Monitoring with Amazon CloudWatch
• Use the CloudWatch monitoring scripts to monitor additional
metrics, e.g. disk space:
# Edit crontab
> crontab -e
# Add command to report disk space utilization to CloudWatch every five minutes
*/5 * * * * <path_to>/mon-put-instance-data.pl --disk-space-util --disk-space-used --disk-
space-avail --disk-path=/ --from-cron

53. Logging with CloudWatch Logs
{
"image": ”nginx:latest",
...
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": ”nginx",
"awslogs-region": "us-east-1"
}
}
{
• Defined within the task definition
• Available log drivers:
• awslogs
• fluentd
• splunk
• syslog

54. Challenges running containers at scale
Resources and State Management
Monitoring
Service Discovery
Deployment

55. Scheduling Containers: Long Running App
•Deploy using least space: minimumHealthyPercent = 50%,
maximumPercent = 100%
Instance 1 Instance 2 Instance 3 Instance 4 Instance 5
Task 1 Task 2 Task 3 Task 4

56. Scheduling Containers: Long Running App
•Deploy using least space: minimumHealthyPercent = 50%,
maximumPercent = 100%
Instance 1 Instance 2 Instance 3 Instance 4 Instance 5
Task 1 Task 2 Task 3 Task 4

57. Scheduling Containers: Long Running App
•Deploy using least space: minimumHealthyPercent = 50%,
maximumPercent = 100%
Instance 1 Instance 2 Instance 3 Instance 4 Instance 5
Task 1 Task 2 Task 3 Task 4

58. Scheduling Containers: Long Running App
•Deploy quickly without reducing service capacity::
minimumHealthyPercent = 100%, maximumPercent = 200%
Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Instance 8
Task 1 Task 2 Task 3 Task 4

59. Scheduling Containers: Long Running App
•Deploy quickly without reducing service capacity::
minimumHealthyPercent = 100%, maximumPercent = 200%
Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Instance 8
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8

60. Scheduling Containers: Long Running App
•Deploy quickly without reducing service capacity::
minimumHealthyPercent = 100%, maximumPercent = 200%
Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Instance 8
Task 5 Task 6 Task 7 Task 8

61. Scheduling Containers: Long Running App
• Blue-Green Deployments
• Define two ECS services
• Each service is associated w/ ELB
• Both ELBs in Route 53 record set
with weighted routing policy, 100%
primary, 0% secondary
• Deploy to blue or green service and
switch weights
TASK A TASK B
Route53 Record Set
With Weighted Routing
Policy
100% 0%

62. IAM Roles for Tasks
DynamoDB
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
S3 Storage

63. Demo – app.catsndogs.lol

64. ECS Placement Strategies
Binpacking Spread Affinity Distinct Instance

65. Placement: Targeting Instance Type
g2.2xlarge t2.small g2.2xlarge g2.2xlarge

66. Placement: Targeting Instance Type & Zone
g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge
t2.small
t2.small t2.medium
us-east-1aus-east-1d

67. Placement: Spread across Zone and Binpack
g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c

68. Placement: Affinity and Anti-Affinity
g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c

69. Placement: Distinct Instances
t2.medium g2.2xlarge t2.micro t2.small
t2.small t2.small g2.2xlarge t2.small
t2.small t2.small
g2.2xlarge t2.small

70. Demo – app.catsndogs.lol

71. AWS
CodeCommit
AWS
CodePipeline
AWS
CodeBuild
Amazon ECR
Amazon ECS
instance
Spot
Instance
AWS
CloudFormation
1. Commit
Code
2. Trigger
Pipeline
3.Build
Artifact
5. Update
Stack
6. Update Service
4. Push Image
Reference Architecture

72. New Features!
• ECS adds console support for Spot fleet creation
• https://aws.amazon.com/about-aws/whats-new/2017/06/amazon-ecs-
adds-console-support-for-spot-fleet-creation/
• ECS adds support for time and event based scheduling
• https://aws.amazon.com/about-aws/whats-new/2017/06/amazon-ecs-now-
supports-time-and-event-based-task-scheduling/

73. Thank You