Containers on AWS - State of the Union - CON201 - re:Invent 2017

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Containers State
Of The Union
D e e p a k S i n g h
D i r e c t o r — C o m p u t e S e r v i c e s

WELCOME

“Cloud native architectures take full advantage
of on-demand delivery, global deployment,
elasticity, and higher-level services. They enable
huge improvements in developer productivity,
availability, utilization, and cost savings.”
– Adrian Cockcroft

MAPBOX
Franziska Schmidt

✌ Hi,
I’m Franziska,
Platform Engineer at Mapbox

A YEAR AGO, WE DECIDED TO MIGRATE
TO ECS FOR
Better cost savings
Faster and more flexible scaling
Increased developer productivity
THE OBVIOUS BENEFITS

THE NON-OBVIOUS BENEFITS
💎 Open source, private infrastructure
💎 Consistency across organisation
💎 Cost monitoring tools
💎 Empowering teams

TEAMS AND SYSTEMS @ MAPBOX
📌Geocoding
👩💻Developer tools
📱Mobile
🗺Maps
🚥Directions
📊Analysis

OUR SYSTEMS CAN BE CLASSIFIED
INTO TWO CATEGORIES
Processing jobs
A processing pipeline that runs
data validation on edits from
Open Street Maps
API services
API-maps, an API webserver
serving vector tiles for a given
set of coordinates

PROCESSING TRAFFIC AT PEAK
70,000 tasks within one hour
200 EC2s at peak
Running in 2 regions

API TRAFFIC AT PEAK
500–1500 tasks
40–200 EC2s
Running in 7 regions

• Consistency across organisation
• Cost clarity
• Empowering teams

SEPARATING APPLICATION CODE FROM
INFRASTRUCTURE CODE HAS BEEN
BENEFICIAL IN A VARIETY OF WAYS
Developers don’t need to touch infrastructure setup
Centralising infrastructure scripts
You can open source application code easier!

👉 MIGRATING TO ECS
helped us separate infrastructure
setup from core domain code

• Cost clarity

Establish patterns, conventions, and best practices across ECS use
Reduce copy/paste shared functionalities
Easier to pick up context in unknown projects
ECS-API
C lou dFormation he lp e r lib rary for
se tting u p an ECS -base d HTTP API

Big system migrations
provide opportunities
for rethinking system designs
💭

💎 Cost clarity

SPOT BY DEFAULT
Running on ECS is making it possible for us to run on spot by default
We have faster task start up, outages are easier to mitigate
We have worked to diversify our clusters to limit impact of spot price outs

👉 WE SAVE COSTS
through spot by default on ECS;
we gain clarity through custom
cost monitoring

💎 Cost clarity
💎 Empowering teams

EMPOWERING TEAMS
Many benefits I’ve talked about can be summarized as empowering teams
Better tools to launch code faster
More ownership over cost and spending

WANT MORE DETAILS?
See you at CON405 on Friday morning!

“Home is behind, the world ahead, and
there are many paths to tread through
shadows to the edge of night, until the
stars are all alight.”

“Home is behind, the world ahead, and
there are many paths to tread through
shadows to the edge of night, until the
stars are all alight.”
From “A Walking Song”, Lord of the Rings

3 years ago

Early 2014

Polyglot
packaging

Portable runtime

$ vi Dockerfile
$ docker build -t
mykillerapp:0.0.1
$ docker run -it
mykillerapp:0.0.1

WHAT WE DID IN 2017
CLI V1.0Container Access to
Environmental Metadata
Support for Docker
Privileged Mode
Network Load
Balancer Support
Container
Instance
Draining
Linux Capabilities
Lifecycle Policies for
Container Images
Seoul Region
Beijing
Region
Support for
Device and
Init Flags
Task Elastic Network Interfaces
Cron and Cloudwatch
Event Task Scheduling
Windows Containers
HIPAA
Eligibility
Console Support for SpotFleet
Override Parameters for
RunTask and StartTask APIs
Console UX Improvements
Application Load
Balancer Support
Add Attributes During Boot

WHY DO CUSTOMERS USE ECS?

THILINA GUNASINGHE

WORLD’S LARGEST
RESTAURANT COMPANY
1.9M
PEOPLE
working for
McDonald’s
and franchisees
64M+
CUSTOMERS
served every day
37K
RESTAURANTS
120
COUNTRIES

HOME DELIVERY OVERVIEW
DIGITAL & DELIVERY
Bringing McDonald’s to you

CRITICAL BUSINESS REQUIREMENTS
Speed to market: quick turnaround for features and functionality
from concept to production
Scalability and reliability: targets of 250K–500K orders per hour
Multi-country support and integration with multiple third-party
food delivery partners
Cost sensitivity: cost model based on low average check amounts

HOME DELIVERY ARCHITECTURE

Key Architecture Principles
• Microservices with clean APIs, service models, isolation,
independent data models and deployability
• Containers and orchestration for handling massive scale,
reliability and speed to market requirements
• PaaS-based architecture model by leveraging AWS components
such as ECS, SQS, RDS, and Elasticache
• Synchronous and event-based programming models based on
requirements

UNDER THE COVERS: USING ECS TO
SCALE UP TO 20K TPS UNDER 100MS

Task
Task
Task
Scalability and Reliability
{Service 1} Task Definition
"placementStrategy": [
{
"field": "attribute:ecs.availability-zone",
"type": "spread"
}
"placementStrategy": [
{
"field": "memory",
"type": "binpack"
}
]
"placementConstraints": [
{
"expression": "task:group == US",
"type": "memberOf"
}
]
ECS instances (EC2)
EC2 Auto
Scaling policy
ECS Service
Scaling policy
Tasks

FINAL THOUGHTS AND KEY TAKEAWAYS
A thought-out microservice architecture is key for scalability, reliability,
and containerization.
Massive scale achievable (north of 20k TPS under 100ms) in a controlled
manner using Auto Scaling policies and task placement strategies.
Moving to containers simplified our development and deployment models
and in turn provided quicker dev/test iterations.
ECS out-of-the-box integration and deployment models further simplified
our DevOps pipeline.

THANK YOU!

Customers Using Containers at Scale

2015 2016 2017
462% increase
in user growth
since Jan 2016

Right now

OVER 100,000 CLUSTERS

MILLIONS OF INSTANCES

HUNDREDS OF MILLIONS OF NEW
CONTAINERS LAUNCHED

EACH WEEK!

PATRICK CULLEN
P r i n c i p a l A r c h i t e c t

ABOUT WASHINGTON POST
Major news publisher
95 million monthly visitors
1 billion monthly page views
June 2015
20,000,000
42,500,000
65,000,000
87,500,000
110,000,000
132,500,000
Oct. 2013 Nov. 2017
MONTHLY UVs

ABOUT ARC PUBLISHING
Fast and efficient content management system (CMS)
Suite of tools engineered for modern publishers
Speed and innovation

ARCHITECTURE EVOLUTION
Washington Post
Monolithic CMS
VIDEO
STORY
PHOTO
Internet
Application
Database
Microservice CMSVIDEO
STORY
PHOTO
Internet
Application
Database
Internet
Application
Database
Internet
Application
Database

ARC PUBLISHING ARCHITECTURE
Microservice CMS
VIDEO
STORY
PHOTO
Internet
Application
Database
Internet
Application
Database
Internet
Application
Database
Washington Post Tenant 1 Tenant 2
Microservice CMS
VIDEO
STORY
PHOTO
Internet
Application
Database
Internet
Application
Database
Internet
Application
Database
Microservice CMS
VIDEO
STORY
PHOTO
Internet
Application
Database
Internet
Application
Database
Internet
Application
Database
Microservice CMS
VIDEO
STORY
PHOTO
Internet
Application
Database
Internet
Application
Database
Internet
Application
Database
Tenant 3 Tenant 4 Tenant 5
Microservice CMS
VIDEO
STORY
PHOTO
Internet
Application
Database
Internet
Application
Database
Internet
Application
Database
Microservice CMS
VIDEO
STORY
PHOTO
Internet
Application
Database
Internet
Application
Database
Internet
Application
Database

RUNNING AT SCALE
50+
Deployments per hour
150+
ECS instances
100+
Microservices
3,000+
Containers

NILE: CONTAINER PLATFORM

THANK YOU

Compliance
9001/27001/27017/27018

99.99

Make AWS the
BEST PLACE
to run containerized
applications

Make
containers a
fundamental
compute
primitive

Make
containers
tasks a
fundamental
compute
primitive

IAM roles
for tasks

Task Auto
Scaling

Task LBs

Task
networking

TASK NETWORKING FOR ECS

TASK NETWORKING
Io
eth0 172.17.0.1/16
Io
Io
ve-c2
(172.17.0.3/16)
ve-c2
(172.17.0.2/16)
Default/Root Global Namespace
Container 1
Container 2
172.16.0.0
172.16.1.0
172.16.2.0
VPC
10.0.0.27/24
docker0
Io
eth0 172.17.0.1/16
Io
Io
ve-c2
(172.17.0.3/16)
ve-c2
(172.17.0.2/16)
Container 3
Container 4
172.16.0.0
172.16.1.0
172.16.2.0
10.0.0.26/24
docker0

TASK NETWORKING
docker0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0
1. Pre ENI Attachment: The
Primary ENI (eth0) is in the
default namespace
2. ENI Attachment: The new
ENI (eth1) is in the default
namespace
3. ENI Provisioned: The ECS Agent
invokes CNI plugins to move the
new ENI into a new namespace and
configure it with addresses and routes
docker0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0eth1
ecs0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0
docker0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0
ve-c1

TASK NETWORKING
ecs0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0
docker0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0
ve-c1
ecs0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0
docker0
Io
eth0
172.16.0.0
172.16.1.0
172.16.2.0
ve-c1
10.0.0.28/24
10.0.0.29/24
10.0.0.26/24
10.0.0.27/24
Task NamespaceTask Namespace

INTRODUCING MANAGED SERVICE
DISCOVERY FOR ECS
Build apps where services are invoked by name and name
resolves to IP/port dynamically
Service discovery is activated during service deployment

ECS SERVICE DISCOVERY IS
POWERED BY ROUTE 53
ECS Route 53
ECS updates service registry
based on naming convention,
task registrations, de-registrations
and health
Route 53 provides Service Registry

BENEFITS OF THIS APPROACH
Managed
• Just turn on
Highly available
• Tied to Route 53 availability,
scale with SLA ~100%
Extensible
• Public APIs that can be used
across AWS
• Works across clusters,
accounts, AZs
• Works across AWS services
Namespace
Service
AZ 1 AZ 2
ECS
Cluster
VPC

AWS CodePipeline and Amazon ECS

AWS CodePipeline for ECS
AWS
CodeCommit
AWS
CodeBuild
Amazon ECR
AWS
CodePipeline
Amazon ECS
Source
Repository Build Deploy
or or

Create Buildspec File and Add Source Location

Cluster
Management
is a relic
of physical
infrastructure

An API for
containers

What is
Fargate?

FARGATE DEMO

“Run Kubernetes for me.”

“Native AWS Integrations.”

”An Open Source Kubernetes Experience.”

+
=
E L A S T I C C O N T A I N E R S E R V I C E F O R K U B E R N E T E S
(EKS)

mycluster.eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl

Native VPC
networking with
CNI plugin
Pods have the same
VPC address inside
the pod as on the VPC
Simple, secure
networking
Open source and
on Github
…{ }

Nginx Pod
Rails Pod
ENI
Secondary IPs:
10.0.0.1
10.0.0.2
Veth IP: 10.0.0.1
Veth IP: 10.0.0.2
Nginx Pod
Rails Pod
ENI
Veth IP: 10.0.0.20
Veth IP: 10.0.0.22
Secondary IPs:
10.0.0.20
10.0.0.22
ec2.associateaddress()
VPC Subnet – 10.0.0.0/24
Instance 1 Instance 2

POLICY ENFORCEMENT WITH CALICO

1.7.41.7.5
Version
1.7
Version
1.8

GENERALLY
AVAILABLE
2018

One more thing

WITH

THANK YOU
deesingh@amazon.com
@mndoci

Containers on AWS - State of the Union - CON201 - re:Invent 2017

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Containers on AWS - State of the Union - CON201 - re:Invent 2017

Similaire à Containers on AWS - State of the Union - CON201 - re:Invent 2017 (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Containers on AWS - State of the Union - CON201 - re:Invent 2017