Deep Dive on Microservices and Amazon ECS

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Johan Broman, Solutions Architect AWS
Jude D´Souza, Architect Wrapp
2016-05-04
Deep Dive on Microservices and
Amazon ECS

2. What to Expect from this Session
Microservices: What are They?
Challenges of Microservices
Microservices on Amazon ECS
Containers @ Wrapp, presented by Jude D´Souza

3. What are Microservices?

4. What are Microservices?
“is a software architecture style in which complex
applications are composed of small, independent
processes communicating with each other using
language-agnostic APIs. These services are small, highly
decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia
https://en.wikipedia.org/wiki/Microservices

5. Monolithic vs. SOA vs. Microservices
SOA
Coarse-
grained
Microservices
Fine-grained
Monolithic
Single unit

6. Order UI User UI
Shipping
UI
Order
Service
User
Service
Shipping
Service
Data
Access
Monolithic Architecture

7. Monolithic Architecture – Scaling

8. Order UI User UI
Shipping
UI
Order
Service
User
Service
Shipping
Service
Microservices Architecture

9. Order UI User UI UI
Order
Service
Service
Shipping
Service
Order UI
Order UI
User UI UIShipping
UI
Order
ServiceOrder
Service
Service
Service
Service
Service
User
Service
Shipping
Service
Microservices Architecture – Scaling

10. What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment

11. Containers are Natural for Services
Simple to model
Any app, any language
Image is the version
Test & deploy same artifact
Stateless servers decrease change risk

12. What are containers?
Self contained
Process isolation
OS virtualization
Automation
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

13. Container advantages
Portable
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

14. Container advantages
Fast
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

15. Container advantages
Efficient
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

16. Managing One Host is Straightforward
Server
Guest OS
Bins/Libs Bins/Libs
App2App1

17. Managing a Fleet is Hard
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
AZ 1 AZ 2
AZ 3

18. What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment

19. Amazon EC2 Container Service

21. Scalable

22. Designed for use with other AWS services
Elastic Load Balancing
Amazon Elastic Block Store
Amazon Virtual Private Cloud
AWS Identity and Access Management
AWS CloudTrail

23. What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment

24. Scheduling Containers on ECS
Batch jobs
ECS task scheduler
Run tasks once
Batch jobs
RunTask (random)
StartTask (placed)
Long-running apps
ECS service scheduler
Health management
Scale-up and scale-down
Zone-aware
Grouped containers
ELB (optional)

25. Scheduling Containers: Long-running App
Optionally run your service behind a load balancer.
ELB currently supports a fixed relationship between the
load balancer port and the container instance port.
If a task fails the ELB health check, the task is killed and
restarted (until service reaches desired capacity).

26. Scheduling Containers: Long-running app
Deploy using the least space: minimumHealthyPercent =
50%, maximumPercent = 100%

27. Scheduling Containers: Long-running App
Deploy quickly without reducing service capacity:
minimumHealthyPercent = 100%, maximumPercent =
200%

28. Scheduling Containers: Long-running App
Blue-Green Deployments
• Define two ECS services
• Each service is associated w/ ELB
• Both ELBs in Route 53 record set
with weighted routing policy, 100%
Primary, 0% Secondary
• Deploy to blue or green service and
switch weights
TaskTask
Route 53
record set
with
weighted
routing
policy
0%
100%

29. What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment

30. Monitoring with Amazon CloudWatch
Metric data sent to CloudWatch in 1-minute periods and
recorded for a period of two weeks
Available metrics: CPUReservation, MemoryReservation,
CPUUtilization, MemoryUtilization
Available dimensions: ClusterName, ServiceName

31. Monitoring with Amazon CloudWatch

32. Monitoring with Amazon CloudWatch
Use the CloudWatch monitoring scripts to monitor
additional metrics: e.g., disk space
# Edit crontab
> crontab -e
# Add command to report disk space utilization to CloudWatch every five minutes
*/5 * * * * <path_to>/mon-put-instance-data.pl --disk-space-util --disk-space-used --disk-
space-avail --disk-path=/ --from-cron

33. Monitoring Amazon ECS with Datadog

34. Monitoring Amazon ECS with Sysdig Cloud

35. Containers @ Wrapp
Powered by AWS EC2 Container Service
Jude D’Souza
Solutions Architect @ Wrapp
Email: jude@wrapp.com

36. About Me …
q Jude D’Souza – Stockholm, Sweden ß Karachi, Pakistan
jude@wrapp.com
q MS – Distributed Systems
KTH Royal Institute of Technology, Stockholm, Sweden
q Solutions Architect @ Wrapp
www.wrapp.com
q Passion for building large scale distributed systems

37. About Wrapp …
Wrapp delivers personal offers and rewards based on where you shop, dine and the brands you
follow.
q Microservice Architecture
q Infrastructure
q 13 ECS Clusters
q 50+ Instances
q 90+ Services
q Technology Stack
q Operating System - Ubuntu
q Consul, Registrator, AWS(EC2, ECS, Lambda), Docker, HAProxy
q Monitoring – Sensu, Riemann
q Logs - Logentries and Sumo
q Languages - Go, Python, Java

38. Agenda
§ Architecture before ECS
– Service Orchestration
– Service Discovery
– Service Deployments
– Service Availability & Scalability
– Container Registry
§ Transition towards ECS
– Microservice Architecture @ Wrapp
§ Why ECS?
§ Next Challenges
§ QA

39. How we did it … Before ECS (1/3)
Service Orchestration
q Via runlist configs
On instance boot:
q Determine autoscaling group
q Pull runlist for autoscaling group
q Run containers specified in config
q Add these to supervisor
misc ops api
misc
Autoscaling group
ops
Autoscaling group
api
Autoscaling group
PULL RUNLIST CONFIGS
Sample Runlist configuration file
ü Service Orchestration
Service Deployments
Service Availability & Scalability
Container Registry

40. Service Deployments
q Find instances where service is running (via serf)
q SSH into these instances
q Do rolling deploy (Terminate container, Launch container with new version)
mis
c ops api
misc
Autoscaling group
- misc-i-abc 10.0.0.1 rewards/http/9090
- misc-i-def 10.0.1.1 rewards/http/9090
- misc-i-ghi 10.0.2.1 rewards/http/9090
- ....
- ....
Sample serf data for services
SERF
CLUSTER
ops
Autoscaling group
api
Autoscaling
group
How we did it … Before ECS (2/3)
ü Service Orchestration
ü Service Deployments
Service Availability & Scalability
Container Registry

41. Service Availability
q Supervisord
Service Scalability
q Service itself can't scale
q Scale only by scaling autoscaling group
How we did it … Before ECS (3/3)
Container Registry
q Hosting our own container registry
q Backed by s3
misc ops api
misc
Autoscaling group
SERF
CLUSTER
ops
Autoscaling group
api
Autoscaling
group
misc
ü Service Orchestration
ü Service Deployments
ü Service Availability & Scalability
ü Container Registry

42. Service Orchestration
q Runlists
q Supervisor
Service Deployments
q Supervisor
Service Availability
q Supervisor
Service Scalability
q EC2 Autoscaling
Container Registry
q Self-hosting
Service Discovery
q Serf
q HAProxy
Service Discovery
q Consul
q Registrator
q HAProxy
Service Orchestration
Service Deployments
Service Availability
Service Scalability (with lambda)
AWS Container Registry
+ Service stats (cpu + memory)
+ Service monitoring
+ Service Provisioning
Transition to ECS

43. Wrapp Microservice Architecture
PUBLIC
SUBNE
T
PRIVATE
SUBNET
Elastic Load Balancer
api api api
Microservices
ZONE A ZONE B ZONE C
api.wrapp.com DNS

44. Opsweb – Tooling around controlling our Infrastructure
Cluster Overview

45. Opsweb – Tooling around controlling our Infrastructure
Services Overview

46. Service Scalability (Concept from Amazon)
mis
c
misc
Autoscaling group
Scaling Amazon ECS Services Automatically Using Amazon CloudWatch and AWS Lambda:
https://aws.amazon.com/blogs/compute/scaling-amazon-ecs-services-automatically-using-amazon-cloudwatch-and-aws-lambda/
CloudWatch Alarm
SNS Topic AWS
Lambda
EC2 Container
Service
ECS Instances
Trigger CloudWatch alarm from
ECS Metrics (CPU/ Memory)
Trigger Lambda function from
SNS Topic
+1 DesiredCount in Service

47. Service (CPU/ Memory) Provisioning
api
emailsauth
rewards
notifications
eventbus
Typical ECS Instance
ü Memory allocation by Service
ü Allocate CPU Units by Service
ü Efficient Resource utilization
(Container placement)

48. Why ECS?
q Our Experience
q Generally good service (ECS is oK)
q Great support
q We don’t need to host these
services ourselves on AWS
q Always improving to make their
services better and better
ü Supports built-in service discovery, loadbalancing and routing
ü Weave Flux: Supports built-in service discovery, loadbalancing and routing
ü Docker Swarm: Supports orchestrating containers
ü Fleet - Container Orchestration

49. api
8080
Consul
server-b
Consul
server-a
Consul
server-c
users
8082
auth
8081
users
8082
Registrator Registrator Registrator
Consul
Agent
Consul
Agent
Consul
Agent
Key/value store
Service
catalog
10.0.0.1 10.0.0.2 10.0.0.3
Service Discovery @ Wrapp (1/2)
HA Consul Cluster

50. Next Challenges (1/2) – Short comings perceived …
Built-in (like kube-proxy, weave-proxy) ?
q Service Discovery
q Service Routing
q Service Load balancing
Approach 1: ELB Per service?
q Costly -90+ services, 70+ ELBs
Approach 2: Single ELB for all services?
ü Service Routing works
ü Service Load balancing works
q No health check per service
q No way to return HTTP 503
(but possible via haproxy http check option)
Benefits:
q Removes tools like Consul and Registrator
Elastic Load Balancer
8080 8081 8082
ECS Boto3:
create_service()
Route53: ELB.dns = elb-services
frontend users-in
bind users:80
mode http
default_backend users
backend users
server users elb-services:8080 …
####
>> curl users/ping
SINGLE ELB FOR MULTIPLE SERVICES

51. Next Challenges (2/2) – Short comings perceived …
Ø Cron scheduling
q Setup ecs tasks to run on cron schedule
Ø ECS Event stream
q Actionables on certain events (when deploy completes)
Ø ECS Container Logging
Ø Optionally Integrate ECS Cluster with EC2 Autoscaling Group

52. Summary
q The Wrapp Architecture before ECS
q Transition to ECS
q Service Discovery @ Wrapp with Consul, Registrator and HAProxy
q Our perceived shortcomings from ECS
q Built-in Service Discovery
q Cron scheduling on Tasks
q Event stream

53. What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment

54. Continuous Delivery to ECS with Jenkins
4. Push image to
Docker registry
2. Build image
from sources
3. Run test on image
1. Code push
triggers build
5. Update service
6. Pull image

55. ECS CI/CD Partners

56. Continuous Delivery to ECS with Shippable

57. Thank you!