Find out more how to scale your applications to 10 Million Users

1. Adrian Hornsby, Technical Evangelist @ AWS
Journey Towards Scaling Your
Application to 10 Million Users
@adhorn

3. Let’s start from…

4. What are we building?

5. Static vs Dynamic
Web app vs Enterprise app
Auth vs no-Auth
Server vs Serverless

6. Select your region

7. AWS Global Infrastructure
16
Regions
42 Availability Zones

8. The “Must” from Day 1
• High quality code
• Version controlled
• CI/CD pipeline
• Infrastructure as code
• Security at every layer
• Cost conscious
• Test everything
• DR procedure
Operational Excellence

9. Users > 1

10. Start simple, basic

11. Amazon Simple Storage Service (S3)
Amazon S3

12. Amazon S3
App 0.0v1
http://poliko.adhorn.me.s3-website-eu-west-1.amazonaws.com
Simple Static Website

13. Amazon Route53
• Traffic Flow
• Latency Based Routing
• Geo DNS
• Private DNS for Amazon VPC
• DNS Failover
• Health Checks and Monitoring
• Domain Registration
• CloudFront Zone Apex Support
• S3 Zone Apex Support
• Weighted Round Robin
Highly available and scalable DNS web service.

14. Amazon S3
App 0.0v2
http://poliko.adhorn.me.s3-website-eu-west-1.amazonaws.com
Simple Static Website
Amazon
Route53 http://poliko.adhorn.me

15. • Cache content at the edge for
faster delivery
• Lower load on origin
• Dynamic and static content
• Streaming video
• Custom SSL certificates
• Low TTLs
Amazon CloudFront (CDN)

16. Amazon S3
Amazon
CloudFront
Amazon
Route53
App 0.0v3
Simple Static Website
http://poliko.adhorn.me

17. Custom backend

18. P2M4 D2 X1 G2T2 R4 I3 C5
General Purpose
GPUGeneral Purpose
Dense storage Large memory
Graphics
intensive
Memory intensive High I/O
Compute intensiveBurstable
Lightsail
Simple VPS
F1
FPGAs
Amazon Elastic Compute Cloud (EC2)
Amazon EC2 Instance Types

19. App 0.1v1
Amazon
EC2
instance
Elastic IP
User
Amazon
Route 53
EC2 backend
www.example.com
54.223.92.16

20. App 0.1v2
Docker
Container
Elastic IP
User
Amazon
Route 53
Containerized backend
www.example.com
54.223.92.16

21. Managed
API Gateway
cache
Amazon
CloudWatch
API Gateway
Endpoints on
Amazon EC2
Any other publicly
accessible endpoint
AWS Lambda
functions
Amazon
CloudFront
API Gateway
User Amazon
Route 53
App 0.1v3
Serverless backend

22. Databases

23. Self-managed Fully managed
Amazon EC2 Amazon
DynamoDB
Amazon RDS
Database options

24. NoSQL vs SQL

25. Why start with SQL?
• Easy to change your data access needs
• Established and well-worn technology.
• Lots of existing code, communities, books, and tools.
• You aren’t going to break SQL DBs in your first 10 million
users. No, really, you won’t.*
• Clear patterns to scalability.
*Unless you are doing something SUPER peculiar with the data or you have MASSIVE amounts of it.
…but even then SQL will have a place in your stack.

26. Why you might need NoSQL?
• Super low-latency applications
• Metadata-driven datasets
• Highly nonrelational data
• Need schema-less data constructs*
• Rapid ingest of data (thousands of records/sec)
• Massive amounts of data (again, in the TB range)
*Need!= “It’s easier to do dev without schemas”

27. Application
Elastic IP
Database
User
Amazon
Route 53
App 0.2
Separate the data layer

28. Separation of content type

29. Application
Elastic IP
Database
User
Amazon
Route 53
App 0.3
Separate static assets from dynamic content
Amazon S3
Amazon
CloudFront
*.js
*.jpeg
*.mp4

30. Users > 1000

31. Availability and Redundancy

32. Elastic Load Balancer
• Highly available
• 1 - 65535
• Health checks
• Session stickiness
• Monitoring / Logging
• Content-based routing
• Container-based apps
• WebSockets
• HTTP/2

33. Web
Instance
RDS DB Instance
Active (Multi-AZ)
Availability Zone Availability Zone
Web
Instance
RDS DB Instance
Standby (Multi-AZ)
Load
balancer
App 0.4
Available & redundant application
User
Amazon
Route 53
Amazon
CloudFront
Amazon S3

34. Caching layer

35. Amazon Elasticache
• Redis and Memcached Compatible
• Fully Managed
• Easily Scalable
• Transient session data
• Shared state
• High-frequency counters
• Queues
• Leaderboards
• Pub/Sub
• Lists, sets, …
In-memory data store and cache

36. Amazon
ElastiCache
RDS DB Instance
Active (Multi-AZ)
Availability Zone Availability Zone
RDS DB Instance
Standby (Multi-AZ)
ELB
User
Amazon
Route 53
Amazon
CloudFront
Amazon S3
App 0.5
Stateless application
Web
Instance
Web
Instance

37. Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Weekly traffic pattern

38. Auto Scaling

39. Auto Scaling
• Maintain your Amazon EC2 instance availability
• Automatically Scale Up and Down your EC2 Fleet
• Scale based on CPU, Memory or Custom metrics

40. RDS DB Instance
Active (Multi-AZ)
Availability Zone Availability Zone
RDS DB Instance
Standby (Multi-AZ)
ELB
App 0.6
Auto scaling groups
User
Amazon
Route 53
Amazon
CloudFront
Amazon S3
Web
Instances
Web
Instances ElastiCache
Auto-Scaling group

41. Users > 100,000

43. Databases (part 1)

44. Read / Write Sharding
RDS DB Instance
Read Replica
App
Instance
App
Instance
App
Instance
RDS DB Instance
Master (Multi-AZ)
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica

45. Database Federation
Users
DB
Products
DB
App
Instance
App
Instance
App
Instance

46. Database Sharding
User ShardID
002345 A
002346 B
002347 C
002348 B
002349 A
CBA
App
Instance
App
Instance
App
Instance

47. Users > 1,000,000

48. Asynchronous patterns

49. Message passing
A
Queue
B
A
Queue
BListener
Pub-Sub
SNS, SQS, Redis, RabbitMQ

50. Async. Architecture (part 1)
Web
Instances
Worker
Instance
Worker
Instance
Queue
API
Instance
API
Instance
API
Instance
API: {DO foo}
PUT JOB: {JobID: 0001, Task: DO foo}
API: {JobID: 0001}
GET JOB: {JobID: 0001, Task: DO foo}
ElastiCache
Result:
{
JobID: 0001,
Result: bar
}

51. Async. Architecture (part 2)
Worker
Instance
Worker
Instance
Queue
API
Instance
API
Instance
API
Instance
ElastiCache
Amazon SNS
Push Notification
User

52. RDS DB Instance
Active (Multi-AZ)
Availability Zone
Elastic Load
Balancer
Web
Instance
Web
Instance
Amazon
Route 53User
Amazon S3
Amazon
Cloudfront
ElastiCache
Worker
Instance
Worker
Instance
App 0.7
Decoupling
Queue Amazon SNS

53. Event-driven patterns

54. Event driven
A B CEvent on B by A triggers C

55. How Lambda works
S3 event
notifications
DynamoDB
Streams
Kinesis
events
Cognito
events
SNS
events
Custom
events
CloudTrail
events
LambdaDynamoDB
Kinesis S3
Any custom
Invoked in response to events
- Changes in data
- Changes in state
Redshift
SNS
Access any service,
including your own
Such as…
Lambda functions
CloudWatch
events

56. Event-driven using Lambda
AWS Lambda:
Resize Images
Users upload photos
S3:
Source Bucket
S3:
Destination Bucket
Triggered on
PUTs

57. Micro-Services

58. Amazon
Route 53User
Amazon
Cloudfront
API Edge Service
Product Listing
Service
Recommendation
Service
Any
Service
Auth.
Service

59. Databases (part 2)

60. Specialized Database
NoSQL Graph DB

61. Users = 10,000,000

62. Happy Scaling!
@adhorn

63. Frank van Boven & Roelof Reitsma | 23-05-2017

69. Business plan

71. Can we make a billion?

72. 2003

73. 2003: scaling everything
Fresh start
Not-scalable homebrew software
Not enough people
Next step: File new project

76. Big Bang release

78. 2005

80. 2010: Scaling issues
No more space
Software quality and quantity improved
Shared hosting could not extend
Server room under kitchen has not enough power
Next step: buy rack in data centre

81. 2010

84. M1
Database

85. M1
Database
Database
Database
Database
Database
Database
Database
Database
Database

86. M1
Database
Database
Database
Database
Database
Database
Database
Database
Database

87. 2013: Scaling issues
Losing control
50+ servers in data centre
Operations team managing infrastructure
Manually created, undocumented and inconsistent configuration
Next step: configuration management

88. 2013

89. Configuration management

90. M1
Database
Database
Database
Database
Database
Database
Database
Database
Database

94. 2015: Scaling issues
Ops team bottleneck
75+ servers in configuration management
Time to deliver under stress
Lot of teams dependent on ops team
Next step: use the force!

95. 2015

97. Puppet: the good, the bad & the ugly

99. Scaling for Christmas 2016
Find appropriate hardware
Get offers and approval
Buy hardware
Install hardware
Install and configure software
1 day
2 months
Ordering new hardware

100. 2016: Scaling issues
Peak scaling
Easy to roll out new systems
Complex infrastructure environment
Big difference in peak and low usage
Next step: go to the cloud

101. 2016

105. Managed Services

108. Can we make a billion?

109. 2017