Short talk about solutions for scalable applications.

1. !
"
#
$
%
&
'
$./run
{
Created by
Fabrício Epaminondas ©2015-2020
Designing
SCALABLE
APPLICATIONS
Short Talk

2. WHAT IS
SCALABILITY?
The concept

3. “Scalability is the capability of a system, network,
or process to handle a growing amount of work, or
its potential to be enlarged in order to
accommodate that growth.”
Bondi, André B. (2000). Characteristics of scalability and their
impact on performance.
3

4. Let's go deeper…
4

5. 5Scalability for sandwiches
How hungry are you?
A bigger sandwich may fit your need
(vertical scaling)

6. 6

7. 7
How many people are hungry?
Think about it…

8. 8
Fast-food restaurant CHAINS
You need to scale the entire solution
(horizontal scaling)

9. Scaling Computing Resources 9
• Processing (CPU)
• Memory
• Storage
• Network
Scale-up vs Scale-out

10. Vertical Scaling (Scale-up) 10
Generally refers to adding more processors and RAM,
buying a more expensive and robust server.
• Pros
•Generally less challenging to implement
•Less power consumption than running multiple servers
•Less licensing costs
• Less network hardware than scaling horizontally.
• Cons
• Hardware costs
•Greater risk of hardware failure causing bigger
outages
•Generally severe vendor lock-in and limited
upgradeability in the future

11. Horizontal Scaling (Scale-out): 11
• Generally refers to adding more servers with less
processors and RAM. This is usually cheaper overall and
can possibly scale infinitely.
• Pros
•Much cheaper than scaling vertically
•Easier to run fault-tolerance
•Easy to upgrade
• Cons
•More licensing fees
•Bigger footprint in the Data Center
•Higher utility cost (Electricity and cooling)
•Possible need for more networking equipment
(switches/routers)

12. Related concepts 12
• Performance
•Average Response Time (ART)
•Latency (usually network and disk I/O)
•Resource consumption
• High Availability
•Single Point of Failure (SPOF)
•Fault tolerance

13. USE CASE
Weather Forecast
Designing the software

14. 14
• Mobile App to show world-
wide weather forecasts
• You planned to reach 1k users
at first week.
• Important notes:
•How many users will access
the system at same time?
•How to model workload?
•How to predict the future?
Weather Forecast App

15. 15
• Use performance tests based on scenarios.
•Test types:
• Performance
•Validates the ART on a scenario (e.g.: systems
response takes 3secs with 100k users)
• Load
• Aims to understand required resources for running
the system (planning infrastructure).
• Stress
•Look for system failures under stress conditions (DoS,
outages, etc.)
Notes

16. 16Solution #1: Two Tier Architecture
(O )
Client Server
Internet
4Gb Mem
1 vCPU*
* This virtual CPU unit represents the processing power of a CPU with 2 cores

17. 17Solution #1: Two Tier Architecture
• Pros
• Easy setup
• Cons
•Single point of failure (1 SPOF)
• Vertical scaling costs (buy better hardware,
CPU, etc.)

18. 18
• Your app is growing fast and
now it reached 10k users
after one month.
Weather Forecast App

19. 19Solution #2: Three Tier Architecture
O )
Client
Web
Server
Data
Server
( 
4Gb Mem
1 vCPU
4Gb Mem
1 vCPU
Network Latency
Internet
Local/Private Network

20. 20
• Pros
• Doubled the computing power
• Dedicated and optimized infrastructure (disk,
memory, etc.)
• IT operations specialists (database, server
management).
• Cons
• More points of failure on server side (2 SPOFs)
•Vertical scaling costs
• More network latency
Solution #2: Three Tier Architecture

21. 21
• Forecast app is being used
in most important country
cities and it has at about 500k
users after two months.
•Some users are noticing
failures and outages while
trying to get the latest weather
information.
Weather Forecast App

22. 22Solution #3: Clustering
O )Client
Web
Cluster
Data
Server
( 
4Gb Mem
1 vCPU
4Gb Mem
1 vCPU
Internet
Local/Private Network
(
Load
Balancer
+

23. 23
• Pros
•High availability and fault tolerance
•Grid computing
•Out-of-box setup scaling for stateless applications
• Cons
•IT management more complex, e.g.: Deployment and
rollback procedures.
•Need to use a data sharing service for stateful
applications or use sticky sessions which may not
scale properly
•If Data Server is the bottleneck, you may also need a
cluster for it.
•More network latency
Solution #3: Clustering

24. 24Solution #3: Stateful vs Stateless
O
)
User2
Web
Cluster
Data
Server
( Internet
Local/Private Network
(
Load
Balancer
A
B
OUser1
Session 1
Session 2
Web session stickiness example
+

25. 25
• After a global market
campaign, some users
started to complain about a
slow UI loading in mobile
application, when opening
the latest weather
information from the internet.
Weather Forecast App

26. 26
• Example of browser caching
Solution #4: Caching
[1] Reference: https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#cache-control

27. 27Solution #4: Caching
O
)
User2
Web
Cluster
Data
Server
(

Internet +
(
Load
Balancer
OUser1
(
Cache
Server
In Memory
Caching
Local
Caching
Local
Caching
Client
Caching
Client
Caching
Proxy
Caching
In Memory
Caching
Example: CDN for Static Web Content: html,
css, images, etc.

28. 28Solution #4: Caching
Content Delivery Network(CDN)

29. 29
• Pros
•Reduce wait time caused by latency problems
•Avoid waste of computing resources by caching long-
running operations like remote calls, disk I/O, etc.
•Cons
• Need to define metadata for control the cache storage.
•May require to minimal hardware for getting better
results (SSD, RAM, etc.), some devices such as mobile
have critical limitations.
• You may see out-of-date results while the old cache
was not invalidated yet.
Solution #4: Caching

30. 30
• Your app is growing too fast,
it is known throughout the
world with more than 1
million users in two month
only.
•It is time to have your product
running in a cloud solution.
Weather Forecast App

31. 31Solution #5: Full Distributed Cloud Service

32. Comments 32
,

33. !
"
#
$
%
&
'
$./run
{
Created by
Fabrício Epaminondas ©2015-2020
Contact
FABRICIO EPAMINONDAS
- linkedin.com/in/fabricioepa
. fabricioepa.wordpress.com
/ fabricioepa
0 @fabricioepa