1. Techniques for Minimizing Cloud Footprint
Arun Kejariwal
March 2013
1 International Conference on Cloud Engineering 2013 © Arun Kejariwal

2. Overview
 Cloud Computing
o Ubiquitous
o IDC – Cloud Services revenue ~100 B by 2016
 Infrastructure-as-a-Service (IaaS)
o Large adoption
 Elasticity
 Scale up and down
 Eliminates cycles for hardware procurement, datacenter maintenance
 Higher product development agility
 Fault Tolerance
 Geographically distributed availability zones (e.g., AWS in US, Ireland, Singapore, Japan, Brazil)
o Growing vendors
2 International Conference on Cloud Engineering 2013 © Arun Kejariwal

3. Capitalizing Cloud Elasticity
 Non-trivial to manage
o Aggressive scale down
 Potentially affect latency and throughput adversely
 Degraded end-user experience - impacts bottomline
o Aggressive scale up
 Balloon cloud footprint
 Adversely impact operational efficiency - impacts bottomline
o Which metric to use?
 Traffic
 Incoming, Outgoing
 CPU
 Load
3 International Conference on Cloud Engineering 2013 © Arun Kejariwal

4. Capitalizing Cloud Elasticity (contd.)
 Non-trivial to manage
o Scaling policy
 Whether to scale up/down by fixed amount
 Whether to scale up/down by percentage of current capacity
 What should be the cooldown period?
 Efficient exploitation of cloud elasticity: Why Bother?
o Handle high traffic
 Deliver best end-user experience
o Contain overall footprint
 Reserved vs. On-Demand instances
4 International Conference on Cloud Engineering 2013 © Arun Kejariwal

5. Key Contributions
 Algorithms
1. Scale up/down by fixed amount
 Target applications: small (< 20-25 nodes) clusters
 Case Study
2. Scale up/down by percentage of current capacity
 Target applications: medium/large clusters
3. Implications of long application start up time?
 Example application: Netflix Recommendation Engine
 Loading of user metadata (such as users preferences)
 Modified Algorithm 2 to capture effects of application start up time
 Case Study
5 International Conference on Cloud Engineering 2013 © Arun Kejariwal

6. Autoscaling in AWS: Quick review
 CloudWatch
o Real-time monitoring of EC2 instances
o Metrics
 CPU utilization, latency, network traffic, application specific etc.
o Alarm
 Change state when metric > threshold
 Action: when metric > threshold for a number of time periods
 Policy
o ScalingAdjustment
 # instances by which to scale
o Adjustment type
 ChangeInCapacity
 PercentChangeInCapacity
 Cooldown
o Allows effects of scaling activity to become visible
6 International Conference on Cloud Engineering 2013 © Arun Kejariwal

7. Techniques: Design Guidelines
 Avoid ping-pong effect
o High latency
o SLA violation
 Be proactive, not reactive
o Application start up
SLA driven need
to scale up
Autoscaling
event
7 International Conference on Cloud Engineering 2013 © Arun Kejariwal

8. Techniques: Design Guidelines (contd.)
 Aggressive upwards, conservative downwards
o Deliver best user experience
 Handle more than expected increase in traffic
 Handle slower ramp down of traffic
o Aggressive scale up
 Provides buffer for increase in traffic during the cooldown period
o Aggressive scale down
 May result in under-provisioning
 Impact user experience
 Scalability Analysis
o Given an SLA, determine maximum throughput
8 International Conference on Cloud Engineering 2013 © Arun Kejariwal

9. Techniques: Properties
 RPS -> Requests per second
1. RPS/node after scale up > Scale down threshold
2. RPS/node after scale down > Scale up threshold
9 International Conference on Cloud Engineering 2013 © Arun Kejariwal

10. Techniques: # 1
 Scale up by fixed amount
o Outline
 : Scale up threshold (RPS per node)
 : Scale up value

 Proactive
 Empirically determined
 Example applications
o Beaconserver, customer service
o It does add up!
10 International Conference on Cloud Engineering 2013 © Arun Kejariwal

11. Techniques: # 1 (contd.)
 Case Study
Aggressive upwards
Conservative downwards
11 International Conference on Cloud Engineering 2013 © Arun Kejariwal

12. Techniques: # 3
 Scale up/down by percentage of current capacity
o Account for application start up
o How to model it?
 Time series of RPS
 : Application start up time, determine rolling RPS change time series
 Compute 99th percentile
 Compute effective scale up threshold
 Consistent with being proactive design guideline
12 International Conference on Cloud Engineering 2013 © Arun Kejariwal

13. Techniques: # 3 (contd.)
Rolling RPS Change Time Series
(Application start up: 30 mins)
13 International Conference on Cloud Engineering 2013 © Arun Kejariwal

14. Techniques: # 3 (contd.)
 Scale up percentage of current capacity
 Outline
 : Scale up threshold (RPS per node)
 : Scale up value

 Proactive
 Empirically determined
 Example applications
o Merchweb, simsservice, recommendation service, API
o Savings up to 50%
14 International Conference on Cloud Engineering 2013 © Arun Kejariwal

15. Techniques: # 3 (contd.)
 Case Study
15 International Conference on Cloud Engineering 2013 © Arun Kejariwal

16. Wrapping up …
 Summary
o Improve operational efficiency in the cloud
o Benefits both small and large clusters
o Up to 50% reduction in operational footprint
 Future work
o How to handle spikes?
o Capture interaction between different services in a SOA
o Autoscale across IaaS vendors
16 International Conference on Cloud Engineering 2013 © Arun Kejariwal

17. Q&A
17 International Conference on Cloud Engineering 2013 © Arun Kejariwal