A high-level overview of Workload Modelling as a part of Performance Testing Life Cycle with focus on the challenges faced in Cloud environment relative to traditional IT infrastructure.

1. An Introduction to Workload Modelling for
the Cloud Infrastructure
0
Ravi Yogesh
Web Performance Engineer, Wells Fargo
September 12, 2017

2. What is Workload Modelling ?
 Workload modeling is an attempt to create a
generalized model that can be used to generate
synthetic workloads, using measured data from
the real system
1

3. Why to do ?
 To ensure correct scope coverage
 To simulate realistic user load pattern in
Application Under Test
 To identify performance bottlenecks
 To identify scalability of the system (1X,2X..)
 Capacity Planning to meet anticipated loads
2

4. When to do ?
 During NFR gathering for a new application
 Every major release for existing applications
3

5. How to do
4

6. How: Things to Consider - Scaling
 Production to Test Env. Scale Factor (No. of Servers)
 Hardware Configuration Scale Factor (CPU, Instances)
 Business Hours (Assume/Derive)
 Peak Volume Days (Black Fridays, Christmas)
5
PRODUCTION TEST ENV

7. How: Little’s Law
N = λ*(Rt+TT)
Where,
N is Number of users.
λ is Arrival Rate.
Rt is Response Time, TT is Think Time.
6
(

8. Workload Modelling in the Cloud:
 Need / Criticality
 Challenges over traditional infrastructure
 Solutions and way forward

9. Workload Modelling in the Cloud:
Need:
1. End to End Performance is not Guaranteed !!
2. Difficulties with virtual resource upscaling and downscaling to
accommodate workload changes (elasticity) can lead to performance
issues (risk of failed transactions/latencies for end user, agility to spin
up before crash ??)

10. Workload Modelling in the Cloud:
Need:
3. To maximize the utilization of resources and minimize running
costs while maintaining Service Level Agreements (SLAs).
 CIOs only use about half of the cloud capacity they've bought !
(An independent survey of 200 UK-based CIOs, by ElasticHost)
 Cloud Capacity worth over $ 2 billion is wasted every year on
ideal hosts.

11. Workload Modelling in the Cloud:
Challenges:
 Highly Distributed and Dynamic Infrastructure :
(variable number of servers -> difficult to assess load/machine)
 Insufficient Trace-logs for Performance Metrics (business and
confidentiality reasons)
 Hardware platforms heterogeneity (non-identical physical
resources)
 Complex Workload (resource sharing by multiple services)

12. Workload Modelling in the Cloud :
Way Forward:
• Too Many Variables ?? Automated predictive analytics backed
with AI can help by maintaining a balance between cost and
performance (3rd party tools: Stacktical, Galileo, TeamQuest)
• Application elasticity testing (Single Tenancy for thresholds vs.
Multi Tenancy Testing for elasticity and smoothness of spinning
up/down)
• AWS Tools: Trusted Advisor, Monthly Calculator
• Amazon uses ML to do capacity planning for AWS

13. 12
Questions

14. References:
1. Performance and Capacity Themes for Cloud Computing, Redpaper IBM
2. How to choose the right cloud model with a workload analysis, IBM
3. Workloads in the Clouds Maria Carla Calzarossa, Marco L. Della Vedova,
Luisa Massari, Dana Petcu, Mo’min I.M. Tabash, Daniele Tessera