Contenu connexe
Similaire à Monitor-Based Testing of Elastic Cloud Computing Applications (20)
Monitor-Based Testing of Elastic Cloud Computing Applications
- 1. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Monitor-Based Testing of
Elastic Cloud Computing
Applications
Michel Albonico
PhD Student - AtlanMod - EMN (Nantes, France)
(michel.albonico@inria.fr)
Jean-Marie Mottu
Gerson Sunyé
1
5thInt.WorkshoponLargeScaleTesting
Delft,Netherlands-2016
- 3. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
3
● Cloud computing elasticity:
The ability of a cloud infrastructure/system modifying its resource
configuration according to demand.
- 4. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
4
● Thresholds:
○ Scale-out threshold: maximum resource usage, e.g., 80% of CPU
usage;
○ Scale-in threshold: minimum resource usage, e.g., 20% of CPU
usage;
○ Used to decide when varying a resource.
Resource Allocation
Resource Demand
Scale-out Threshold
Scale-in Threshold
Scale-out Threshold Breaching
Scale-in Threshold Breaching
Time (s)
Resource
(Processors)
1
Legend
80% 0.8
20% 0.2
- 5. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
5
● Resource demand varies according to workload variations.
○ Example:
■ number of users increases from 1 to 2, the resource
demand doubles.
Resource Allocation
Resource Demand
Scale-out Threshold
Scale-in Threshold
Scale-out Threshold Breaching
Scale-in Threshold Breaching
Time (s)
Resource
(Processors)
1
Legend
80% 0.8
20% 0.2
- 6. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
6
● Resource demand varies over time;
● Scale-out threshold breaching.
Resource Allocation
Resource Demand
Scale-out Threshold
Scale-in Threshold
Scale-out Threshold Breaching
Scale-in Threshold Breaching
Time (s)
Resource
(Processors)
1
Legend
scale-out threshold breaching
80% 0.8
20% 0.2
- 7. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
7
● Resource demand varies over time;
● Scale-out threshold breaching;
● Scale-out reaction time;
Resource Allocation
Resource Demand
Scale-out Threshold
Scale-in Threshold
Scale-out Threshold Breaching
Scale-in Threshold Breaching
Time (s)
Resource
(Processors)
1
Legend
scale-out reaction time
80% 0.8
20% 0.2
- 8. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
8
● Resource demand varies over time;
● Scale-out threshold breaching;
● Scale-out reaction time;
● Scale-out time, then the thresholds are updated.
Resource Allocation
Resource Demand
Scale-out Threshold
Scale-in Threshold
Scale-out Threshold Breaching
Scale-in Threshold Breaching
Time (s)
Resource
(Processors)
2
1
80% 0.8
Legend
scale-out time
80% 1.6
20% 0.2
20% 0.4
- 9. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
9
● Scale-in:
○ Scale-in threshold breaching;
○ Scale-in reaction time (resource is no longer available);
■ Thresholds reconfiguration.
○ Scale-in time.
Resource Allocation
Resource Demand
Scale-out Threshold
Scale-in Threshold
Scale-out Threshold Breaching
Scale-in Threshold Breaching
Time (s)
Resource
(Processors)
2
1
Legend
scale-in time
scale-in reaction time
80% 0.8
80% 1.6
20% 0.2
20% 0.4
- 11. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
● Elasticity states transition.
● Related work only test during the ready state;
● Scaling states:
○ Considerable time: in our experiments, scaling-out takes more
than 90 seconds (Amazon EC2);
○ Great part of the adaptation tasks: replication data, leader
election, etc.
Motivation
11
- 12. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Test Procedure
● Test cloud systems during all the elasticity states;
● Execute tests dynamically:
○ Associate test cases to a set of elasticity states;
○ Execute the test according to the current elasticity state.
● Test execution:
○ Periodically monitor the resource during the test execution;
■ Current elasticity state.
○ (Re)-execute the associated test cases during the current
elasticity state.
12
- 13. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
● Research questions (answered by the experiments):
1. Is it necessary to run the test during different elasticity states?
a. Does a cloud system react distinctly depending on the
elasticity state?
2. Is it possible to execute the test during different elasticity states
and to assign the test verdicts accordingly?
13
- 14. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
Question 1: system behavior during different elasticity state.
● First experiment:
Measure the performance of a cloud system during different
elasticity states.
○ Manually executed;
○ Workload (50% read / 50% write);
○ 2500 operations per second (ops).
14
- 15. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
● First experiment results:
○ 2000 ops: covers all the performance drops;
○ Elasticity states extracted from the log files;
RQ1:
It is necessary to run the test during different elasticity states.
15
Performance-OperationsperSecond(ops)
Minimal Performance
Measured Performance
R R R R R RSISISO SO SOSI
200
400
1000
1200
800
600
Time (s)
- 16. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Question 2: test execution during different elasticity states + test
verdicts assignment.
● Second experiment: (same workload)
○ We use our test procedure;
○ We monitor the elasticity states throughout the test
execution;
○ Test Case:
■ answered operation >= 2000 ops -> pass
■ otherwise -> fail
○ Same test case associated to every elasticity state.
■ Test case re-executed throughout the cloud system
execution.
Experiments
16
- 17. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
● Result of the second experiment:
○ Test through different elasticity states;
○ Assign test verdicts to different elasticity states;
■ Proportional to the previous experiment (correct elasticity
states).
RQ2:
It is possible to execute the test according to the elasticity state, and
we are able to assign the test verdicts correctly.
17
- 18. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Conclusion and Future Work
● Identify all the performance problems;
● Assign the test verdicts to the correct elasticity states (at
runtime);
● Address the scaling states, which are not addressed by related
work;
● Future work:
○ Write functional test cases;
○ Apply to other study cases;
○ Generate test cases based on elasticity states.
18
- 19. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Monitor-Based Testing of
Elastic Cloud Computing
Applications
Michel Albonico
PhD Student - AtlanMod - EMN (Nantes, France)
(michel.albonico@inria.fr)
Jean-Marie Mottu
Gerson Sunyé
19
5thInt.WorkshoponLargeScaleTesting
Delft,Netherlands-2016