Berlin QA // Because blame doesn´t fix bugs

1. Performance testing & beers
Berlin QA // Because blame doesn´t fix bugs
José Aracil // Operations Manager // Globe Testing

2. About us
We are fun!
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

3. 16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
About us Focussed on Software Testing and ALM
Knowledgeable and experienced staff
Innovation: in the forefront of our services
Obsessed with excellence and attention to
detail

4. 16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
Our services
Functional Testing Performance Testing Performance Management
Test Execution
(manual or automated)
Test management
Performance,
load,stress,
soak…
System
optimization
Availability
and
performance
Understand end
user experience
Proactive
problem
resolution
Foundation
Traceability & Control Vision & ReportingProcess Standardization
(i.e. Waterfall, Agile)
DevOps
Application Lifecycle Management
Training & Coaching Outsourcing & turnkey projects
Diagnostics
Consulting ServicesQA Tools Deployment

5. Current situation
Where we stand right now
Confidential & Proprietary

6. Application evolution
• Valuable, but growth is capped
• Costly, brittle, monolithic and proprietary
• Must change structure to evolve
• Next generations of SAP and Oracle based
on SOA
• SOA driving new custom applications and
legacy integrations
• New Web 2.0 models
• “Mashups” in the enterprise
New and Agile Business Capabilities
Shared Services / Composite Apps
Yesterday’s applications: Today’s applications:
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

7. The challenges of our modern world
Composite Applications
Service oriented applications
Modern, mobile applications
Agile & Modern business require
MODERN and FAST applications
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

8. 16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
The challenges of our modern world
Agile & Modern business require
MODERN and FAST applications
Application Unavailable
10x more performance risks
Functional defects

9. Performance Testing
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

10. Performance
• Optimal application performance is in the
experience of the end user.
Why is my application not performing?
• Looks simple, if we optimize all layers in isolation,
the resulting system should perform.
Performance is in the eyes of the beholder

11. Performance
• Optimal application performance is in the
experience of the end user.
Why is my application not performing?
• Looks simple, if we optimize all layers in isolation,
the resulting system should perform.
Performance is in the eyes of the beholder

12. Performance
• Optimal application performance is in the
experience of the end user.
Why is my application not performing?
• Looks simple, if we optimize all layers in isolation,
the resulting system should perform.
Performance is in the eyes of the beholder

13. Probably a MEMORY leak…
or may be a CODE issue…
I guess it is NETWORK!
Does your application have
performance issues?

14. It is probably a memory
leak issue!
No, it may be a CODE
issue!
I guess is Network
related!
We are not ready
for the 100, 1000,
100000 users!
We SHOULD have
TESTED
Performance!

15. How HP LoadRunner Works
5 Analysis, diagnostics and reporting
DatabaseInternet/WAN
Middle Tier
(HTTP, WebSphere, Server, Network)ApplicationEnd user
Thousands of users
are simulated
Transaction recording
“Transfer
Funds”
“Check Balance”
“Pay Bills”
Scripting1
Test
Scenario
Test planning2
Load generation3 Monitorization4
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

16. Planning
Recipe for success
Test Strategy
•Set expectations
•Project planning
•Environment description
•Responsible people
•Test scenario definition
•Test case definition
•Monitors to be used
•“Live” document
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

17. Planning
Preparation
Test Env. + Data
Recipe for success
Test Strategy
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
•Env. Setup (pre-prod)
•Data load to
resemble prod.

18. Planning
Preparation
Tool setup
Recipe for success
Test Strategy
Test Env. + Data
•Performance testing
tools
•Monitorization tools
•Other tools
(reporting,…)
16/08/2013

19. Planning
Preparation
Test case design
•Transactions
•Image and text
checks
•Test data
Recipe for success
Test Strategy
Tool setupTest Env. + Data
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

20. Planning
Preparation
Monitor config.
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
•Select any
“interesting” metrics
that should be
monitored
•Identify advanced
metrics
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

21. Planning
Preparation
Scripting
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config.
•Turn test cases into
automated scripts
•Complexity changes
depending on the tool
used
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

22. Planning
Preparation
Smoke Test
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
•Check that previous
phases have been
done properly
•It doesn´t add value
to carry on if there are
pending items
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

23. Planning
Preparation
Execution
Baseline
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
•Obtain a standard for
comparison
•If performance is not
good, it doesn´t add
value to carry on with
other test
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

24. Planning
Preparation
Execution
Performance Testing
•Iterative process
•Minimum 2 iterations
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

25. Planning
Preparation
Execution
SOAK Test
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Performance
Testing
Análisis+Changes+
Report
Other tests
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
Load Testing

26. Planning
Preparation
Execution
Stress
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Performance
Testing
Análisis+Changes+
Report
•Optional test
•Only if previous test
are successful
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

27. Planning
Preparation
Execution
Reporting
Final report
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Performance
Testing
Análisis+Changes+
Report
•Summary of the test
executed, problems
found and solutions
applied
•Too much detail
doesn´t add value to
the project
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

28. Planning
Preparation
Execution
Reporting
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Final report
Performance
Testing
Análisis+Changes+
Report
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing

29. Planning
Preparation
Execution
Reporting
Final reportStressSOAK TestPerformance TestingBaselineSmoke TestScriptingMonitor config.Test case designTool setupTest Env. + Data
•Transactions
•Image and text
checks
•Test data
•Iterative process
•Minimum 2 iterations
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Test Strategy
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Final report
Performance
Testing
Análisis+Changes+
Report
•Set expectations
•Project planning
•Environment description
•Responsible people
•Test scenario definition
•Test case definition
•Monitors to be used
•“Live” document
•Performance testing
tools
•Monitorization tools
•Other tools
(reporting,…)
•Select any
“interesting” metrics
that should be
monitored
•Identify advanced
metrics
•Turn test cases into
automated scripts
•Complexity changes
depending on the tool
used
•Check that previous
phases have been
done properly
•It doesn´t add value
to carry on if there are
pending items
•Obtain a standard for
comparison
•If performance is not
good, it doesn´t add
value to carry on with
other test
Other tests
•Optional test
•Only if previous test
are successful
•Summary of the test
executed, problems
found and solutions
applied
•Too much detail
doesn´t add value to
the project
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
Load Testing•Env. Setup (pre-prod)
•Data load to
resemble prod.

30. Thank you for your time
José Aracil // jose.aracil@globetesting.com
Confidential and Propietary // All rights reserved // Globe Testing