Por qué los planteamientos de pruebas de rendimiento tradicionales ya no funcionan, y cuál es el acercamiento adecuado para asegurar que los sistemas soportan la carga esperada cuando pasan a producción.

1. Madrid, 26th to 28th May 2014
Reality Shock:
Performance Testing
Jose Aracil - Operations Manager
jose.aracil@globetesting.com
@pep0 - @globetesting

2. Madrid, 26th to 28th May 2014
A radically different kind of user
System-centric User-centric
Users born after 1980
Mainframe DevicesWebClient/Server
Time

3. Madrid, 26th to 28th May 2014
2014
A tipping point for app. delivery
“New application
projects surge in 2014
and beyond.”
“Social, mobile, analytics,
cloud have reached the
tipping point in 2014.”
2015
36 per app
2020
120 per app
2010
4 per app
Number of releases
per app per year
• Apps everywhere
• Accessed through multiple devices
• 30x increase in application releases
• Closed feedback loop with customers
• Focus on time-to-value and ROI
What we expect:

4. Madrid, 26th to 28th May 2014
Changes are coming faster
Cloud and composite applications are increasingly complex
Customers demand mobile access services
The pace of change is only increasing
Project and delivery teams are both agile and global
Businesses need applications to connect with:
• Customers
• Partners
• Suppliers…

5. Madrid, 26th to 28th May 2014
Dev/test lab
Developer
And the need for speed changes everything
Tester
Agile Development
“The new QA is extensively
automated and executed
by a continuous
integration process”--
Forrester Research
Provision
environments at
“agile speed”
Shift left, test
continuously
Agile
Development,
weekly drops

6. Madrid, 26th to 28th May 2014
Continuous Testing for
faster app delivery
“Shift-Left”
With Confidence
Now Continuous Testing is more critical than ever
• Improve development
efficiency
• Reduce the test burden
• Accelerate application
delivery
Design Develop
Provision
Env.
Deploy
Test App.
Run test
Test results

7. Madrid, 26th to 28th May 2014

8. Madrid, 26th to 28th May 2014
Application usage has
changed
Performance testing must change
Application development
has changed

9. Madrid, 26th to 28th May 2014
You must
leave the black
box

10. Madrid, 26th to 28th May 2014
Sorry… I changed my mind!

11. Madrid, 26th to 28th May 2014
Sorry… I changed my mind!
updated

12. Madrid, 26th to 28th May 2014
Planning Test Strategy
Reporting Final report
Preparation
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Execution
Baseline
Soak Test Load Testing Other tests
Stress
Performance Testing
Análisis+Changes+
Report

13. Madrid, 26th to 28th May 2014
Planning
Preparation
Execution
Reporting
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Final report
Performance Testing
Análisis+Changes+
Report
But what get’s in the way?
Dependencies

14. Madrid, 26th to 28th May 2014
Good for one project

15. Madrid, 26th to 28th May 2014
Good for one project

16. Madrid, 26th to 28th May 2014
New set of problems…
Multiple
projects
Continuous
Integration
Auditing
Multiple people
Deployment of
LoadGenerators
Dependencies
Asset sharing
Reporting
Security
Collaboration
Best
practices
Network
conditions
Cloud
Low application
maturity
Agile
(good and bad)

17. Madrid, 26th to 28th May 2014
Project based testing challenges
Project A
HP LoadRunner 8.1
1500 Vusers
Project B
HP LoadRunner 11.0
1500 Vusers
Project C
Other load test SW
2500 Vusers
Project D
No load test budget
“Hope for the best”
Unrealistic “code freeze”
No sharing of best practices & skills – each project is unique and isolated
Duplicated resources (HW and SW) in separate projects
Some test cases not fully working (missing components)
End-to-end behaviour not fully tester

18. Madrid, 26th to 28th May 2014
End to end
Search PayBuy Ship Track
EXAMPLE:
The Online Shopping
Process
Product PaymentCustomer Profiles Shipping
Track package
Public CloudTraditionalManaged CloudComposite
APIs enable integration
Each have SLAs and performance
dependencies
Components anywhere– in-house or cloud

19. Madrid, 26th to 28th May 2014
Virtualisation Four-legged table
User
Network
Service
Data
User
• Virtualise user conditions to simulate real
world conditions
Network
• Virtualise network conditions to simulate
production architecture and real user
conditions
Service
• Virtualise services and service interactions
to simulate E2E business processes
Data
• Virtualise production data and transaction
responses to simulate application request
and responses

20. Madrid, 26th to 28th May 2014
We are now able to truly apply a
agile approached to load testing
given the reduction in scripting and
build time.
Our scripting time is reduced by
more than 75% and resource
productivity across projects has
improved by more that 50%.
Source: TechValidate. TVID: B40-ED8-557
User

21. Madrid, 26th to 28th May 2014
Enable Agile performance Delivery
User
Test
Build Load Test scripts during Dev
Run Unit Tests in Load Test Scenarios
Add Performance Tests as build steps in
continuous integration tools

22. Madrid, 26th to 28th May 2014
Mobile is different
Analyze Mobile Performance on Network
Backend
monitoring
Mobile
performance
automation
Mobile Network
characteristics
Network
Identify bottlenecks before end
users
Test in both good and BAD
network connections
Understand impact of mobile on
entire system

23. Madrid, 26th to 28th May 2014
Existing Infrastructure
Service Virtualization Concept
Mainframe
JDBC
SOAP
RFC/IDO
C
MQ/CIC
S
Third Party
Application Under Test
Mobile
App
Web
browser
Composite
Application
API
SAP System
Existing database
Web service and
Legacy application
REST
Pay-per-transaction
Under
construction
Service

24. Madrid, 26th to 28th May 2014
Existing Infrastructure
Service Virtualization Concept
Virtual services are created to
stand in for the real service when
developers or testers need
access
Mainframe
JDBC
SOAP
RFC/IDO
C
MQ/CIC
S
Third Party
Application Under Test
Mobile
App
Web
browser
Composite
Application
API
SAP System
Existing database
Web service and
Legacy application
REST
Pay-per-transaction
Under
construction
Service
Virtualization
SOAP
REST
JDBC
MQ
RFC
CICS
Simulatio
n
Data
Perf.
Config
Service

25. Madrid, 26th to 28th May 2014
Service Virtualization is the answer
Functional test
Performance Test
Regression test
Dependencies

26. Madrid, 26th to 28th May 2014
Service Virtualization is the answer
Functional test
Performance Test
Regression test
Dependencies
Virtualization
Allow frequent access to app components, APIs and
services
Remove constraints with virtual version
No access to a component? Use virtual component
Enable performance test teams to set up simulated
services

27. Madrid, 26th to 28th May 2014
Bring production data into performance
testing as baseline
• Collect, analyze and compare
performance data from production
• Updated log file analysis of production
data for IIS
• Integrate with Google Analytics
DataPerformance application lifecycle
Value:
Performance testing closer to reality with accurate,
realistic, reliable test: reduce risk and improve test results

28. Madrid, 26th to 28th May 2014
Data
Virtualise production data and
transaction responses to simulate
application request and responses
Data

29. Madrid, 26th to 28th May 2014
3 Keys to better
performance
• Test early
• Test often
• Monitor and leverage
data from monitoring
as baseline for test
The NEW Reality
User Network Service Data

30. Madrid, 26th to 28th May 2014
Q&A

31. Madrid, 26th to 28th May 2014
IF YOU LIKE IT
PLEASE RATE ME
Visit our blog to get all the slides
WWW.GLOBETESTING.COM
jose.aracil@globetesting.com