In response to increasing market demand for high performance applications, many organizations implement performance testing projects, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. With specific examples from recent client implementations, Scott Barber shares the fundamentals of implementing T4APM™ a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The T4APM™ approach hinges on applying a simple and unobtrusive "Target, Test, Trend, Tune” cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.

1. MD
AM Tutorial
9/30/2013 8:30:00 AM
"Application Performance
Testing: A Simplified Universal
Approach"
Presented by:
Scott Barber
PerfTestPlus, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Scott Barber
PerfTestPlus, Inc.
CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system
performance and testing, Scott Barber describes himself as a “tester, author, speaker, disrupter,
and dad.” Scott is the author of Web Load Testing for Dummies and co-/contributing author of
Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the
Cost of Testing.

3. 8/27/2013
Managing Application Performance:
A Simplified Universal Approach
1
Managing Application Performance:
A Simplified Universal Approach
Created for:
By:
Scott Barber
Chief Performance Evangelist
SmartBear Software, Inc.
1

4. 8/27/2013
Chief Performance Evangelist
SmartBear Software, Inc.
scott.barber@smartbear.com
@sbarber
Co-Founder: Workshop On Performance and Reliability
www.performance-workshop.org
Author:
Co-Author:
Contributing Author:
Books: www.perftestplus.com/pubs
About me: about.me/scott.barber
What is Performance?
System or application characteristics related to:
Speed:
- responsiveness
- user experience
Scalability:
- capacity
- load
- volume
Stability:
- consistency
- reliability
- stress
2

5. 8/27/2013
What is a Performant System?
A system optimized to balance:
- Cost
- Time to Market
- Capacity
…while remaining focused on:
Quality of Service to System Users
What is Performance Testing?
Testing designed to:
Inform software system optimization by
balancing:
- Cost
- Time to Market
- Capacity
while remaining focused on the quality of
service to system users.
3

6. 8/27/2013
Performance vs. Load Testing?
The Performance Lifecycle is:
Conception to Headstone
Not
Cradle to Grave
4

7. 8/27/2013
Prevent Poor Performance…
… don’t just react when it happens.
© PerfTestPlus, Inc.
Who is Responsible?
5

8. 8/27/2013
© PerfTestPlus, Inc.
Where does it fit in the SDLC?
6

9. 8/27/2013
Delivering Performant Systems;
Commonly, Y2K -> Today
SPE (Software Performance Engineering)
Math-centric Architect/Design method
+ Profiling
Context-free code optimization
+ Load Testing
Demonstrate (too late) poor performance characteristics
+ Capacity Planning
Math-centric guessing to determine hardware needs
+ APM (Application Performance Management)
Proactively react to production issues
=> Inefficient & Reactionary Delivery & Maintenance
Delivering Performant Systems;
Moving Forward
Minimize Need for Complex Mathematics
Few teams have mathematicians w/ advanced degrees
Disconnect Accuracy from Empirical Production Data
Because we can’t depend on what we don’t have yet
Remove Socio-Political Barriers
Collaboration improves information sharing & efficiency
Keep/Adapt most Valuable Practices
Many good concepts have been poorly implemented
Distribute Valuable Practices Throughout Life Cycle
Many valuable practices have suffered from stove-piping
I’m testing a model for the entire ‘Performance Journey’
7

10. 8/27/2013
Performant Application Delivery
© PerfTestPlus, Inc.
Key Points
•
•
•
•
Entire Lifecycle
Entire Team Focused
Individual, Role, & Team Accountability & Responsibility
Continuous & Collaborative
“DevOps”
•
•
•
•
•
•
•
Code Profiling
Requirements Revision
Design Validation
Budgets
Units & Components
Monitoring
Tuning
Perf Team
•
•
•
•
•
Load/Stress Testing
Environment Config
Capacity Planning
Concurrency Checks
Situation Analysis
Individual
•
•
•
•
Snapshots
Comparisons
Niche Concerns
Mini-Field Studies
(UAT)
• “What-if…”
© PerfTestPlus, Inc.
8

11. 8/27/2013
Preventing Poor Performance with
a little work…
every day…
from every one.
© PerfTestPlus, Inc.
Proactive
Micro & Macro
Establish Goals
Update Targets
Dev & Prod
Times
Resources
Sizes
Frequencies
Dashboard!
Units
Stories
Tiers
Resources
Goals
Assess
Compare
Investigate
Accept
Answer
© PerfTestPlus, Inc.
9

12. 8/27/2013
© PerfTestPlus, Inc.
Unit-Level Testing Tools
(‘cause folks always ask)
FireBenchmarks; Performance testing addin for NUnit
JUnitPerf; a collection of JUnit test decorators for performance
Firefox Performance Tester's Pack
HTTPerf
10

13. 8/27/2013
© PerfTestPlus, Inc.
An Ounce of Prevention…
11

14. 8/27/2013
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
12

15. 8/27/2013
What is it?
Load Tests
Stress Tests
Endurance Tests
Infrastructure/Configuration Verification
Load Variability Analysis
Background Load for Acceptance Tests…
Or any other deliberate approximation of
Production Conditions
© PerfTestPlus, Inc.
13

16. 8/27/2013
RPT is:
Inspired by Rapid Software Testing
Consistent with Rapid Software Testing themes
Sanctioned by James Bach, Michael Bolton & the RST
instructors to as a specific implementation of the
Rapid Testing Methodology
For more information about RST, visit:
http://www.satisfice.com/info_rst.shtml
Evolved from:
“What have we got?
What do we want?
How do we get there…?”
--Bob Barber (Scott’s dad)
… as quickly, simply, and
cheaply as possible?
14

17. 8/27/2013
…a common man’s way of
expressing the problem solving
approach that classical engineers
employ.
• Given: “What have we got?”
• Find: “What do we want?”
• Solve: “How do we get there?”
What is it?
An approach to respond to a specific
performance-related question after 4 or fewer
hours of team effort with 1 or more of:
A) The answer
B) A partial answer
•
•
To determine the value of additional effort
The level of effort to provide the answer
C) Better questions to address the underlying
concern
15

18. 8/27/2013
Conceptual Approach
1. Receive Question
•
•
2.
3.
4.
5.
6.
Clarify the question
Understand the driver(s) behind the question
Generate Test Coverage Outline (TCO) (~20 min)
•
Simplest path to (partial) answer(s)
•
Comprehensive path to (partial) answer(s)
Transform TCO into Rapid Strategy (~20 min)
•
Only tasks that fit in time box
•
Stick to tasks requiring available resources
Execute Strategy (~2.5 hrs)
•
Snapshots are your friends
•
Anecdotal is sufficient
Consolidate/Analyze Data (~30 min)
•
Identify patterns
•
Confirm patterns (time permitting)
Report Results (~20 min)
•
Answer(s) / Time or Effort to get answer(s)
•
Follow-on questions of interest
~4 hr
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
16

19. 8/27/2013
© PerfTestPlus, Inc.
The Bottom Line
© PerfTestPlus, Inc.
17

20. 8/27/2013
Addendum
Questions?
18

21. 8/27/2013
Contact Info
about.me/scott.barber
Chief Performance Evangelist
SmartBear Softwear, Inc.
E-mail:
Scott.barber@smartbear.com
Web Site:
SmartBear.com
Blog:
scott-barber.blogspot.com
Twitter:
@sbarber
19