Zijn performantietesten noodzakelijk voor je project? Hoe begin je eraan? Welke kennis, tools en ervaring heb je nodig of moet je inhuren? Verschilt de aanpak voor een online applicatie van een backoffice applicatie? Wat gedaan als de performantie onvoldoende is? Wouter Soubry, Software Quality & Application Performance Consultant bij Axxes, maakt in deze presentatie komaf met al deze pitfalls bij Performance Testing.

1. Click
to
edit
Master
/tle
style
Pi2alls
in
Performance
Tes/ng
June
2015
Wouter
Soubry
1
#AxxesCC

2. Why
are
we
here?
2
#AxxesCC

3. 3
Load
Tes/ng
Failures
–
Taxonweb.be
#AxxesCC

4. 4
Load
Tes/ng
Failures
–
healthcare.gov
#AxxesCC

5. Technical
point
of
view:
Single
user
performance
Acceptable
slowdown
under
load/stress
Not
only
websites!
Every
project
should
consider
performance
explicitly
5
What
is
(applica/on)
performance?
#AxxesCC

6. hWp://survey.neuronsparks.be
hWp://splunk.neuronsparks.be/en-­‐US/app/search/speed_survey
6
Experiment
#AxxesCC

7. “The
site
needs
to
be
sufficiently
fast”
“Page
must
load
in
4s”
PDF
genera/on
should
not
take
>
30s
The
site
must
be
able
to
handle
1000
users
Trading
system
must
be
able
to
do
1000
TPS
7
Performance
Requirements
#AxxesCC

8. Pi2all
1
Wrong,
ambiguous
or
non-­‐existent
performance
requirements
8
#AxxesCC

9. Specific:
200
concurrent
users,
scenario
walkthrough,
4s
@
95
percen/le
response
all
pages
upon
first
/me
visit,
pc
on
dsl
Measurable:
at
the
client
side,
with
preload
Achievable:
‘heavy’
page
in
200ms?
Relevant
Above
the
fold
load
/me
“Onload”
event
versus
speed
index
Time-­‐bound:
golive,
gradual
adop/on,
campaign
start
9
“SMART”
Performance
Requirements
#AxxesCC

10. Throughput,
load
envelope
Response
/me
User
behaviour
(think
/me,
flow)
Device,
connec/on
Context:
/cket
sale,
tax
deadline
10
Aspects
of
Performance
Reqs
#AxxesCC

11. Staging
Produc/on
11
#AxxesCC

12. Pi2all
2
Linear
extrapola/on
12
#AxxesCC

13. Scalability
needs
to
be
built-­‐in
(and
tested)
What
is
the
uncertainty
of
your
expected
load?
Is
scalability
important?
How
fast
can
you
provision?
Business
impact
of
out
of
spec
performance?
13
Avoiding
Surprises
#AxxesCC

14. 14
Invite
your
Friends?
#AxxesCC

15. 15
Tooling
#AxxesCC

16. 16
L’Embarras
du
Choix
ApacheBench
?#AxxesCC

17. 17
Tooling:
Cost
/
ROI
#AxxesCC
/

18. Pi2all
3
A
fool
with
a
tool
is
s/ll
a
fool
(Grady
Booch)
18
#AxxesCC

19. Simple
>
complex
Understand
what
the
tool
is
doing
What
do
the
metrics
mean
What
is
measured
exactly
Tools
are
no
subs/tute
for
thinking,
but…
Good
repor/ng
helps
Intui/ve
interface++
Find
similar
use
cases
–
how
did
they
do
it?
19
Choose
Your
Weapon
Wisely
#AxxesCC

20. Simple
>
complex
Demo
with
ApacheBench
+
nmon
20
Choose
Your
Weapon
Wisely
#AxxesCC

21. 21
Load
Test
Recording
#AxxesCC
Proxy
Interceptor
JMeter
recording
demo

22. Generate
load
Measure
end-­‐user
perf
“RUM”/synthe/c
Transac/on
breakdown
Code-­‐level
profiling
Infrastructure
monitoring
Business
analy/cs
22
Tool
Categories
}
APM
#AxxesCC

23. SaaS
‘cloud’
load
genera/on
Blazemeter,
loadstorm,…
APM
suites
New
Relic
AppDynamics
Dynatrace
…
Big
(machine)
data
Splunk,
Loggly,
logentries,
Sumologic,
Logstash/elk
Monitoring
infrastructure
23
Tool
Convergence
#AxxesCC

24. Domain
knowledge
System
administra/on
Networking
exper/se
Database
adminstra/on
Applica/on
stack
knowledge:
frameworks
Tools
knowledge
Performance
tes/ng
knowhow
(+
some
sta/s/cs)
24
Needed
Skills
#AxxesCC

25. 25
#AxxesCC

26. Pi2all
4
Not
leveraging
in-­‐house
exper/se
26
#AxxesCC

27. Large
bank
online
security
overhaul
Compliance
log
subsystem
27
A
Tale
from
the
Trenches
#AxxesCC

28. Pi2all
5
Premature
or
late
op/miza/on
28
#AxxesCC

29. End-­‐user
performance
sufficient?
Transac/on
drilldown
Iden/fy
&
eliminate
boWleneck
29
Chasing
the
BoWleneck
#AxxesCC

30. Week
before
golive
Performance
tes/ng
completed,
green
light
Func/onal
freeze
Cri/cal
bugfixes
only
Launch
goes
30
Recipe
for
Disaster
#AxxesCC

31. Pi2all
6
No
con/nuous
profiling
31
#AxxesCC

32. Regression
tes/ng
performance
CI
tools:
Jenkins,
Atlassian
Bamboo,
TFS,
…
Webpagetest/sitespeed.io
Single-­‐user
performance
Performance
budget
Speed
index
Demo:
Jmeter/Jenkins
32
#AxxesCC

33. Integra/on
environment
Not
produc/on
representa/ve
Regression
detec/on
Rule
out
startup
effects
Iden/fy
cri/cal
measurements
HTTP
APIs
DB
transac/ons
XML
processing
33
Jenkins
and
JMeter
#AxxesCC

34. Pi2all
7
No
proper
configura/on
management
34
#AxxesCC

35. Annota/on
overlays
on
metrics
hWp://play.grafana.org/dashboard/db/annota/ons
What
to
annotate:
Deploys
Config
changes
Anything
with
poten/al
performance
impact
Newrelic
deployments
demo
35
What
introduced
the
problem?
#AxxesCC

36. It’s
about
the
user.
The
User.
Tools:
use
what
you
know
best,
simpler
=
beWer
Monitoring:
develop
a
(healthy)
data
addic/on
Config
management
hygiene
Knowledge:
stand
on
the
shoulders
of
giants
36
Key
Takeaways
#AxxesCC
Thank
You