6. From infrastructure perspective
How:
● much it utilises network bandwidth
● much requests the infrastructure can serve
● many servers it needs
● warm/loud is it at server room
7. From end user perspective
How fast the service feels.
9. Apdex Read more
Apdex.org
“Apdex is a numerical measure of user satisfaction
with the performance of enterprise applications.”
Based on three zones of application
responsiveness:
1. Satisfied
2. Tolerating
3. Frustrated
10. Satisfied Read more
Apdex.org
The user is fully productive.
This represents the time value which
users are not impeded by application
response time (“T”).
11. Tolerating Read more
Apdex.org
The user notices performance lagging
within responses greater than “T”
seconds, but continues the process.
19. Realistic user
An realistic user comes to the service:
● with specific task in mind
● might get attracted to another task
● with different devices
● having limited accessibility
● causing probably external API calls too
Best way to specify this is to check from analytics service.
20. Unrealistic user
This is not realistic:
● 1000 users load one page, or random pages
● Some tool used to run GET requests and not even loading
embedded resources such as JS/CSS and images
● Goes pages through without reading times
22. Example case: Food Standards Agency
“Our job is to use our expertise and influence so that people can trust
that the food they buy and eat is safe and honest.”
● Visitors may read news and alerts around food safety
● Guidance for business owners
● Information for visitors about food safety and hygiene
● Food hygiene ratings
● Reporting functionalities
23. User tasks in the service
● Checking hygiene rating for an restaurant
● Reading and subscribing news and alerts
● Seeking information about food hygiene and safety
● Reporting a food problem
● Giving feedback
● Finding contact details
● Finding business services
● Seeking scientific studies about certain topics
24. Visitor looking for a food rating
1. Visitor lands on a homepage
2. Clicks “Food hygiene ratings” link
3. Two segments:
○ 80% User types writes a search keyword and hits enter
○ 20% Opens advanced options and uses it along with a keyword
4. Visitor clicks to a search result
33. http://httpd.apache.org/docs/2
.4/programs/ab.html
ApacheBench (ab) Good for..
testing HTTP server
static files deliveries
and for bandwidth
testing
● Command line tool
● Originally designed for Apache HTTP server,
but widely used for other servers too
● Single-threaded (might become the
bottleneck on high concurrency tests)
● Does not have any response body parsing
features
http://httpd.apache.org
/docs/2.4/programs/ab.
html
34. Gatling Good for..
Java based web
projects.
● Written in Scala
● Licensed in Apache License 2.0
● Has recording feature/tool
● Based in France, part of France Open Source
Taskforce
● Tests written in Java
https://gatling.io/
37. Locust Good for..
projects that needs
web user interface for
controlling tests.
● Web UI
● Tests written in Python
● Supports distributed running machines, can
scale!
● “Slaves” has multiple implementations
(boomer, golang, your custom)
● License MIT
https://locust.io/
39. Molotov Good for..
performing simple
Python based test
scripts
● Tests written in Python
● Simple Python based test framework
● Uses coroutines
● Funded by Mozilla
molotov.readthedocs.io
41. LoadBooster Good for..
load testing web
applications.
● Simulates user actions at browser, not traffic
● SaaS service
● Run from NodeJS client cli
● Service scales up to 10k users
● Users CasperJS, possibility to share
behavioural and performance tests?
loadbooster.com
43. Apache JMeter Good for..
simulating traffic for
complex scenarios.
● Tests are created using GUI (Java), stored as
XML
● Has recording feature
● License Apache License 2.0
● Very backed piece of software
● Funded by lots of sponsors (Apache
Foundation’s sponsors)
jmeter.apache.org
51. Taurus
What?
Taurus is a side project
by BlazeMeter that is
aiming for continuous
testing.
● Great tool for running automation-friendly
continuous testing
● Can be provisioned locally or cloud
● Supports almost all tools mentioned in this
presentation (20 executors)
● Recommended for projects where
performance must be quaranteed (make it
part of your “Definition of Done”)
52. What?
Taurus is a side project
by BlazeMeter that is
aiming for continuous
testing.
53. What?
Taurus is a side project
by BlazeMeter that is
aiming for continuous
testing.
Picture source: gettaurus.org
54. Summary Did you like?
Please tweet and follow
us on Twitter!
@iMiksu
@Wunder_io
● Use Apdex score as part of your DoD
● Specify good scenarios for performance testing
● Run different scenarios simultaneously
● Pick the right tool, JMeter is not ideal for all
cases!
● Consider continuous testing too