Talk held at the conference Coding Serbia in Novi Sad. Performance of web applications is a crucial dissatisfier for users and thus an important quality criteria -- also used by Google to rank their result lists. As with other quality aspects, performance testing cannot be done at the end of a project but is an integral part of the development process. The practice presentations submitted explains web performance testing along practical examples in order to better understand and judge cause and effect of behavior observed. Usually few causes have a disproportionate effect on bad performance. In addition, it is important to understand diverse load and test scenarios to optimize application behavior. The presentation also introduces a methodology to systematically define and assess performance metrics of an application. The content is based on open source tools and the presentation includes live testing to illustrate the excellent cost benefit ratio of systematically white box testing of performance using an HTTP proxy.

1. Coding Serbia.
Systematic Load Testing
of Web Applications.
Jürg Stuker. CEO. Partner.
October 9, 2015

2. Namics.Oct. 9, 2015 2 Coding Serbia. Systematic Load Testing of Web Applications.
Performance Tuning 1%
Load Test Basics (Client View) 25%
My Experience 20%
Questions & Answers 15%
Tool Demo 39%
% Daily Value*
Nutrition Facts
Serving Size about 45 Minutes
No significant source of Business Blah Blah.
* Percent Daily Values are based on a tech diet.

3. Namics.
Please shoot!
 The sum of the expertise of the people in the audience is
greater than the sum of expertise of the people on stage,
so please…
#WebLoadTest
@jstuker
Oct. 9, 2015 3 Coding Serbia. Systematic Load Testing of Web Applications.

4. Why, When and Where?
Oct. 9, 2015 4 Coding Serbia. Systematic Load Testing of Web Applications.

5. Namics.
Why should I test?
 Load capacity and performance
 How many users can the application serve (in parallel)?
 How long does it take to deliver pages?
 How much bandwith does the system use?
 Stability
 Over time  Memory leaks & overflows
 Under load  Concurrency & deadlocks
 Fittness of Infrastructure
 Reproduction of problems in order to fix them
Why, When and Where?
Oct. 9, 2015 5 Coding Serbia. Systematic Load Testing of Web Applications.

6. Namics.
When to think about testing?
Why, When and Where?
Oct. 9, 2015 6 Coding Serbia. Systematic Load Testing of Web Applications.
Requirements
Proof of
Concept Development Going Live Operations
Specification needs
• Concurrent Users
• User Stories
• Reponse Time
Test of
Frameworks
Test of overall
Application
Integration /
Deployment
At each
Release / Fix

7. Namics.
Where to test?
Why, When and Where?
Oct. 9, 2015 7 Coding Serbia. Systematic Load Testing of Web Applications.
 Client
 Traffic Shaper / Firewall
 LoadBalancer
 Reverse Proxy
 Webserver
 Frontend Server
 Application Code
 Backend Server
 Application Code
 Data Persistance
 Application Code
1. Do not test the
environment first
2. Treat application
as a black box
3. When there is an issue
• Look into black box
• Change point of
measurement

8. Oct. 9, 2015 8 Coding Serbia. Systematic Load Testing of Web Applications.
Tools

9. Namics.
Very many… – what we use regularly (1/2)
 ApacheBench
 Hammer on application
 http://httpd.apache.org/docs/2.2/programs/ab.html
 Apache JMeter
 Loadtest workhorse, weak usability
 http://jmeter.apache.org/
 Gatling
 Loadtest modern, easy to integrate
 http://gatling.io/
 Dynatrace
 Application monitoring, white box, very powerful
 http://www.dynatrace.com/
Tools
Oct. 9, 2015 9 Coding Serbia. Systematic Load Testing of Web Applications.

10. Namics.
Very many… – what we use regularly (2/2)
 Proxy Sniffer
 http://www.proxy-sniffer.com/ or
https://www.apicasystem.com/load-testing/tools/
 Originally built in Switzerland by David Fischer,
now owned by Apica Systems
 Free Version (20 virtual users, duration 10 minutes)
 Browser based, just needs > Java 1.5 (runs on Unix-like,
Mac and Windows)
 Verrrrrry flexible (scriptable, cloud and small footprint)
 Nice documentation and reporting
Tools
Oct. 9, 2015 10 Coding Serbia. Systematic Load Testing of Web Applications.

11. Namics.
Components of Proxy Sniffer Application
 Just two files: prxsniff.jar and prxsniff.key
Tools
Oct. 9, 2015 11 Coding Serbia. Systematic Load Testing of Web Applications.
port 7990
WebAdmin / GUI
port 7993
Exec Agent
port 7999
HTTP Proxy
recording
execution

12. Namics.
Sequence of a Test with Proxy Sniffer
Tools
Oct. 9, 2015 12 Coding Serbia. Systematic Load Testing of Web Applications.
record scenario
validate recording
(clean it up)
generate test
(java code)
compile test
(class file)
execute
interpret
tweak (one thing
at the time)

13. Oct. 9, 2015 13 Coding Serbia. Systematic Load Testing of Web Applications.
Let‘s do some work

14. Namics.
A word of caution
Load testing generates load ;-)
Let‘s do some work...
Oct. 9, 2015 14 Coding Serbia. Systematic Load Testing of Web Applications.

15. Namics.
Live Demo or Recording ;)
 https://youtu.be/0Akj5qw8_So
Let‘s do some work...
Oct. 9, 2015 15 Coding Serbia. Systematic Load Testing of Web Applications.

16. Oct. 9, 2015 16 Coding Serbia. Systematic Load Testing of Web Applications.
Interpreting the Outcome

17. Namics.
Linear Water Channel
 More water == more througput
 Pass through time is (close to) constant
Interpreting the Outcome
Oct. 9, 2015 17 Coding Serbia. Systematic Load Testing of Web Applications.
Throughput: Liters per minute
Pass through time

18. Namics.
Unlimited Linear Model
Interpreting the Outcome
Oct. 9, 2015 18 Coding Serbia. Systematic Load Testing of Web Applications.
Water per minute
Throughput:
Liters per minute
Water per minute
Pass through time

19. Namics.
Limited Water Channel
Interpreting the Outcome
Oct. 9, 2015 19 Coding Serbia. Systematic Load Testing of Web Applications.
Throughput: Liters per minute
Pass through time
narrowing

20. Namics.
Limited Linear Model
Interpreting the Outcome
Oct. 9, 2015 20 Coding Serbia. Systematic Load Testing of Web Applications.
Water per minute
Throughput:
Liters per minute
Water per minute
Pass through time
maximim capacity reached,
after that jam (saturation)

21. Namics.
Non Linear System
Interpreting the Outcome
Oct. 9, 2015 21 Coding Serbia. Systematic Load Testing of Web Applications.
Water per minute
Throughput:
Liters per minute
Pass through time
maximum capacity reached,
after that jam (saturation)
system collaps
system blocked
Water per minute
∞
pressure brake(?)

22. Namics.
Example of test results (1 of 4)
Interpreting the Outcome
Oct. 9, 2015 22 Coding Serbia. Systematic Load Testing of Web Applications.

23. Namics.
Example of test results (2 of 4)
Interpreting the Outcome
Oct. 9, 2015 23 Coding Serbia. Systematic Load Testing of Web Applications.
85Mb / second

24. Namics.
Example of test results (3 of 4)
Interpreting the Outcome
Oct. 9, 2015 24 Coding Serbia. Systematic Load Testing of Web Applications.
concurrency problem

25. Namics.
Example of test results (4 of 4)
Interpreting the Outcome
Oct. 9, 2015 25 Coding Serbia. Systematic Load Testing of Web Applications.
swell

26. Namics.
Tipps for executing the test
 To find the correct scenario is really hard
 Look at historic logfiles / stats (plus safety margin)
 Ask domain experts
 Take the user stories (“follow the money”)
 What to go for
 Bandwith  Fetch one large item (again and again)
 # Transactions  Short test cycles with 5, 10, 20, 40... users to
find out where the systems levels (or decreases or crashes)
 Long runner  Execute a slow test during hours
 What’s nice to know too
 Start system under load
 Stop system under load
 Degradation (i.e. cluster node shutdown)
Interpreting to Outcome
Oct. 9, 2015 26 Coding Serbia. Systematic Load Testing of Web Applications.

27. Namics.
Watch out for
 Realistic scenario
 request mix
 think time
 read vs write (POST requests)
 caching
 Get 100% CPU of the tested system first
 Look at the database (slow queries)
 Look at the HTTP response content/headers of the test
 Did the system that executes the test deliver the load?
 Error logs
Interpreting to Outcome
Oct. 9, 2015 27 Coding Serbia. Systematic Load Testing of Web Applications.

28. Namics.
The usual suspects
 Network (Pipe, Router, Firewall) < 1%
 Load Balancer 5 %
 Reverse Proxy (Product) 5 %
 Reverse Proxy (Configuration) 10 %
 OS configuration (TCP/IP Stack) 5 %
 Application Framework 10 %
 Configuration of Application 20 %
 Programming/Code of Application 40 %
 Database 5%
Interpreting to Outcome
Oct. 9, 2015 28 Coding Serbia. Systematic Load Testing of Web Applications.

29. Oct. 9, 2015 29 Coding Serbia. Systematic Load Testing of Web Applications.
Wrap Up

30. Namics.
Summary
 Load testing MUST be part of every project plan
 Product owner must define what’s needed
 Do not test/optimize performance early in the dev cycle
 Test as close to the application as you can get
 and only when you find issues  change view
 Percentile statistics are OK (90% of requests fit)
 Scenarios are though to get
 Validate test results....
Wrap up
Oct. 9, 2015 30 Coding Serbia. Systematic Load Testing of Web Applications.

31. Namics.
Homework
 Read the HTTP Specification
 https://tools.ietf.org/html/rfc7230
Wrap up
Oct. 9, 2015 31 Coding Serbia. Systematic Load Testing of Web Applications.

32. Oct. 9, 2015 32 Coding Serbia. Systematic Load Testing of Web Applications.
About Namics

33. Namics.
Who is Namics?
About Namics
Oct. 9, 2015 33 Coding Serbia. Systematic Load Testing of Web Applications.
Focus
 High-end e-business services for
large, global customers
 Leading also in complex large-
scale projects
Stability
 Long-term customer relationships
and stable organisation
 Owned by a group of 29 employees
(Namics Partners)
 Company operating since 1995

34. Namics.
Namics figures in a Nutshell
About Namics
Oct. 9, 2015 34 Coding Serbia. Systematic Load Testing of Web Applications.
 Quality as growth engine
 Best of Swiss Web
agency in 2013 & 2014
(Best of Swiss Web
Association)
 Proven track record
 Stable relationship with
our key clients
 More than 70% of our
clients stay with us for
3+ years
2009 2010
55
50
45
40
35
30
25
20
10
Revenue (in Mio. CHF)
and number of employees
450
400
350
300
250
200
150
100
50
2011 2012 2013
60 500
2014

35. Namics.
Namics in Serbia
Highlights
 newest location in Europe
 cross country teams,
interdisciplinary
Belgrade in a nutshell
 start-up culture
 new 480m2 office
 collaboration
 Awesome since 1995.
Now in Serbia. Namics.
About Namics
Oct. 9, 2015 35 Coding Serbia. Systematic Load Testing of Web Applications.

36. Namics.
We are looking for new talent in Belgrade!
 http://namics.com/jobs
About Namics
Oct. 9, 2015 36 Coding Serbia. Systematic Load Testing of Web Applications.

37. Download-Link:
http://nam.to/webloadtest
jurg.stuker@namics.com
@jstuker