The presentation reviews various tools and libraries which help us add continuous performance testing to the development process. Using Hudson CI to manage our build system, JUnit based performance tests, and JMeter and Grinder load tests. See demo project @ http://www.doboism.com/blog/2010/11/25/continuous-performance-testing/

1. Continuous Performance Testing

2. What you are about to witness
● Continious integration bla bla
● What is hudson?
● Performance Unit tests
● Load testing with Jmeter
● Load testing with Grinder

3. Continuous Integration?
● Scheduled, automated build
● Know of problems early on
● Have a reasonably stable product at any
time
● Track various metric (tests, coverage,
static code analysis, coding conventions)

4. Hudson CI
● A breeze to install, simple to configure
● Architecture:
– Triggers (scheduling)
– Hook to source control
– Build tools (executable, ant, maven)
– Report generators (test results)
– Notifiers (email, RSS, etc.)
● Other goodies: fingerprinting,
distribution, ...

5. Continuous performance
testing?
● Track regressions in product performance
● Meet specified performance requirements
● Put guards for bugs resulting in errors
threads
● Establish a benchmark for the application
● See (and show) progress

6. Performance Unit tests
● Junit – not very much (timeout?)
● Roll your own
● Test decorators to help
Threading, logging, more detailed
requirements:
– JUnitPerf (Junit 3.5 – decorate suits()
– JUnitBench
– ContiPerf

7. When to test units for
performance?
● Sanity tests
● Performance critical units
● Before making changes that might cause
regressions
● Note: Junit can be harnessed for sub-
system tests, not just units

8. System tests
● Try to emulate ”Real life” conditions: Test
plan, deployment
● Focus on finding out if something is wrong,
not what
● Check how to application as a whole is
working
● Build to scale when needed
● Update the test as your application
changes (example – cached responses)

9. JMeter
● Tests are written in UI
● Various samplers: Http, DB, java
● Complicated tests are achived with
different test plan elements, scripts, plugins
● Distributed agents
● Hudson plugin

10. Grinder
● Generic and minimal ( == simple) approach
● Jython (python) scripts based
● Instrument tested code
● Instrument your call
● Http - out of the box support
● Distributed test support via console

11. What we did not cover
● Complex testing architecutre (distributed
testing, multiple deployment targets)
● Measuring server metrics (i.e. CPU, IO,
Memory)
● Micro performance testing – instrumented
code, profiling

12. Amitay Dobo
Http://www.doboism.com