This document discusses benchmarking various hardware platforms and operating systems for optimal OTM performance. It provides an agenda for a presentation that will teach how to benchmark OTM platforms using tools like VolanoMark, DaCapo, Soap Stone and Hammerora. The presentation will show hands-on exercises for running the benchmarks and interpreting the results. Higher scores are better for VolanoMark and Soap Stone, while lower scores indicate better performance for DaCapo and Hammerora. Online resources for monitoring performance and learning more about the benchmarks are also provided.

1. Benchmarking OTM and Java:
Is Your Platform Limiting
Performance?
OTM SIG User Conference ‘09

2. Agenda
Setting Expectations
OTM – Hardware and Platforms
Tools – System and Monitoring
Benchmarks
Hands-on Benchmarking (Real-time)
Interpreting and Applying to OTM
Q&A

3. This Presentation WILL:
Teach you to benchmark various OTM
platforms
Show you how to interpret the benchmark
results
Show real-world platform results

4. This Presentation will NOT:
Teach you to load-test an OTM installation
Show you how to predict OTM SCA (scalability
/ clustering) performance
Utterly fail (I hope!)

5. Hardware / Platforms
CPU and Hardware Platform Matter! A LOT!
CPU Speed – Not a Good Indicator of
Performance
Other factors (cores, memory bandwidth, on-chip
cache) necessitate benchmarking
OTM Requires both high multi-threading AND
high single-thread performance
Lots of cores and high per-core performance
Performance of Current Platforms
Linux / x86-64
Windows / x86-64 (note: memory limitations)
Solaris
HP-UX / PA-RISC
Note: HP-UX / Itanium currently unknown
AIX / POWER

6. Operating System / Stats – Helpful but Limited
Review system performance under production
load for the previous 2 weeks
Utilize System Tools to Monitor
sar / kSar
top / prstat / topas / etc
Utilize Tools to Trend
Nagios / Munin / etc

7. Benchmarking – Which Benchmarks?
VolanoMark
DaCapo
Soap Stone
Hammerora
Why?
Java based
Replicate OTM behavior
Easily repeatable

8. Why Not Load-Test OTM?
Complicated to setup
Time-consuming to run
Requires OTM installed AND
configured
Not Necessary to determine
platform capacity

9. Benchmarking - VolanoMark
Running VolanoMark (Hands-On Exercise)
Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/6-
benchmarking-part-1-java-jvm-performance-
volanomark.html

10. Benchmarking - VolanoMark
VolanoMark
Java-based benchmark that simulates high
transactional and multi-threaded load
Reflects the performance of the following OTM
activities
Web UI, Agents, Integration, General Workflow, General OTM
Activities (not including optimization and planning based)
Higher numbers are better

11. Benchmarking - DaCapo
Running DaCapo (Hands-On Exercise)
Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/12
-benchmarking-part-3-java-jvm-performance-
dacapo.html

12. Benchmarking - DaCapo
DaCapo
Java-based benchmark that simulates highly
computational, algorithmic, single-threaded
processing
Reflects the performance of the following OTM
activities
Optimization and Planning / Bulk Planning
Lower numbers are better

13. Benchmarking – Soap Stone
Running Soap Stone (Hands-On Exercise)
Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/13
-benchmarking-part-4-real-world-network-
performance-soap-stone.html

14. Benchmarking – Soap Stone
Soap Stone
Java-based benchmark that tests data throughput
between servers and replicates application protocols,
such as HTTP, RMI and RAW.
Reflects the throughput and protocols utilized
between the various OTM Tiers
Browser / Web: HTTP
Web / App: RMI
App / DB: RAW
Higher numbers are better

15. Benchmarking – Hammerora
Running Hammerora (Hands-On Exercise)
Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/11
-benchmarking-part-2-oracle-db-performance-
hammerora.html

16. Benchmarking – Hammerora
Just Kidding!!!
There’s no way we’d have enough time

17. Benchmarking – Hammerora
Hammerora
Benchmark based on the TPC-C and TPC-H
benchmarks.
Reflects the performance and scalability of the DB
Tier
Lower numbers are better

18. Interpreting the Results
These benchmarks have linear trends (twice as
fast is twice as fast)
VolanoMark
Higher is better
Applies to Web and App tiers
DaCapo
Lower is better
Applies to App tier
Soap Stone
Higher is better
40Mbit or faster Web App
400Mbit or faster App DB
Hammerora
Higher is better
Applies to all tiers

19. Online Resources
Performance
kSar
http://ksar.atomique.net/
Nagios
http://www.nagios.org/
Munin
http://munin.projects.linpro.no/
Benchmarking
Full Replication Details
http://www.otmfaq.com/forums/blogs/chrisplough/
VolanoMark
http://www.volano.com/benchmarks.html
DaCapo
http://dacapobench.org/
Soap Stone
http://soap-stone.sourceforge.net/
Hammerora
http://hammerora.sourceforge.net/

20. Q & A and Discussion
Questions?

21. Thank You!
chris.plough@mavenwire.com
866.343.4870 x701
www.MavenWire.com