Presentation review best ways to accomplish database load testing and analysis of database performance. Presentation targeting major RDBMS:systems - Oracle and SQL Server as well as tolls necessary for database load testing, Oracle performance tuning, SQL Server performance tuning, Windows and Linux performance optimization

1. Enteros, Inc.
Starwest 2012
Database Load Testing
and Performance Analysis:
New Approaches for Fast Results
Presentation outline
By Enteros CTO
Ron Warshawsky
ron@enteros.com
408-207-8408
October 1, 2013

2. 2
Enteros STARWEST 2012
Load Testing – Why database load testing approach?
1) Faster way to reveal root causes of bottlenecks
2) Faster way to end results
3) Easier to develop
4) Easier to maintain

3. 3
Enteros STARWEST 2012
Fact: Database is the resource that is hardest to scale.
Thus it’s optimization is of critical importance.
Benefits to expect from database optimization
1) SLA compliance of business flows
2) Increase in system availability
3) Increase in system scalability
4) Better resistance to resource spikes

4. 4
Enteros STARWEST 2012
Databases overview

5. 5
Enteros STARWEST 2012
Database architecture (Oracle):

6. 6
Enteros STARWEST 2012
Database architecture (SQL Server):

7. 7
Enteros STARWEST 2012
Database architecture (Hadoop):

8. 8
Enteros STARWEST 2012
Database contentions:

9. 9
Enteros STARWEST 2012
Database performance metrics:

10. 10
Enteros STARWEST 2012
Database load testing process

11. 11
Enteros STARWEST 2012
Database load testing process
1) Capture production database traffic or individual business
flows (Network and SQL trace)
2) Convert captured traffic into database load test scenario
3) Prepare load test database
4) Execute load test scenario to generate load
5) Analyze results of load testing, get optimization
suggestions
6) Implement optimizations
7) Retest

12. 12
Enteros STARWEST 2012
Capture database network traffic
1) Windows - Wireshark
2) UNIX – tcpdump
3) Network switch mirror port / SPAN

13. 13
Enteros STARWEST 2012
Capture database SQL traffic
1) Oracle RAT – real application testing
Oracle EE only, 11g replay, 10g/11g capture (9i with special
patch bundle)
2) Oracle 10046 trace
http://www.oracle-base.com/articles/10g/sql-trace-10046-trcsess-and-tkprof-
10g.php
3) SQL Server Profiler – profiler traces
1) http://msdn.microsoft.com/en-us/library/ms187929(v=sql.105).aspx

14. 14
Enteros STARWEST 2012
Convert traffic into load test
1) Convert database TCP traffic
Mostly commercial implementations:
Enteros Load2Test for Databases
2) Convert Oracle SQL traffic
Oracle RAT
Custom traffic parser for Oracle 10046 trace (Perl, Python)
3) Convert SQL Server SQL traffic
Profiler
Custom traffic parser (Perl, Python)

15. 15
Enteros STARWEST 2012
Create database load test scenario – similar to UI
load testing
1) Identify load testing databases
2) Identify proper mix of business flows or production
replay scenario
3) Identify sessions concurrency
4) Set rendezvous points
5) Set load test duration

16. 16
Enteros STARWEST 2012
Execute load test
1) Monitor and capture load test activity
2) Monitor and capture database activity
3) Monitor and capture OS activity

17. 17
Enteros STARWEST 2012
Database load testing tools
1) Oracle RAT (production traffic)
2) SQL Server trace replay
3) Enteros Load2Test for Databases (production traffic and
business flows)
4) Hammerora (open source, 10046 trace) and SwingBench –
TPC-C, TPC-H
5) Quest benchmark factory (TPC-C, SQL flows, 10046 trace)
6) Grinder (open source) – custom, business flows

18. 18
Enteros STARWEST 2012
Analyze load test results
1) Correlate database analysis timeline with spikes in business
flow response time
2) Start with waits and look for spikes correlated with spikes in
flow response time
3) Drill down to SQLs correlated with waits (different types),
eliminate waits is possible
4) Drill down to SQL consuming Top amount of CPU, tune SQL
5) Drill down to I/O contentions
6) Drill down to SQL consuming Top amount of I/O, tune SQL
7) Drill down to segments and datafiles consuming Top amount
of I/O, tune I/O subsystem

19. 19
Enteros STARWEST 2012
Analyze load test results

20. 20
Enteros STARWEST 2012
Analyze load test results (2)

21. 21
Enteros STARWEST 2012
Analyze load test results (3)

22. 22
Enteros STARWEST 2012
Analyze load test results (4)

23. 23
Enteros STARWEST 2012
Analyze database load test results
1) Visual Tools
Oracle OEM, SPA
Enteros Performance Explorer-i
Enteros High Load Capture
SQL Server management studio, MDB reports
2) Text/HTML Report Tools
Oracle AWR/Statspack reports
Oracle ASH reports
Oracle AWR SQL reports
SQL Server performance views / MDW

24. 24
Enteros STARWEST 2012
Analyze OS load test results
1) Windows – perfmon, Oracle OS Watcher for Windows
UI: PAL – perfmon analysis tool - http://pal.codeplex.com/
UI: Enteros High Load Capture
2) UNIX/Linux – Oracle OS Watcher
Oracle OS Watcher
Oracle support note: OS Watcher Black Box User Guide [ID
301137.1]
UI - Enteros High Load Capture
OS performance tools: top, vmstat, iostat, mpstat, sar
UI – Nagios, cacti

25.  Questions?
 Email: ron@enteros.com
 Site: http//www.enteros.com
 Enteros: oil in database engine
Enteros Q&A