B2conference performance 2004

15-Nov-07

Introduction to Software
Performance Engineering
Blackboard Performance Center of Excellence

Steve Feldman, Tim Moore and Ryan O’Neil
perfeng@blackboard.com
©2004 BLACKBOARD, INC. ALL RIGHTS RESERVED.

Any statements in this presentation about future
expectations, plans and prospects for the Company, including
statements about the Company, the Building Blocks Program
and other statements containing the words “believes,”
“anticipates,” “plans,” “expects,” “will,” and similar
expressions, constitute forward-looking statements within
the meaning of The Private Securities Litigation Reform Act
of 1995. Actual results may differ materially from those
indicated by such forward-looking statements as a result of
various important factors, including: product development,
and other factors discussed in our Registration Statement
filed on Form S-1 with the SEC. In addition, the forward-
looking statements included in this press release represent
the Company’s views as of July 26, 2004. The Company
anticipates that subsequent events and developments will
cause the Company’s views to change. However, while the
Company may elect to update these forward-looking
statements at some point in the future, the Company
specifically disclaims any obligation to do so. These forward-
looking statements should not be relied upon as representing
the Company’s views as of any date subsequent to July 26,
2004.

Read the Fine Print
Any statements in this presentation about future expectations, plans and prospects for the Company, including statements about the Company, the Building Blocks Program and other
Any statements in this presentation about future expectations, plans and prospects for the Company, including statements about the Company, the Building Blocks Program and other
statements containing the words “believes,” “anticipates,” “plans,” “expects,” “will,” and similar expressions, constitute forward-looking statements within the meaning of The Private Securities
statements containing the words “believes,” “anticipates,” “plans,” “expects,” “will,” and similar expressions, constitute forward-looking statements within the meaning of The Private Securities
Litigation Reform Act of 1995. Actual results may differ materially from those indicated by such forward-looking statements as a result of various important factors, including: product
Litigation Reform Act of 1995. Actual results may differ materially from those indicated by such forward-looking statements as a result of various important factors, including: product
development, and other factors discussed in our Registration Statement filed on Form S-1 with the SEC. In addition, the forward-looking statements included in this press release represent the
development, and other factors discussed in our Registration Statement filed on Form S-1 with the SEC. In addition, the forward-looking statements included in this press release represent the
Company’s views as of July 26, 2004. The Company anticipates that subsequent events and developments will cause the Company’s views to change. However, while the Company may elect
Company’s views as of July 26, 2004. The Company anticipates that subsequent events and developments will cause the Company’s views to change. However, while the Company may elect
to update these forward-looking statements at some point in the future, the Company specifically disclaims any obligation to do so. These forward-looking statements should not be relied upon
to update these forward-looking statements at some point in the future, the Company specifically disclaims any obligation to do so. These forward-looking statements should not be relied upon
as representing the Company’s views as of any date subsequent to July 26, 2004.
as representing the Company’s views as of any date subsequent to July 26, 2004.


Purpose of Today’s Session
Introduction to SPE (5 Minutes)

SPE Methodology (15 Minutes)

Applying SPE to Building Blocks (5 Minutes)

Performance Analysis Tools and Techniques (5
Minutes)

Demonstration of Building Block (10 minutes)

Answer some questions… (10 minutes)


Introduction to SPE
Q: What is software performance?

A: Any characteristic of a software product that you
could quantifiably measure?

Q: Why is software performance important?

A: The world we live in today is becoming more digitally
sophisticated. We expect our digital transactions to be
faster then our paper transactions.

Q: How do we manage performance in the SDLC?

A: Plan, Predict, Prove, Improve performance
throughout the SDLC?

Introduction to SPE
Performance is at the top of everyone’s mind.

If Performance is poor, software adoption will decline
and potentially cause usage attrition or adoption of
alternative products/processes.

Performance Failures cost institutions a large amount
of unplanned money.

Performance problems can create a huge trust
barrier between user and provider of technology.

So then what is SPE?


Introduction to SPE
SPE is a methodology that…

Provides a systematic, quantitative approach to
constructing software systems that meet performance
objectives.

Provides a software-oriented approach to
architecture, design and implementation choices.

Prescribes principles and performance patterns for
creating responsive software.
Performance antipatterns for recognizing and correcting common problems,
the data required for evaluation, procedures for obtaining performance
specifications, and guidelines for the types of evaluation to be conducted at
each development stage.

SPE Methodology
SPE is a 7 Step Cyclical Process…

Assess Performance Risk

Identify Critical Use Cases

Select Key Performance Scenarios

Establish Performance Objectives

Construct Performance Models

Determine Software Resource Requirements

Add System Resource Requirements


SPE Methodology
Assess Performance Risk

Distinguish between new development and refactor.

Work with requirements specialists to understand
the problem/domain issues for the work.

Understand potential inter- versus intra-system
integration.

Apply common sense and basic business logic from
past experiences and similar development projects.


SPE Methodology
Identify Critical Use Cases

Most important operations/actions in the feature or
system.

Responsiveness driven.

Risk driven.
You look for use cases where there is a risk that, if performance goals are
not met, the system will fail or be less than successful.


SPE Methodology
Select Key Performance Scenarios

Unlikely that all critical use cases will be important to
performance.

The key performance scenarios are those that are
executed frequently, or those that are critical to the
perceived performance of the system.

Each performance scenario corresponds to a
workload.

Performance scenarios represented through sequence
diagrams augmented with some useful extensions.

SPE Methodology
Establish Performance Objectives

Identify and define performance objectives.
Specify the quantitative criteria for evaluating the performance
characteristics of the system under development.
Response time, throughput, or constraints on resource usage

Identify and define workload objectives.
Specify the level of usage for the scenario.
They are specified as an arrival rate (e.g., number of Web site hits per
hour), number of concurrent users or number of parallel transactions.


SPE Methodology
Construct Performance Models

Use of Execution Graphs to represent software
processing steps in the performance model.

The sequence diagram representations of the key
performance scenarios are translated into execution
graphs.

Reverse Engineer Reverse Engineer Reverse
Engineer


SPE Methodology
Determine Software Resource Requirements

Understand at an atomic level the types of software resource
requirements.
Method Calls
SQL Executions
Data Loading
Caching

Understand the effects of this software to inter- and intra-
system components.

Understand at best- and worst-case perspective.
90% of the time
Peak time.


SPE Methodology
Add System Resource Requirements

Apply Workload Understanding

Based on findings in the software resource
requirements phase you should be able to understand at
an atomic level...
CPU Cycles consumed by the method call and/or sql operation.
Network overhead (packet transfer)
Memory requirements
Processing Thread/DB Connection Requirements


Applying SPE to Blocks
We chose to create our own building block for this demo…

Santa’s Little Helper

B2 that provides performance/capacity metrics of sub-system
usage within Blackboard.

Parses an Apache access.log file

Generate a graphical display of the following metrics
– Sub-System Breakdown (Primary Graphical Requirement)
– Hits Per Day in terms of Visits (Secondary Graphical Requirement)
– Average Length Per Visit (Nice to Have Graphical Requirement)

Generate a textual summary of the following metrics
– Sub-System Breakdown (Primary Graphical Requirement)
– Top 50 Click Paths (Primary Graphical Requirement)
– Hits Per Day in terms of Visits (Secondary Graphical Requirement)
– Average Length Per Visit (Nice to Have Graphical Requirement)


SPE Phase Application of B2 to SPE
Assess Performance Risk Relatively low use case, maybe executed by 1 person on a system a
handful of times per day or week. No major database activity,
performed in isolation on the application server.
Identify Critical Use Cases 1. Textual Report (Default Report) of Sub-System Breakdown.

2. Textual Report of Hits Per Day.

Select Key Performance Scenarios Limited only to administrator role access to this sub-system. Expected
to be executed daily.

Establish Performance Objectives 1. Ability to Parse up to 10mb in size. (Manage via a log rotation
parameter)

2. 1mb file processed in 90 seconds.

3. Conversion of text into graphic 10 seconds or less.
Construct Performance Models Sequence Diagram Attached

Determine Software Resource Requirements Review Profiling Metrics

Add System Resource Requirements Report from Grinder and Study System Resource Usage



Example
Sequence/Workflow
Diagram


SPE Tools and Techniques
Load Induced Tools for Simulation

Java Profiling Interface to plug into the JVMPI API.

SQL Profiler


Load Induced Tools for Simulation

Commercially Available Tools
Mercury LoadRunner (Primary Tool of BB Perfeng)
Seque Silk Performer

Freeware Tools
The Grinder (Primary Tool of BB Dev)
Volano
Countless others on Sourceforge


Java Profilers

JProbe (Used by BB Perfeng)
Performasure
OptimizeIt
Wily Enterprise Application Analyzer
HyPerformix

Freeware Tools
HPJmeter
JProbe Light (Used by BB Dev)
Eclipse Profiler (Used by BB Dev)


SQL Profilers

Hotsos (Used by BB Perfeng)
IronEye SQL

Freeware Tools
Oracle SQL Analyze (Used by BB Perfeng and Dev)
SQL Profiler for MS SQL Server (Used by BB Perfeng and Dev)


Demonstration…

Here is our new
Building Block


Demonstration…

Click Here to
Study Usage
Statistics


Demonstration…

Sub-Systems are Categorized and
Distributed by Hits

Total System Hits for the Day


Demonstration…
Contents of the Grinder.Properties File
grinder.processes=1
grinder.threads=10
grinder.runs=5
grinder.duration=0
grinder.script=tests/usage_analysis.py
grinder.logDirectory=logs
grinder.useConsole=false
grinder.initialSleepTime=5000
grinder.sleepTimeFactor=1
grinder.sleepTimeVariation=1.0
grinder.logProcessStreams=true
grinder.recordTime=true

Demonstration…

Example Grinder .py file
for Load Testing
Santa’s Little Helper


Demonstration…
Test Iterations Errors Mean Mean Response Response Transaction
Transaction Response Bytes Per Errors
Time (ms) Length Second

Test 1 50 0 494.32 36565.00 NA 0 "My
Institution"

Test 2 50 0 2547.30 43182.00 NA 0 "Admin
Panel"

Test 3 50 0 106592.2 16566.94 NA 0 "Usage
Stats"


Demonstration…


Question and Answer…


B2conference performance 2004

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (8)

Similaire à B2conference performance 2004

Similaire à B2conference performance 2004 (20)

Plus de Steve Feldman

Plus de Steve Feldman (20)

B2conference performance 2004