SlideShare une entreprise Scribd logo
1  sur  109
Télécharger pour lire hors ligne
TR
Half-day Tutorials
5/6/2014 1:00:00 PM
Test Estimation in Practice
Presented by:
Rob Sabourin
AmiBug.com
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Rob Sabourin
AmiBug.com
Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of software
development professionals. A well-respected member of the software engineering community, Rob has managed,
trained, mentored, and coached hundreds of top professionals in the field. He frequently speaks at conferences and
writes on software engineering, SQA, testing, management, and internationalization. Rob wrote I am a Bug!, the
popular software testing children's book; works as an adjunct professor of software engineering at McGill University;
and serves as the principle consultant (and president/janitor) of AmiBug.Com, Inc. Contact Rob at Contact Rob
at rsabourin@amibug.com.
Introduction
TEST Estimation IN
PRACTICE
Course
timing
Course
timing
Electron
ic
devices
Electron
ic
devices
Smokin
g
Smokin
g
MealsMeals
Faciliti
es
Faciliti
es
Break
s
Break
s
Administrivia
6© 2012 SQE Training V3.0
Introductions
Name
Company
Name
Business
Environment
Hardware
Software
Experience
Testing
Management
Personal Course Objective(s)
Name
Company
Name
Business
Environment
Hardware
Software
Experience
Testing
Management
Personal Course Objective(s)
7© 2012 SQE Training V3.0
Course Agenda
1. Estimation
2. Estimation Techniques
8© 2012 SQE Training V3.0
1
estimation
Estimation
Estimate:
– A tentative evaluation or rough calculation
– A preliminary calculation of the cost of a
project
– A judgment based upon one’s impressions;
opinion
—The American Heritage Dictionary
It is very difficult to make a vigorous, plausible, and job-
risking estimate that is derived by no quantitative method,
supported by little data and certified chiefly by hunches of the
managers.
— Fred Brooks
10© 2012 SQE Training V3.0
The best estimatesThe best estimates
Represent the collective wisdom of practitioners
(testers, developers, users, etc.) and have their buy-in
Provide specific, detailed catalogs of the costs,
resources, tasks, and people involved
Present, for each activity estimated, the most likely
cost, effort, and duration
Estimation: the creation of an approximate
target for costs and completion dates
Estimation: the creation of an approximate
target for costs and completion dates
Test Estimation
11© 2012 SQE Training V3.0
Factors that can influence cost, effort, and duration include:Factors that can influence cost, effort, and duration include:
Required level of quality of the system
Size of the system to be tested
Historical data
Process factors (process maturity, etc.)
Material factors (tools, data, etc.)
People factors (skills, experience, managers, etc.)
Test Estimation (cont.)
12© 2012 SQE Training V3.0
Test Estimation (cont.)
• Delivery of estimates should include
justification
• Negotiation and re-work of estimates is
normal
• Final estimates represent a balance of
organizational and project goals in the areas of
quality, schedule, budget, and features
13© 2012 SQE Training V3.0
How Good an Estimator Are You?
Low Estimate High Estimate
___________ to ____________
.
.
.
© Steve McConnell, Software Estimation, Microsoft Press, 2006
Surface temperature of the sun
Latitude of ShanghaI
Area of the Asian continent
Birth year of Alexander the Great
Total value of US currency in 2004
Total volume of the Great Lakes
Worldwide box office receipts of Titanic
Total length of Pacific Ocean coastline
Heaviest blue whale ever recorded
# of books published in US since 1776
14© 2012 SQE Training V3.0
How Good an Estimator Are You?
• Surface temperature of the sun 10,0000 F / 6,000 C
• Latitude of ShanghaI 31° 10′ N
• Area of the Asian continent 17,212,000 sq mi
• Birth year of Alexander the Great 356 BC
• Total value of US currency in 2004 $719.9 Billion
• Total volume of the Great Lakes 5,500 cu mi
• Worldwide box office receipts of Titanic $1.835 billion
• Total length of Pacific Ocean coastline 84,300 miles
• Heaviest blue whale ever recorded 380,000 pounds
• Number of books published in US since 1776 22 million
15© 2012 SQE Training V3.0
How Good Is Our Industry (at Estimating)?
• Tata: 62% of projects fail to meet schedule
49% have budget overruns
• Moiokken and Jorgensen: 30-40%
overruns
16© 2012 SQE Training V3.0
Class Discussion
Why is estimating not done well?
Your top five reasons:
1) Too many variables____________________
2) ____________________________________
5) ____________________________________
6) ____________________________________
5) ____________________________________
17© 2012 SQE Training V3.0
Why Estimates Are Inaccurate ― Part I
• Lack of estimating experience
• Lack of historical data on which to base estimates
• Lack of systematic estimation process, sound techniques, or
models suited to the project
• Failure to include essential activities and products within the
scope of the estimates
• Unrealistic expectations or assumptions
• Failure to recognize and address the uncertainty inherent in
project estimates
Practical Software Measurement
Addison-Wesley, 2001
18© 2012 SQE Training V3.0
Why Estimates Are Inaccurate ― Part II
• Lack of education and training
• Confusing the target with the estimate
• Hope-based planning
• Inability to communicate and support estimates
• Incomplete, changing, and creeping requirements
• Quality surprises (test and re-fix)
—adapted from Linda M. Laird
The Limitations of Estimation
19© 2012 SQE Training V3.0
Bohem’s Cone of Uncertainty
20© 2012 SQE Training V3.0
NHC Track Forecast Cone
21© 2012 SQE Training V3.0
“Testing” Track Forecast Cone
Best case
Expected case
Worst case
T i m e
Tasks
(or why it is important to constantly re-estimate)
22© 2012 SQE Training V3.0
What would have to happen to deliver this in four
weeks?
What would have to happen to deliver this in four
weeks?
What should the estimate have been?What should the estimate have been?
The Fantasy Factor
Weeks
Today
0 1 2 3 4 5 6 7 8 9
1st 3rd2nd
23© 2012 SQE Training V3.0
Tim
e
Tim
e
SizeSize
Quali
ty
Quali
ty
Res
our
ces
Res
our
ces
Estimation
1, 2, 3, or 4 Variables + Many Modifiers:
If it’s not variable, then it’s fixed.
24© 2012 SQE Training V3.0
Time vs. Resources
=
25© 2012 SQE Training V3.0
2
Estimation Techniques
Test Estimation Techniques ― Examples
• Intuition and guess
• Work-breakdown-structures
• Three-point estimates
• Company standards and norms
• % of project effort or staffing
• Industry averages and predictive models (e.g., FP, TPA )
• Team estimation sessions
– Wideband Delphi
– Story point sizing
– Poker estimation
– T-shirt sizing
28© 2012 SQE Training V3.0
What do intuition and guess
imply?
What do intuition and guess
imply?
No formal estimation process
Based on the experience of the team
member(s) usually without the benefit of
recorded metrics
Intuition and Guess
29© 2012 SQE Training V3.0
When would we use this technique?When would we use this technique?
Experienced staff members
Little or no recorded metrics available
Great accuracy is not required
Same type of project we’ve done in the
past
Or at the other extreme, we don’t have a
clue (often called a WAG or SWAG)
Intuition and Guess (cont.)
30© 2012 SQE Training V3.0
Project LevelProject Level
Phase
Level
Phase
Level
Task LevelTask Level
Phase
Level
Phase
Level
Task LevelTask Level
------
Task LevelTask Level
------
------
------
Task LevelTask Level
------
Phase
Level
Phase
Level
Task LevelTask Level
Phase
Level
Phase
Level
Task LevelTask Level
Work Breakdown Structure
31© 2012 SQE Training V3.0
When would we use this technique?When would we use this technique?
When the test team lacks credibility with their
estimates
When we are doing an estimate for a project that is
dissimilar to projects we have done in the past
When the project is too large to get our “arms
around it” (e.g., it is too big to easily see the big
picture)
Work Breakdown Structure (cont.)
32© 2012 SQE Training V3.0
When would we not use this technique?When would we not use this technique?
When the project has lots of turbulence!
Work Breakdown Structure (cont.)
33© 2012 SQE Training V3.0
Approach:Approach:
Develop goal
Break goal into deliverables
Break deliverables into activities
Decompose work into small manageable
components that are of sufficient detail to
allow you to make estimates of time and/or
resources
Work Breakdown Structure ― Top Down
34© 2012 SQE Training V3.0
Work Breakdown Structure (cont.)
Testing for Completeness
• Status of a task is measurable
• Start and end events are clearly defined
• Further breakdown makes no sense
• Each activity has a deliverable
• Time or resources for an activity can be estimated
• Duration of activity is within acceptable limits
• Each activity is independent
• Each activity forms a unique package of work that could
be outsourced
35© 2012 SQE Training V3.0
Work Breakdown Structure (cont.)
• Level of detail guidelines:
– No single activity or group of activities should
exceed 80 hours of effort*
– No activity or group of activities should exceed
the reporting period (e.g., if the reporting period
is weekly, no activity should exceed one week)
– “If it makes sense” rule: The duration of each
activity must pass the common sense rule
36© 2012 SQE Training V3.0
Three–Point Estimates
• Three-point estimates is an analytical technique using
three cost or duration estimates:
– Optimistic
– Most likely
– Pessimistic
• This technique is applied to improve the accuracy of
the estimates of cost or duration
• Three-point estimation techniques can be employed
with other estimating techniques such as Delphi
37© 2012 SQE Training V3.0
% of Project Effort or Staffing
• Based on the premise that there is a
predictable correlation between development
effort/time and test effort/time
• The preferred/“dictated” method of
estimation in some organizations
• Sometimes associated with developer/tester
ratios
• One method recognized by the ISTQB
38© 2012 SQE Training V3.0
When Would We Use This Technique?When Would We Use This Technique?
An organization is producing more or less the same type of
product on an ongoing basis
The organization (development and testing) uses consistent
practices and has a stable staff
A crude estimate is needed
It is mandated
It works
% of Project Effort or Staffing
39© 2012 SQE Training V3.0
Issues:Issues:
We are basing our estimates on development
estimates which are probably suspect
Some products that are relatively easy to develop
may be difficult to test and vice versa
This technique may not take into account test
automation, etc.
This method may not take the stated quality goals
into account
Test Estimates Based on Development Estimates
40© 2012 SQE Training V3.0
Class Discussion
What is the ideal developer/tester ratio?
= ?
41© 2012 SQE Training V3.0
McConnell Developer to Tester Ratio
Environment Relative Importance Ratio
(Developer:Tester Ratio)
Business Systems 3:1 to 20:1
Shrink Wrap 1:1 to 5:1
Scientific Projects 5:1 to 20:1
Systems Projects 1:1 to 5:1
Safety Critical Software 5:1 to 1:2
Microsoft Windows 2000 1:2
NASA Space Shuttle Flight Control Software 1:10
42© 2012 SQE Training V3.0
Test Point Analysis
• A method with the possibility to perform a
technology-independent measurement of the test
size of an information system on the basis of a
function point analysis, and to use this
measurement as a basis for a productivity
measurement, an estimate of the required
resources, and project management
Software Testing: A Guide to the TMap Approach
Martin Pol, Ruud Teunissen, Erik van Veenendaal
43© 2012 SQE Training V3.0
Test Point Analysis (cont.)
For more details on TPA,
look in this book.
44© 2012 SQE Training V3.0
Principles of Test Point Analysis
• TPA only considers measurable quality characteristics that fall
within the range of system or acceptance testing
• TPA is (in principle) analyst independent
• TPA depends on the availability of function points
• Sufficient knowledge of the test team is a precondition
• TPA estimates are based on the assumption that, on average,
one complete re-test will be conducted
45© 2012 SQE Training V3.0
Team Estimates
• Teams often produce better estimates than
individuals (See The Wisdom of Crowds)
• Team estimates can facilitate buy-in and
commitment from the entire team
• Almost all estimating methods can be done
using teams
46© 2012 SQE Training V3.0
The Wisdom of Crowds
Four elements required to form a wise crowd:
• Diversity of opinion
• Independence
• Decentralization
• Aggregation
47© 2012 SQE Training V3.0
Test Estimation Sessions ― Wideband Delphi
48© 2012 SQE Training V3.0
Test Estimation Sessions ― Wideband Delphi
• Developed by Barry Boehm and John Farquhar
in the 1970s*
• A variant of the existing Delphi method but
with more interaction
• Popularized in Boehm’s book Software
Engineering Economics in 1981
• Similar to the Platinum Poker method used in
some agile projects
49© 2012 SQE Training V3.0
When would we use this technique?When would we use this technique?
Estimation for a new product or technology
To develop an “order of magnitude”
estimate
Requirements are shaky or incomplete
Multiple disciplines are involved
Wideband Delphi
50© 2012 SQE Training V3.0
Test Estimation Sessions — Wideband Delphi
Steps from Barry Boehm’s book:
1. Coordinator presents each expert with a specification
and estimation form
2. Experts discuss issues in group meeting
3. Experts fill out form anonymously
4. Coordinator distributes a summary
5. In a group meeting experts discuss variances
6. Experts fill out forms again for as many rounds as
necessary
51© 2012 SQE Training V3.0
Wideband Delphi Process Flow
Assemble
estimates
and
assumptions
Estimation
meeting
Individual
preparation
Planning and
kickoff
meeting
Re-estimate
Accept
able
Done
No Yes
Adapted from Karl Wiegers
52© 2012 SQE Training V3.0
Sample Estimation Form — Wideband Delphi
Task Estimate #1 Estimate #2 Estimate #3 Estimate #4 Final
Change
Total
53© 2012 SQE Training V3.0
Discussion Items:Discussion Items:
What are story points?
q
Measure of size/complexity, not time
q
Relative unit of measure
Is there any relation to function
points or test points?
Estimating Using Story Points ― Agile
54© 2012 SQE Training V3.0
Estimating Using Story Points ― Agile
55© 2012 SQE Training V3.0
Estimating Using Story Points ― Agile
• Story points are used for long term or release planning
and tracking
• Point-estimated stories along with team velocity can be
used to provide rough release-level scheduling and
project progress
• Since story points are relative size indicators, a two-
point story is always twice as big as a one-point story
56© 2012 SQE Training V3.0
Why Use Story Points?
• Cheaper
• Allow us to change our minds as new
information becomes available
• Don’t take a lot of time
• Foster collaboration
• Consistent
• Provide credibility
• Can use Planning Poker Cards with story points
57© 2012 SQE Training V3.0
Planning Poker
58© 2012 SQE Training V3.0
Planning Poker
• Based on the Wideband Delphi method espoused by Barry Boehm and
others in the 60s and 70s (and Boehm’s work was possibly based on
earlier works dating to the 40s)
• Most commonly used in agile software development
• A study published by the IEEE in April 2007 indicated that Planning
Poker achieved less optimistic and more accurate estimates than those
obtained through mechanical combinations of individual estimates
59© 2012 SQE Training V3.0
Planning Poker Rules
1. Form a group of no more than ten estimators and a moderator. The
product owner is usually present but cannot estimate
2. Each participant gets a deck of cards. The decks frequently use a
doubling sequence (½, 1, 2, 4, 8, 16, 32…) or the Fibonacci sequence (1,
2, 3, 5, 8, 13, 21, 34, 55, 89, 144…) or a “modified” Fibonacci sequence
such as (0, 1, 2, 3, 5, 8, 13, 20, 40, 100)
3. The moderator reads one user story at a time to the team
4. The product owner answers questions about the story
60© 2012 SQE Training V3.0
Planning Poker Rules (cont.)
5. Each estimator privately selects a card from his or her deck
representing their estimate of the “size” of the user story
6. When everyone is ready, all of the cards are flipped over at
the same time
7. If there is a consensus, the results are recorded, and the team
moves on to the next story
8. If the estimates vary widely, the owners of the high and low
estimates defend their positions to the rest of the team
61© 2012 SQE Training V3.0
Planning Poker Rules (cont.)
9. The group briefly debates the arguments
10. Repeat from step 5 until the estimates converge
11. Repeat for all stories
62© 2012 SQE Training V3.0
Planning Poker Rules ― Tips
1. Those who actually could do the work are the ones who vote
2. Managers don’t vote
3. When there is a tie in the voting between two sizes that are
consecutive, just pick the larger size and move on
4. Stop implementation discussions before they go too deep
5. Use an “I need a break” card
63© 2012 SQE Training V3.0
Planning Poker Rules ― Tips (cont.)
6. Use a timer to limit discussions
7. If consensus cannot be reached by the end of the third
round of voting, pick the largest size and move on
8. Have the person creating the user stories meet with QA and
Development leads prior to playing poker to answer the
most obvious questions
9. Remember the baseline
10.Have fun!
64© 2012 SQE Training V3.0
T-Shirt Sizing
An estimation technique, usually used in agile projects, that uses
relative sizing based upon a limited number of values (e.g., t-
shirt sizes): S, M, L, XL
Steps:
– Make S, M, L, XL cards
– Stories are read one at a time
– Each developer gives each story a t-shirt size
– All developers raise their cards simultaneously
– Discuss differences
– Go back to step 3
– Compile the stories into size buckets
– Estimate the time to complete a S, M, L,XL
65© 2012 SQE Training V3.0
Hadden’s Size/Complexity Technique
Rita Hadden
Small Medium Large
Simple
Moderate
Complex
Complexity
Size
66© 2012 SQE Training V3.0
Sample Estimate
67© 2012 SQE Training V3.0
Test Objective Size Complexity Effort
1 to 3 1 to 3 Sessions 1 2 3
to001 2 1 4 1 1 4 8
to002 2 3 16 2 4 8 16
to003 2 3 16 3 8 16 32
to004 2 1 4
to005 3 1 8
to006 2 2 8
to007 3 2 16
Typical Usage Scenarios 2 2 8
total 80 sessions
20 days
Size
Complexity
Simple Defect Estimation Models
• Defect Detection Percentage (DDP)
• Historical Extrapolation
• “Bug Budgets”
68© 2012 SQE Training V3.0
Karl Wiegers’s Estimation Safety Tips
• A goal is not an estimate
• The estimate you produce should be unrelated
to what you think the requester wants to hear
• The correct answer to any request for an
estimate is “Let me get back to you on that”
• Avoid giving single point estimates
• Incorporate contingency buffers into estimates
69© 2012 SQE Training V3.0
Rick Craig’s Tips for Better Estimates
• Do it!
• Collect metrics
• Remember the “fantasy” factor
• Don’t “pad” your estimates*
• Don’t spend a ton of time
• Estimates don’t have to be perfect
– Estimates are just estimates
– They will change/constantly as you re-estimate
– Remember planning risks and contingencies
– Remember Brooke’s Law
• If the date is fixed, estimate something else
• Use tools
• Use ranges of value instead of discrete numbers
70© 2012 SQE Training V3.0
Robert Sabourin’s Tips for Better Estimates
• Consider estimating the amount of test
coverage possible given a fixed amount
of testing effort
• Present stakeholders with alternatives
(Plan-a,Plan-b,Plan-9)
• Use ranges of values
• Highlight business organizational and
technical factors which impact the
estimate
• Using used envelopes to present rough
estimates
71© 2012 SQE Training V3.0
Thank You
• On behalf of SQE Training, we
appreciate your attendance at this
course
• If you have additional training or
consulting needs, please think of us
73© 2012 SQE Training V3.0
Appendix
Instructor Material
Brainstorming Testing Estimates
• Min Max Missing
– Identify participants
– Hold kick off meeting
– Individuals build list
– Logging meeting
– Estimate depth of testing
– Estimate size in sessions
– Identify new, missing, or
redundant ideas
75© 2012 SQE Training V3.0
BrainstormingTesting Ideas
• Min Max Missing
– PMI Estimation
– Given three values
q
MIN = BEST CASE
q
MAX = WORST CASE
q
EXPECTED = NORMAL CASE
q
Estimate = (MIN + 4*EXPECTED + MAX)/6
76© 2012 SQE Training V3.0
Effort Estimation
Reading
Steve McConnell, Rapid Development: Taming Wild
Software Schedules. Chapter 8
77© 2012 SQE Training V3.0
Estimating Software Projects
• Duration (on time)
– Schedule time
– Delivery
• Quality (on quality)
– Defect density
– Defect arrival
• Cost (on budget)
– Money
q
Fixed
q
Variable
– Opportunity
q
What else could we be doing?
78© 2012 SQE Training V3.0
Estimating Software Projects
• Effort
– Work
– Manpower
• Staffing
– Team definition
– Skills, experiences
79© 2012 SQE Training V3.0
Estimation Process
McConnell Basic Steps:
1. Estimate size
2. Estimate effort
3. Estimate schedule
80© 2012 SQE Training V3.0
Estimation Process
1 - Estimate Size
81© 2012 SQE Training V3.0
Estimation Size
• SLOC
– Source lines of code
• FP
– Function points
– Synthetic measure of software size
82© 2012 SQE Training V3.0
Estimation Size
• Software Tools
– Calibration base on relevant experience
– Combine several methods
– Estimator (free!)
q
www.construx.com
83© 2012 SQE Training V3.0
Estimation Size
• Analogy
– Similar past projects
– Relative to past experience
– Must take into account
q
Team
q
Technology
q
Complexity
q
Development process maturity
84© 2012 SQE Training V3.0
Estimating Size
• Algorithmic
– Function points
– Given requirements determine the number and
complexity of
q
Inputs
q
Outputs
q
Inquiries
q
Logical files
q
External interfaces
– Calculate and adjust based on “type of
project/technology”
85© 2012 SQE Training V3.0
Function Points
Low Complexity Medium Complexity High Complexity
Program Characteristic count multiplier count multiplier count multiplier total
Inputs 6 3 2 4 3 6 44
Outputs 7 4 7 5 0 7 63
Inquiries 0 3 2 4 4 6 32
Logical Internal Files 5 7 2 10 3 15 100
External Interface Files 9 5 0 7 2 10 65
Unadjusted Function Point Total 304
Influence Multiplier 1.15
Adjusted Function Point Total 349.6
86© 2012 SQE Training V3.0
Function Points ― Influence Factors
1 Data communications How many communication facilities are there to aid in
the transfer or exchange of information with the
application or system?
2 Distributed data processing How are distributed data and processing functions
handled?
3 Performance Did the user require response time or throughput?
4 Heavily used configuration How heavily used is the current hardware platform
where the application will be executed?
5 Transaction rate How frequently are transactions executed daily, weekly,
monthly, etc.?
6 On-line data entry What percentage of the information is entered on-line?
7 End-user efficiency Was the application designed for end-user efficiency?
87© 2012 SQE Training V3.0
Function Points ― Influence Factors
8 On-line update How many ILFs are updated by on-Llne transaction?
9 Complex processing Does the application have extensive logical or
mathematical processing?
10 Reusability Was the application developed to meet one or many
user’s (users’) needs?
11 Installation ease How difficult is conversion and installation?
12 Operational ease How effective and/or automated are start-up, back
up, and recovery procedures?
13 Multiple sites Was the application specifically designed,
developed, and supported to be installed at
multiple sites for multiple organizations?
14 Facilitate change Was the application specifically designed,
developed, and supported to facilitate change?
88© 2012 SQE Training V3.0
Function Points ― Influence Factors Score
Score Influence
0 Not present, or no influence
1 Incidental influence
2 Moderate influence
3 Average influence
4 Significant influence
5 Strong influence throughout
89© 2012 SQE Training V3.0
Function Points Influence Factors Score
INF = 0.65 + SCORE/100
90© 2012 SQE Training V3.0
Function Points ― Input
• Input
– Number of screens, forms, dialogues, controls, or
messages through which an end user or another
program adds, deletes, or changes data
91© 2012 SQE Training V3.0
Function Points ― Output
• Output
– Screens, reports, graphs, or messages generated for
use by end users or other programs
92© 2012 SQE Training V3.0
Function Points ― Inquiries
• Inquiries
– Direct access to data in database
93© 2012 SQE Training V3.0
Function Points ― Logical Files
• Logical Files
– Major groups of end user data; could be a “file” or
“database table”
94© 2012 SQE Training V3.0
Function Points ― External Interfaces
• External Interface Files
– Files controlled by other applications with which the
program interacts
95© 2012 SQE Training V3.0
Function Points ― Advantages
• Advantages include
– Technology independent
– Strong relationship to effort
– Commonly used
• Disadvantages include
– Requires specialized training to
compute
96© 2012 SQE Training V3.0
Estimation Process
2 - Estimate Effort
97© 2012 SQE Training V3.0
Effort Estimate
• Jones’s First-Order Estimation Practice
• Effort = (FP count) exponent
98© 2012 SQE Training V3.0
Jones’s First Order Exponents
Organization Capability
Kind of Software Best Average Worst
Systems 0.43 0.45 0.48
Business 0.41 0.43 0.46
Shrink Wrap 0.39 0.42 0.45
99© 2012 SQE Training V3.0
Estimation Process
3 - Estimate Schedule
100© 2012 SQE Training V3.0
Estimating Schedule
• Rule of Thumb (McConnell 8-1)
• Schedule = 3.0 * (effort) 1/3
101© 2012 SQE Training V3.0
Estimating Schedule
• Software Engineering Past Data
– Example McConnell table 8-8, 8-9
q
LOC vs. Schedule and Effort
q
Different project types
102© 2012 SQE Training V3.0
Estimation Convergence
Effort Range Schedule Range
Project Phase Optimistic Pessimistic Optimistic Pessimistic
Initial product definition 0.25 4.00 0.60 1.60
Approved product definition 0.50 2.00 0.80 1.25
Requirement specification 0.67 1.50 0.85 1.15
Product design specification 0.80 1.25 0.90 1.10
Detailed design specification 0.90 1.10 0.95 1.05
Product complete 1.00 1.00 1.00 1.00
103© 2012 SQE Training V3.0
Initial
product
definition
Approved
product
definition
Requirements
specification
Product
design
specification
Detailed
design
specification
Product
complete
1.0×
0.25×
4×
2×
0.5×
1.5×
0.67×
1.25×
0.8×
1.0×
0.6×
1.6×
1.25×
0.8×
1.15×
0.85×
1.1×
0.9×
Project Cost (effort and size) Project Schedule
Estimate ― Convergence Graph
104© 2012 SQE Training V3.0
More Relationships
Code Volume Approx. 100 LOC/FP, varies widely
[Pressman, p. 94]
Schedule (FP)0.4
Calendar months
Staffing (FP)/100
Average – follows Raleigh curve
Effort Schedule * Staffing = (FP1.4)/150
105© 2012 SQE Training V3.0
Rules of Thumb
106© 2012 SQE Training V3.0
Rules of Thumb (cont.)
• Experience from several projects
indicates that the amount of testing
required in a project is proportional to
the amount of development required
• Ratio depends on
– History
– Organization, culture, structure
– Projects
– Teams
107© 2012 SQE Training V3.0
Rules of Thumb (cont.)
• Hadden’s Size/Complexity Technique
Small Medium Large
Simple
Moderate
Complex
Complexity
Size
108© 2012 SQE Training V3.0
Developer/Tester Ratios
109© 2012 SQE Training V3.0
Sample Estimate Project A
110© 2012 SQE Training V3.0
Test Objective Size Complexity Effort
1 to 3 1 to 3 Sessions 1 2 3
to001 2 1 4 1 1 4 8
to002 2 3 16 2 4 8 16
to003 2 3 16 3 8 16 32
to004 2 1 4
to005 3 1 8
to006 2 2 8
to007 3 2 16
Typical Usage Scenarios 2 2 8
total 80 sessions
20 days
Size
Complexity
Sample Estimate Project B
111© 2012 SQE Training V3.0
Test Objective Size Complexity Effort
1 to 3 1 to 3 Sessions 1 2 3
to001 1 1 1 1 1 4 8
to002 1 1 1 2 4 8 16
to003 2 1 4 3 8 16 32
to004 2 1 4
to005 2 1 4
to006 2 1 4
to007 1 1 1
to008 1 1 1
to009 3 2 16
to010 2 2 8
to011 2 2 8
to012 2 2 8
to013 2 2 8
to014 3 2 16
to015 3 2 16
to016 2 2 8
Typical Usage Scenarios 2 2 8
total 116 sessions
29 days
Complexity
Size
Sample Estimate Project C
112© 2012 SQE Training V3.0
Test Objective Size Complexity Effort
1 to 3 1 to 3 Sessions 1 2 3
to001 3 2 16 1 1 4 8
to002 1 1 1 2 4 8 16
to003 1 1 1 3 8 16 32
to004 1 1 1
to005 1 1 1
to006 1 1 1
to007 1 2 4
to008 1 2 4
to009 1 1 1
to010 1 2 4
to011 1 2 4
to012 2 2 8
to013 2 2 8
to014 2 2 8
to015 1 1 1
to016 1 1 1
to017 2 1 2
to018 2 2 8
to019 3 2 16
Typical Usage Scenarios 2 2 8
total 98 sessions
24.5 days
Complexity
Size
Sample Estimate Project D
113© 2012 SQE Training V3.0
Test Objective Size Complexity Effort
1 to 3 1 to 3 Sessions 1 2 3
t001 1 1 1 1 1 4 8
t002 1 1 1 2 4 8 16
t003 1 1 1 3 8 16 32
t004 2 1 4
t005 2 1 4
t006 2 2 8
t007 1 1 1
t008 1 1 1
t009 1 1 1
t010 2 1 4
t011 2 2 8
t012 1 1 1
t013 1 2 4
t014 1 2 4
t015 1 2 4
t016 1 1 1
t017 2 1 4
t018 1 1 1
t019 1 2 4
t020 2 1 4
t021 1 1 1
t022 1 1 1
t023 2 2 4
t024 1 1 1
t025 2 1 4
t026 1 1 1
t027 1 1 1
t028 2 1 4
t029 1 2 4
t030 1 2 4
t031 2 1 4
t032 1 1 1
Scenarios 2 2 8
total 99 sessions
24.75 days
Complexity
Size

Contenu connexe

Tendances

Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle Kari Kakkonen
 
Chapter 5 - Test Automation Reporting and Metrics
Chapter 5 - Test Automation Reporting and MetricsChapter 5 - Test Automation Reporting and Metrics
Chapter 5 - Test Automation Reporting and MetricsNeeraj Kumar Singh
 
Test Early, Test Often, Test Left
Test Early, Test Often, Test LeftTest Early, Test Often, Test Left
Test Early, Test Often, Test LeftSmartBear
 
Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8a34sharm
 
Testing in Agile Projects
Testing in Agile ProjectsTesting in Agile Projects
Testing in Agile Projectssriks7
 
Risk based testing a new case study
Risk based testing   a new case studyRisk based testing   a new case study
Risk based testing a new case studyBassam Al-Khatib
 
Software Testing Services
Software Testing ServicesSoftware Testing Services
Software Testing ServicesFuad Mak
 
ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2Yogindernath Gupta
 
Test Strategy-The real silver bullet in testing by Matthew Eakin
Test Strategy-The real silver bullet in testing by Matthew EakinTest Strategy-The real silver bullet in testing by Matthew Eakin
Test Strategy-The real silver bullet in testing by Matthew EakinQA or the Highway
 
ISTQB Foundation Level Basic
ISTQB Foundation Level BasicISTQB Foundation Level Basic
ISTQB Foundation Level BasicErol Selitektay
 
Software Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | EdurekaSoftware Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | EdurekaEdureka!
 
Introducing QA Into an Agile Environment
Introducing QA Into an Agile EnvironmentIntroducing QA Into an Agile Environment
Introducing QA Into an Agile EnvironmentJoseph Beale
 
Manual testing interview questions and answers
Manual testing interview questions and answersManual testing interview questions and answers
Manual testing interview questions and answersTestbytes
 
Risk-Based Testing for Agile Projects
Risk-Based Testing for Agile ProjectsRisk-Based Testing for Agile Projects
Risk-Based Testing for Agile ProjectsTechWell
 
Risk-based Testing
Risk-based TestingRisk-based Testing
Risk-based TestingJohan Hoberg
 
20180421 Issueの書き方と伝えかた勉強会
20180421 Issueの書き方と伝えかた勉強会20180421 Issueの書き方と伝えかた勉強会
20180421 Issueの書き方と伝えかた勉強会Kouichi Akiyama
 

Tendances (20)

Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle
 
ISTQB Advanced Test Manager Training 2012 - Testing Process
ISTQB Advanced Test Manager Training 2012 - Testing Process ISTQB Advanced Test Manager Training 2012 - Testing Process
ISTQB Advanced Test Manager Training 2012 - Testing Process
 
Chapter 5 - Test Automation Reporting and Metrics
Chapter 5 - Test Automation Reporting and MetricsChapter 5 - Test Automation Reporting and Metrics
Chapter 5 - Test Automation Reporting and Metrics
 
Test Early, Test Often, Test Left
Test Early, Test Often, Test LeftTest Early, Test Often, Test Left
Test Early, Test Often, Test Left
 
Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8Role Of Qa And Testing In Agile 1225221397167302 8
Role Of Qa And Testing In Agile 1225221397167302 8
 
Testing in Agile Projects
Testing in Agile ProjectsTesting in Agile Projects
Testing in Agile Projects
 
Risk based testing a new case study
Risk based testing   a new case studyRisk based testing   a new case study
Risk based testing a new case study
 
Introduction to Software Test Automation
Introduction to Software Test AutomationIntroduction to Software Test Automation
Introduction to Software Test Automation
 
Testing Centre Of Excellence From AppLabs
Testing Centre Of Excellence From AppLabsTesting Centre Of Excellence From AppLabs
Testing Centre Of Excellence From AppLabs
 
Software Testing Services
Software Testing ServicesSoftware Testing Services
Software Testing Services
 
ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2
 
Test Strategy-The real silver bullet in testing by Matthew Eakin
Test Strategy-The real silver bullet in testing by Matthew EakinTest Strategy-The real silver bullet in testing by Matthew Eakin
Test Strategy-The real silver bullet in testing by Matthew Eakin
 
ISTQB Foundation Level Basic
ISTQB Foundation Level BasicISTQB Foundation Level Basic
ISTQB Foundation Level Basic
 
Software Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | EdurekaSoftware Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
Software Testing Life Cycle (STLC) | Software Testing Tutorial | Edureka
 
Introducing QA Into an Agile Environment
Introducing QA Into an Agile EnvironmentIntroducing QA Into an Agile Environment
Introducing QA Into an Agile Environment
 
Manual testing interview questions and answers
Manual testing interview questions and answersManual testing interview questions and answers
Manual testing interview questions and answers
 
Risk-Based Testing for Agile Projects
Risk-Based Testing for Agile ProjectsRisk-Based Testing for Agile Projects
Risk-Based Testing for Agile Projects
 
Risk-based Testing
Risk-based TestingRisk-based Testing
Risk-based Testing
 
20180421 Issueの書き方と伝えかた勉強会
20180421 Issueの書き方と伝えかた勉強会20180421 Issueの書き方と伝えかた勉強会
20180421 Issueの書き方と伝えかた勉強会
 
Unit testing, principles
Unit testing, principlesUnit testing, principles
Unit testing, principles
 

Similaire à Test Estimation in Practice

Getting Started with Risk-based Testing
Getting Started with Risk-based TestingGetting Started with Risk-based Testing
Getting Started with Risk-based TestingTechWell
 
Pm0017 project quality management
Pm0017 project quality managementPm0017 project quality management
Pm0017 project quality managementconsult4solutions
 
Risk Driven Testing
Risk Driven TestingRisk Driven Testing
Risk Driven TestingJorge Boria
 
SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...
SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...
SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...Steelwedge
 
Project Management Framework
Project Management FrameworkProject Management Framework
Project Management FrameworkRahul Sudame
 
Software Test Estimation
Software Test EstimationSoftware Test Estimation
Software Test EstimationJatin Kochhar
 
PMP preparation and PMI Framework
PMP preparation and PMI FrameworkPMP preparation and PMI Framework
PMP preparation and PMI FrameworkUzma Khan
 
Big Apple Scrum Day 2015 - Advanced Scrum Metrics Presentation
Big Apple Scrum Day 2015 - Advanced Scrum Metrics PresentationBig Apple Scrum Day 2015 - Advanced Scrum Metrics Presentation
Big Apple Scrum Day 2015 - Advanced Scrum Metrics PresentationJason Tice
 
Test Planning and Test Estimation Techniques
Test Planning and Test Estimation TechniquesTest Planning and Test Estimation Techniques
Test Planning and Test Estimation TechniquesMurageppa-QA
 
Software Quality Metrics for Testers - StarWest 2013
Software Quality Metrics for Testers - StarWest 2013Software Quality Metrics for Testers - StarWest 2013
Software Quality Metrics for Testers - StarWest 2013XBOSoft
 
Improve phase lean six sigma tollgate template
Improve phase   lean six sigma tollgate templateImprove phase   lean six sigma tollgate template
Improve phase lean six sigma tollgate templateSteven Bonacorsi
 
Improve phase lean six sigma tollgate template
Improve phase   lean six sigma tollgate templateImprove phase   lean six sigma tollgate template
Improve phase lean six sigma tollgate templateSteven Bonacorsi
 
Asset Finance Systems: Project Initiation "101"
Asset Finance Systems: Project Initiation "101"Asset Finance Systems: Project Initiation "101"
Asset Finance Systems: Project Initiation "101"David Pedreno
 
Pm0011 project planning and scheduling
Pm0011 project planning and schedulingPm0011 project planning and scheduling
Pm0011 project planning and schedulingconsult4solutions
 
Best practice for_agile_ds_projects
Best practice for_agile_ds_projectsBest practice for_agile_ds_projects
Best practice for_agile_ds_projectsKhalid Kahloot
 
Software testing career growth path explained
Software testing career growth path explainedSoftware testing career growth path explained
Software testing career growth path explainedintervietips
 
Project Report on Employee Management System.docx
Project Report on Employee Management System.docxProject Report on Employee Management System.docx
Project Report on Employee Management System.docxDhineshkumarPrakasam
 
Essential Test Management and Planning
Essential Test Management and PlanningEssential Test Management and Planning
Essential Test Management and PlanningTechWell
 

Similaire à Test Estimation in Practice (20)

Business Analyst interview Questions
Business Analyst interview QuestionsBusiness Analyst interview Questions
Business Analyst interview Questions
 
Getting Started with Risk-based Testing
Getting Started with Risk-based TestingGetting Started with Risk-based Testing
Getting Started with Risk-based Testing
 
Pm0017 project quality management
Pm0017 project quality managementPm0017 project quality management
Pm0017 project quality management
 
Risk Driven Testing
Risk Driven TestingRisk Driven Testing
Risk Driven Testing
 
SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...
SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...
SCOR Project Workshop - Sales & Operations Planning (S&OP) Health Check - How...
 
Project Management Framework
Project Management FrameworkProject Management Framework
Project Management Framework
 
Software Test Estimation
Software Test EstimationSoftware Test Estimation
Software Test Estimation
 
PMP preparation and PMI Framework
PMP preparation and PMI FrameworkPMP preparation and PMI Framework
PMP preparation and PMI Framework
 
Big Apple Scrum Day 2015 - Advanced Scrum Metrics Presentation
Big Apple Scrum Day 2015 - Advanced Scrum Metrics PresentationBig Apple Scrum Day 2015 - Advanced Scrum Metrics Presentation
Big Apple Scrum Day 2015 - Advanced Scrum Metrics Presentation
 
Test Planning and Test Estimation Techniques
Test Planning and Test Estimation TechniquesTest Planning and Test Estimation Techniques
Test Planning and Test Estimation Techniques
 
Software Quality Metrics for Testers - StarWest 2013
Software Quality Metrics for Testers - StarWest 2013Software Quality Metrics for Testers - StarWest 2013
Software Quality Metrics for Testers - StarWest 2013
 
Profile Seema Wadhwa
Profile Seema WadhwaProfile Seema Wadhwa
Profile Seema Wadhwa
 
Improve phase lean six sigma tollgate template
Improve phase   lean six sigma tollgate templateImprove phase   lean six sigma tollgate template
Improve phase lean six sigma tollgate template
 
Improve phase lean six sigma tollgate template
Improve phase   lean six sigma tollgate templateImprove phase   lean six sigma tollgate template
Improve phase lean six sigma tollgate template
 
Asset Finance Systems: Project Initiation "101"
Asset Finance Systems: Project Initiation "101"Asset Finance Systems: Project Initiation "101"
Asset Finance Systems: Project Initiation "101"
 
Pm0011 project planning and scheduling
Pm0011 project planning and schedulingPm0011 project planning and scheduling
Pm0011 project planning and scheduling
 
Best practice for_agile_ds_projects
Best practice for_agile_ds_projectsBest practice for_agile_ds_projects
Best practice for_agile_ds_projects
 
Software testing career growth path explained
Software testing career growth path explainedSoftware testing career growth path explained
Software testing career growth path explained
 
Project Report on Employee Management System.docx
Project Report on Employee Management System.docxProject Report on Employee Management System.docx
Project Report on Employee Management System.docx
 
Essential Test Management and Planning
Essential Test Management and PlanningEssential Test Management and Planning
Essential Test Management and Planning
 

Plus de TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and RecoveringTechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartTechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyTechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowTechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityTechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyTechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipTechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsTechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GameTechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsTechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationTechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessTechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateTechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessTechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTechWell
 

Plus de TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Dernier

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfhans926745
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 

Dernier (20)

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 

Test Estimation in Practice

  • 1. TR Half-day Tutorials 5/6/2014 1:00:00 PM Test Estimation in Practice Presented by: Rob Sabourin AmiBug.com Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2. Rob Sabourin AmiBug.com Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of software development professionals. A well-respected member of the software engineering community, Rob has managed, trained, mentored, and coached hundreds of top professionals in the field. He frequently speaks at conferences and writes on software engineering, SQA, testing, management, and internationalization. Rob wrote I am a Bug!, the popular software testing children's book; works as an adjunct professor of software engineering at McGill University; and serves as the principle consultant (and president/janitor) of AmiBug.Com, Inc. Contact Rob at Contact Rob at rsabourin@amibug.com.
  • 6. Course Agenda 1. Estimation 2. Estimation Techniques 8© 2012 SQE Training V3.0
  • 8. Estimation Estimate: – A tentative evaluation or rough calculation – A preliminary calculation of the cost of a project – A judgment based upon one’s impressions; opinion —The American Heritage Dictionary It is very difficult to make a vigorous, plausible, and job- risking estimate that is derived by no quantitative method, supported by little data and certified chiefly by hunches of the managers. — Fred Brooks 10© 2012 SQE Training V3.0
  • 9. The best estimatesThe best estimates Represent the collective wisdom of practitioners (testers, developers, users, etc.) and have their buy-in Provide specific, detailed catalogs of the costs, resources, tasks, and people involved Present, for each activity estimated, the most likely cost, effort, and duration Estimation: the creation of an approximate target for costs and completion dates Estimation: the creation of an approximate target for costs and completion dates Test Estimation 11© 2012 SQE Training V3.0
  • 10. Factors that can influence cost, effort, and duration include:Factors that can influence cost, effort, and duration include: Required level of quality of the system Size of the system to be tested Historical data Process factors (process maturity, etc.) Material factors (tools, data, etc.) People factors (skills, experience, managers, etc.) Test Estimation (cont.) 12© 2012 SQE Training V3.0
  • 11. Test Estimation (cont.) • Delivery of estimates should include justification • Negotiation and re-work of estimates is normal • Final estimates represent a balance of organizational and project goals in the areas of quality, schedule, budget, and features 13© 2012 SQE Training V3.0
  • 12. How Good an Estimator Are You? Low Estimate High Estimate ___________ to ____________ . . . © Steve McConnell, Software Estimation, Microsoft Press, 2006 Surface temperature of the sun Latitude of ShanghaI Area of the Asian continent Birth year of Alexander the Great Total value of US currency in 2004 Total volume of the Great Lakes Worldwide box office receipts of Titanic Total length of Pacific Ocean coastline Heaviest blue whale ever recorded # of books published in US since 1776 14© 2012 SQE Training V3.0
  • 13. How Good an Estimator Are You? • Surface temperature of the sun 10,0000 F / 6,000 C • Latitude of ShanghaI 31° 10′ N • Area of the Asian continent 17,212,000 sq mi • Birth year of Alexander the Great 356 BC • Total value of US currency in 2004 $719.9 Billion • Total volume of the Great Lakes 5,500 cu mi • Worldwide box office receipts of Titanic $1.835 billion • Total length of Pacific Ocean coastline 84,300 miles • Heaviest blue whale ever recorded 380,000 pounds • Number of books published in US since 1776 22 million 15© 2012 SQE Training V3.0
  • 14. How Good Is Our Industry (at Estimating)? • Tata: 62% of projects fail to meet schedule 49% have budget overruns • Moiokken and Jorgensen: 30-40% overruns 16© 2012 SQE Training V3.0
  • 15. Class Discussion Why is estimating not done well? Your top five reasons: 1) Too many variables____________________ 2) ____________________________________ 5) ____________________________________ 6) ____________________________________ 5) ____________________________________ 17© 2012 SQE Training V3.0
  • 16. Why Estimates Are Inaccurate ― Part I • Lack of estimating experience • Lack of historical data on which to base estimates • Lack of systematic estimation process, sound techniques, or models suited to the project • Failure to include essential activities and products within the scope of the estimates • Unrealistic expectations or assumptions • Failure to recognize and address the uncertainty inherent in project estimates Practical Software Measurement Addison-Wesley, 2001 18© 2012 SQE Training V3.0
  • 17. Why Estimates Are Inaccurate ― Part II • Lack of education and training • Confusing the target with the estimate • Hope-based planning • Inability to communicate and support estimates • Incomplete, changing, and creeping requirements • Quality surprises (test and re-fix) —adapted from Linda M. Laird The Limitations of Estimation 19© 2012 SQE Training V3.0
  • 18. Bohem’s Cone of Uncertainty 20© 2012 SQE Training V3.0
  • 19. NHC Track Forecast Cone 21© 2012 SQE Training V3.0
  • 20. “Testing” Track Forecast Cone Best case Expected case Worst case T i m e Tasks (or why it is important to constantly re-estimate) 22© 2012 SQE Training V3.0
  • 21. What would have to happen to deliver this in four weeks? What would have to happen to deliver this in four weeks? What should the estimate have been?What should the estimate have been? The Fantasy Factor Weeks Today 0 1 2 3 4 5 6 7 8 9 1st 3rd2nd 23© 2012 SQE Training V3.0
  • 22. Tim e Tim e SizeSize Quali ty Quali ty Res our ces Res our ces Estimation 1, 2, 3, or 4 Variables + Many Modifiers: If it’s not variable, then it’s fixed. 24© 2012 SQE Training V3.0
  • 23. Time vs. Resources = 25© 2012 SQE Training V3.0
  • 25. Test Estimation Techniques ― Examples • Intuition and guess • Work-breakdown-structures • Three-point estimates • Company standards and norms • % of project effort or staffing • Industry averages and predictive models (e.g., FP, TPA ) • Team estimation sessions – Wideband Delphi – Story point sizing – Poker estimation – T-shirt sizing 28© 2012 SQE Training V3.0
  • 26. What do intuition and guess imply? What do intuition and guess imply? No formal estimation process Based on the experience of the team member(s) usually without the benefit of recorded metrics Intuition and Guess 29© 2012 SQE Training V3.0
  • 27. When would we use this technique?When would we use this technique? Experienced staff members Little or no recorded metrics available Great accuracy is not required Same type of project we’ve done in the past Or at the other extreme, we don’t have a clue (often called a WAG or SWAG) Intuition and Guess (cont.) 30© 2012 SQE Training V3.0
  • 28. Project LevelProject Level Phase Level Phase Level Task LevelTask Level Phase Level Phase Level Task LevelTask Level ------ Task LevelTask Level ------ ------ ------ Task LevelTask Level ------ Phase Level Phase Level Task LevelTask Level Phase Level Phase Level Task LevelTask Level Work Breakdown Structure 31© 2012 SQE Training V3.0
  • 29. When would we use this technique?When would we use this technique? When the test team lacks credibility with their estimates When we are doing an estimate for a project that is dissimilar to projects we have done in the past When the project is too large to get our “arms around it” (e.g., it is too big to easily see the big picture) Work Breakdown Structure (cont.) 32© 2012 SQE Training V3.0
  • 30. When would we not use this technique?When would we not use this technique? When the project has lots of turbulence! Work Breakdown Structure (cont.) 33© 2012 SQE Training V3.0
  • 31. Approach:Approach: Develop goal Break goal into deliverables Break deliverables into activities Decompose work into small manageable components that are of sufficient detail to allow you to make estimates of time and/or resources Work Breakdown Structure ― Top Down 34© 2012 SQE Training V3.0
  • 32. Work Breakdown Structure (cont.) Testing for Completeness • Status of a task is measurable • Start and end events are clearly defined • Further breakdown makes no sense • Each activity has a deliverable • Time or resources for an activity can be estimated • Duration of activity is within acceptable limits • Each activity is independent • Each activity forms a unique package of work that could be outsourced 35© 2012 SQE Training V3.0
  • 33. Work Breakdown Structure (cont.) • Level of detail guidelines: – No single activity or group of activities should exceed 80 hours of effort* – No activity or group of activities should exceed the reporting period (e.g., if the reporting period is weekly, no activity should exceed one week) – “If it makes sense” rule: The duration of each activity must pass the common sense rule 36© 2012 SQE Training V3.0
  • 34. Three–Point Estimates • Three-point estimates is an analytical technique using three cost or duration estimates: – Optimistic – Most likely – Pessimistic • This technique is applied to improve the accuracy of the estimates of cost or duration • Three-point estimation techniques can be employed with other estimating techniques such as Delphi 37© 2012 SQE Training V3.0
  • 35. % of Project Effort or Staffing • Based on the premise that there is a predictable correlation between development effort/time and test effort/time • The preferred/“dictated” method of estimation in some organizations • Sometimes associated with developer/tester ratios • One method recognized by the ISTQB 38© 2012 SQE Training V3.0
  • 36. When Would We Use This Technique?When Would We Use This Technique? An organization is producing more or less the same type of product on an ongoing basis The organization (development and testing) uses consistent practices and has a stable staff A crude estimate is needed It is mandated It works % of Project Effort or Staffing 39© 2012 SQE Training V3.0
  • 37. Issues:Issues: We are basing our estimates on development estimates which are probably suspect Some products that are relatively easy to develop may be difficult to test and vice versa This technique may not take into account test automation, etc. This method may not take the stated quality goals into account Test Estimates Based on Development Estimates 40© 2012 SQE Training V3.0
  • 38. Class Discussion What is the ideal developer/tester ratio? = ? 41© 2012 SQE Training V3.0
  • 39. McConnell Developer to Tester Ratio Environment Relative Importance Ratio (Developer:Tester Ratio) Business Systems 3:1 to 20:1 Shrink Wrap 1:1 to 5:1 Scientific Projects 5:1 to 20:1 Systems Projects 1:1 to 5:1 Safety Critical Software 5:1 to 1:2 Microsoft Windows 2000 1:2 NASA Space Shuttle Flight Control Software 1:10 42© 2012 SQE Training V3.0
  • 40. Test Point Analysis • A method with the possibility to perform a technology-independent measurement of the test size of an information system on the basis of a function point analysis, and to use this measurement as a basis for a productivity measurement, an estimate of the required resources, and project management Software Testing: A Guide to the TMap Approach Martin Pol, Ruud Teunissen, Erik van Veenendaal 43© 2012 SQE Training V3.0
  • 41. Test Point Analysis (cont.) For more details on TPA, look in this book. 44© 2012 SQE Training V3.0
  • 42. Principles of Test Point Analysis • TPA only considers measurable quality characteristics that fall within the range of system or acceptance testing • TPA is (in principle) analyst independent • TPA depends on the availability of function points • Sufficient knowledge of the test team is a precondition • TPA estimates are based on the assumption that, on average, one complete re-test will be conducted 45© 2012 SQE Training V3.0
  • 43. Team Estimates • Teams often produce better estimates than individuals (See The Wisdom of Crowds) • Team estimates can facilitate buy-in and commitment from the entire team • Almost all estimating methods can be done using teams 46© 2012 SQE Training V3.0
  • 44. The Wisdom of Crowds Four elements required to form a wise crowd: • Diversity of opinion • Independence • Decentralization • Aggregation 47© 2012 SQE Training V3.0
  • 45. Test Estimation Sessions ― Wideband Delphi 48© 2012 SQE Training V3.0
  • 46. Test Estimation Sessions ― Wideband Delphi • Developed by Barry Boehm and John Farquhar in the 1970s* • A variant of the existing Delphi method but with more interaction • Popularized in Boehm’s book Software Engineering Economics in 1981 • Similar to the Platinum Poker method used in some agile projects 49© 2012 SQE Training V3.0
  • 47. When would we use this technique?When would we use this technique? Estimation for a new product or technology To develop an “order of magnitude” estimate Requirements are shaky or incomplete Multiple disciplines are involved Wideband Delphi 50© 2012 SQE Training V3.0
  • 48. Test Estimation Sessions — Wideband Delphi Steps from Barry Boehm’s book: 1. Coordinator presents each expert with a specification and estimation form 2. Experts discuss issues in group meeting 3. Experts fill out form anonymously 4. Coordinator distributes a summary 5. In a group meeting experts discuss variances 6. Experts fill out forms again for as many rounds as necessary 51© 2012 SQE Training V3.0
  • 49. Wideband Delphi Process Flow Assemble estimates and assumptions Estimation meeting Individual preparation Planning and kickoff meeting Re-estimate Accept able Done No Yes Adapted from Karl Wiegers 52© 2012 SQE Training V3.0
  • 50. Sample Estimation Form — Wideband Delphi Task Estimate #1 Estimate #2 Estimate #3 Estimate #4 Final Change Total 53© 2012 SQE Training V3.0
  • 51. Discussion Items:Discussion Items: What are story points? q Measure of size/complexity, not time q Relative unit of measure Is there any relation to function points or test points? Estimating Using Story Points ― Agile 54© 2012 SQE Training V3.0
  • 52. Estimating Using Story Points ― Agile 55© 2012 SQE Training V3.0
  • 53. Estimating Using Story Points ― Agile • Story points are used for long term or release planning and tracking • Point-estimated stories along with team velocity can be used to provide rough release-level scheduling and project progress • Since story points are relative size indicators, a two- point story is always twice as big as a one-point story 56© 2012 SQE Training V3.0
  • 54. Why Use Story Points? • Cheaper • Allow us to change our minds as new information becomes available • Don’t take a lot of time • Foster collaboration • Consistent • Provide credibility • Can use Planning Poker Cards with story points 57© 2012 SQE Training V3.0
  • 55. Planning Poker 58© 2012 SQE Training V3.0
  • 56. Planning Poker • Based on the Wideband Delphi method espoused by Barry Boehm and others in the 60s and 70s (and Boehm’s work was possibly based on earlier works dating to the 40s) • Most commonly used in agile software development • A study published by the IEEE in April 2007 indicated that Planning Poker achieved less optimistic and more accurate estimates than those obtained through mechanical combinations of individual estimates 59© 2012 SQE Training V3.0
  • 57. Planning Poker Rules 1. Form a group of no more than ten estimators and a moderator. The product owner is usually present but cannot estimate 2. Each participant gets a deck of cards. The decks frequently use a doubling sequence (½, 1, 2, 4, 8, 16, 32…) or the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144…) or a “modified” Fibonacci sequence such as (0, 1, 2, 3, 5, 8, 13, 20, 40, 100) 3. The moderator reads one user story at a time to the team 4. The product owner answers questions about the story 60© 2012 SQE Training V3.0
  • 58. Planning Poker Rules (cont.) 5. Each estimator privately selects a card from his or her deck representing their estimate of the “size” of the user story 6. When everyone is ready, all of the cards are flipped over at the same time 7. If there is a consensus, the results are recorded, and the team moves on to the next story 8. If the estimates vary widely, the owners of the high and low estimates defend their positions to the rest of the team 61© 2012 SQE Training V3.0
  • 59. Planning Poker Rules (cont.) 9. The group briefly debates the arguments 10. Repeat from step 5 until the estimates converge 11. Repeat for all stories 62© 2012 SQE Training V3.0
  • 60. Planning Poker Rules ― Tips 1. Those who actually could do the work are the ones who vote 2. Managers don’t vote 3. When there is a tie in the voting between two sizes that are consecutive, just pick the larger size and move on 4. Stop implementation discussions before they go too deep 5. Use an “I need a break” card 63© 2012 SQE Training V3.0
  • 61. Planning Poker Rules ― Tips (cont.) 6. Use a timer to limit discussions 7. If consensus cannot be reached by the end of the third round of voting, pick the largest size and move on 8. Have the person creating the user stories meet with QA and Development leads prior to playing poker to answer the most obvious questions 9. Remember the baseline 10.Have fun! 64© 2012 SQE Training V3.0
  • 62. T-Shirt Sizing An estimation technique, usually used in agile projects, that uses relative sizing based upon a limited number of values (e.g., t- shirt sizes): S, M, L, XL Steps: – Make S, M, L, XL cards – Stories are read one at a time – Each developer gives each story a t-shirt size – All developers raise their cards simultaneously – Discuss differences – Go back to step 3 – Compile the stories into size buckets – Estimate the time to complete a S, M, L,XL 65© 2012 SQE Training V3.0
  • 63. Hadden’s Size/Complexity Technique Rita Hadden Small Medium Large Simple Moderate Complex Complexity Size 66© 2012 SQE Training V3.0
  • 64. Sample Estimate 67© 2012 SQE Training V3.0 Test Objective Size Complexity Effort 1 to 3 1 to 3 Sessions 1 2 3 to001 2 1 4 1 1 4 8 to002 2 3 16 2 4 8 16 to003 2 3 16 3 8 16 32 to004 2 1 4 to005 3 1 8 to006 2 2 8 to007 3 2 16 Typical Usage Scenarios 2 2 8 total 80 sessions 20 days Size Complexity
  • 65. Simple Defect Estimation Models • Defect Detection Percentage (DDP) • Historical Extrapolation • “Bug Budgets” 68© 2012 SQE Training V3.0
  • 66. Karl Wiegers’s Estimation Safety Tips • A goal is not an estimate • The estimate you produce should be unrelated to what you think the requester wants to hear • The correct answer to any request for an estimate is “Let me get back to you on that” • Avoid giving single point estimates • Incorporate contingency buffers into estimates 69© 2012 SQE Training V3.0
  • 67. Rick Craig’s Tips for Better Estimates • Do it! • Collect metrics • Remember the “fantasy” factor • Don’t “pad” your estimates* • Don’t spend a ton of time • Estimates don’t have to be perfect – Estimates are just estimates – They will change/constantly as you re-estimate – Remember planning risks and contingencies – Remember Brooke’s Law • If the date is fixed, estimate something else • Use tools • Use ranges of value instead of discrete numbers 70© 2012 SQE Training V3.0
  • 68. Robert Sabourin’s Tips for Better Estimates • Consider estimating the amount of test coverage possible given a fixed amount of testing effort • Present stakeholders with alternatives (Plan-a,Plan-b,Plan-9) • Use ranges of values • Highlight business organizational and technical factors which impact the estimate • Using used envelopes to present rough estimates 71© 2012 SQE Training V3.0
  • 69. Thank You • On behalf of SQE Training, we appreciate your attendance at this course • If you have additional training or consulting needs, please think of us 73© 2012 SQE Training V3.0
  • 71. Brainstorming Testing Estimates • Min Max Missing – Identify participants – Hold kick off meeting – Individuals build list – Logging meeting – Estimate depth of testing – Estimate size in sessions – Identify new, missing, or redundant ideas 75© 2012 SQE Training V3.0
  • 72. BrainstormingTesting Ideas • Min Max Missing – PMI Estimation – Given three values q MIN = BEST CASE q MAX = WORST CASE q EXPECTED = NORMAL CASE q Estimate = (MIN + 4*EXPECTED + MAX)/6 76© 2012 SQE Training V3.0
  • 73. Effort Estimation Reading Steve McConnell, Rapid Development: Taming Wild Software Schedules. Chapter 8 77© 2012 SQE Training V3.0
  • 74. Estimating Software Projects • Duration (on time) – Schedule time – Delivery • Quality (on quality) – Defect density – Defect arrival • Cost (on budget) – Money q Fixed q Variable – Opportunity q What else could we be doing? 78© 2012 SQE Training V3.0
  • 75. Estimating Software Projects • Effort – Work – Manpower • Staffing – Team definition – Skills, experiences 79© 2012 SQE Training V3.0
  • 76. Estimation Process McConnell Basic Steps: 1. Estimate size 2. Estimate effort 3. Estimate schedule 80© 2012 SQE Training V3.0
  • 77. Estimation Process 1 - Estimate Size 81© 2012 SQE Training V3.0
  • 78. Estimation Size • SLOC – Source lines of code • FP – Function points – Synthetic measure of software size 82© 2012 SQE Training V3.0
  • 79. Estimation Size • Software Tools – Calibration base on relevant experience – Combine several methods – Estimator (free!) q www.construx.com 83© 2012 SQE Training V3.0
  • 80. Estimation Size • Analogy – Similar past projects – Relative to past experience – Must take into account q Team q Technology q Complexity q Development process maturity 84© 2012 SQE Training V3.0
  • 81. Estimating Size • Algorithmic – Function points – Given requirements determine the number and complexity of q Inputs q Outputs q Inquiries q Logical files q External interfaces – Calculate and adjust based on “type of project/technology” 85© 2012 SQE Training V3.0
  • 82. Function Points Low Complexity Medium Complexity High Complexity Program Characteristic count multiplier count multiplier count multiplier total Inputs 6 3 2 4 3 6 44 Outputs 7 4 7 5 0 7 63 Inquiries 0 3 2 4 4 6 32 Logical Internal Files 5 7 2 10 3 15 100 External Interface Files 9 5 0 7 2 10 65 Unadjusted Function Point Total 304 Influence Multiplier 1.15 Adjusted Function Point Total 349.6 86© 2012 SQE Training V3.0
  • 83. Function Points ― Influence Factors 1 Data communications How many communication facilities are there to aid in the transfer or exchange of information with the application or system? 2 Distributed data processing How are distributed data and processing functions handled? 3 Performance Did the user require response time or throughput? 4 Heavily used configuration How heavily used is the current hardware platform where the application will be executed? 5 Transaction rate How frequently are transactions executed daily, weekly, monthly, etc.? 6 On-line data entry What percentage of the information is entered on-line? 7 End-user efficiency Was the application designed for end-user efficiency? 87© 2012 SQE Training V3.0
  • 84. Function Points ― Influence Factors 8 On-line update How many ILFs are updated by on-Llne transaction? 9 Complex processing Does the application have extensive logical or mathematical processing? 10 Reusability Was the application developed to meet one or many user’s (users’) needs? 11 Installation ease How difficult is conversion and installation? 12 Operational ease How effective and/or automated are start-up, back up, and recovery procedures? 13 Multiple sites Was the application specifically designed, developed, and supported to be installed at multiple sites for multiple organizations? 14 Facilitate change Was the application specifically designed, developed, and supported to facilitate change? 88© 2012 SQE Training V3.0
  • 85. Function Points ― Influence Factors Score Score Influence 0 Not present, or no influence 1 Incidental influence 2 Moderate influence 3 Average influence 4 Significant influence 5 Strong influence throughout 89© 2012 SQE Training V3.0
  • 86. Function Points Influence Factors Score INF = 0.65 + SCORE/100 90© 2012 SQE Training V3.0
  • 87. Function Points ― Input • Input – Number of screens, forms, dialogues, controls, or messages through which an end user or another program adds, deletes, or changes data 91© 2012 SQE Training V3.0
  • 88. Function Points ― Output • Output – Screens, reports, graphs, or messages generated for use by end users or other programs 92© 2012 SQE Training V3.0
  • 89. Function Points ― Inquiries • Inquiries – Direct access to data in database 93© 2012 SQE Training V3.0
  • 90. Function Points ― Logical Files • Logical Files – Major groups of end user data; could be a “file” or “database table” 94© 2012 SQE Training V3.0
  • 91. Function Points ― External Interfaces • External Interface Files – Files controlled by other applications with which the program interacts 95© 2012 SQE Training V3.0
  • 92. Function Points ― Advantages • Advantages include – Technology independent – Strong relationship to effort – Commonly used • Disadvantages include – Requires specialized training to compute 96© 2012 SQE Training V3.0
  • 93. Estimation Process 2 - Estimate Effort 97© 2012 SQE Training V3.0
  • 94. Effort Estimate • Jones’s First-Order Estimation Practice • Effort = (FP count) exponent 98© 2012 SQE Training V3.0
  • 95. Jones’s First Order Exponents Organization Capability Kind of Software Best Average Worst Systems 0.43 0.45 0.48 Business 0.41 0.43 0.46 Shrink Wrap 0.39 0.42 0.45 99© 2012 SQE Training V3.0
  • 96. Estimation Process 3 - Estimate Schedule 100© 2012 SQE Training V3.0
  • 97. Estimating Schedule • Rule of Thumb (McConnell 8-1) • Schedule = 3.0 * (effort) 1/3 101© 2012 SQE Training V3.0
  • 98. Estimating Schedule • Software Engineering Past Data – Example McConnell table 8-8, 8-9 q LOC vs. Schedule and Effort q Different project types 102© 2012 SQE Training V3.0
  • 99. Estimation Convergence Effort Range Schedule Range Project Phase Optimistic Pessimistic Optimistic Pessimistic Initial product definition 0.25 4.00 0.60 1.60 Approved product definition 0.50 2.00 0.80 1.25 Requirement specification 0.67 1.50 0.85 1.15 Product design specification 0.80 1.25 0.90 1.10 Detailed design specification 0.90 1.10 0.95 1.05 Product complete 1.00 1.00 1.00 1.00 103© 2012 SQE Training V3.0
  • 101. More Relationships Code Volume Approx. 100 LOC/FP, varies widely [Pressman, p. 94] Schedule (FP)0.4 Calendar months Staffing (FP)/100 Average – follows Raleigh curve Effort Schedule * Staffing = (FP1.4)/150 105© 2012 SQE Training V3.0
  • 102. Rules of Thumb 106© 2012 SQE Training V3.0
  • 103. Rules of Thumb (cont.) • Experience from several projects indicates that the amount of testing required in a project is proportional to the amount of development required • Ratio depends on – History – Organization, culture, structure – Projects – Teams 107© 2012 SQE Training V3.0
  • 104. Rules of Thumb (cont.) • Hadden’s Size/Complexity Technique Small Medium Large Simple Moderate Complex Complexity Size 108© 2012 SQE Training V3.0
  • 106. Sample Estimate Project A 110© 2012 SQE Training V3.0 Test Objective Size Complexity Effort 1 to 3 1 to 3 Sessions 1 2 3 to001 2 1 4 1 1 4 8 to002 2 3 16 2 4 8 16 to003 2 3 16 3 8 16 32 to004 2 1 4 to005 3 1 8 to006 2 2 8 to007 3 2 16 Typical Usage Scenarios 2 2 8 total 80 sessions 20 days Size Complexity
  • 107. Sample Estimate Project B 111© 2012 SQE Training V3.0 Test Objective Size Complexity Effort 1 to 3 1 to 3 Sessions 1 2 3 to001 1 1 1 1 1 4 8 to002 1 1 1 2 4 8 16 to003 2 1 4 3 8 16 32 to004 2 1 4 to005 2 1 4 to006 2 1 4 to007 1 1 1 to008 1 1 1 to009 3 2 16 to010 2 2 8 to011 2 2 8 to012 2 2 8 to013 2 2 8 to014 3 2 16 to015 3 2 16 to016 2 2 8 Typical Usage Scenarios 2 2 8 total 116 sessions 29 days Complexity Size
  • 108. Sample Estimate Project C 112© 2012 SQE Training V3.0 Test Objective Size Complexity Effort 1 to 3 1 to 3 Sessions 1 2 3 to001 3 2 16 1 1 4 8 to002 1 1 1 2 4 8 16 to003 1 1 1 3 8 16 32 to004 1 1 1 to005 1 1 1 to006 1 1 1 to007 1 2 4 to008 1 2 4 to009 1 1 1 to010 1 2 4 to011 1 2 4 to012 2 2 8 to013 2 2 8 to014 2 2 8 to015 1 1 1 to016 1 1 1 to017 2 1 2 to018 2 2 8 to019 3 2 16 Typical Usage Scenarios 2 2 8 total 98 sessions 24.5 days Complexity Size
  • 109. Sample Estimate Project D 113© 2012 SQE Training V3.0 Test Objective Size Complexity Effort 1 to 3 1 to 3 Sessions 1 2 3 t001 1 1 1 1 1 4 8 t002 1 1 1 2 4 8 16 t003 1 1 1 3 8 16 32 t004 2 1 4 t005 2 1 4 t006 2 2 8 t007 1 1 1 t008 1 1 1 t009 1 1 1 t010 2 1 4 t011 2 2 8 t012 1 1 1 t013 1 2 4 t014 1 2 4 t015 1 2 4 t016 1 1 1 t017 2 1 4 t018 1 1 1 t019 1 2 4 t020 2 1 4 t021 1 1 1 t022 1 1 1 t023 2 2 4 t024 1 1 1 t025 2 1 4 t026 1 1 1 t027 1 1 1 t028 2 1 4 t029 1 2 4 t030 1 2 4 t031 2 1 4 t032 1 1 1 Scenarios 2 2 8 total 99 sessions 24.75 days Complexity Size