The document contains metrics collected from three iterations to measure project performance. It includes metrics on backlogs, defects injected, velocity, cost of quality, cost of non-quality, and percentage of features passing tests (RTF). Data is provided for three products - Audi A6, Falcon, and Mustang across the three iterations. Formulas, collection frequency and reporting frequency are also defined for each metric.
1. Collecting Reporting
S No Metric Business Objective Metric Category Measures to be Collected Formula UOM
Frequency Frequency
To Measure How Captured as Number Of
After Completion
Backlogs / Long Work Comments From Demo - Backlogs Demo Comments From
1 Delivery Timeliness. Number Ongoing of the Iteration
Iteration Items/Features (I.e [Enhancements will be Marked Uniquely] Product Owner
Feature Completeness
Product Debts) are
waiting in the Queue
Defects Number Of Defects Raised After Completion
Number of Defects Raised By the Product
2 Injected / Improve Quality of Iteration Quality
Test Team against the Iteration Build.
in Bugzilla against the Number Ongoing of the Iteration
Iteration Deliverables. Team Iteration Build.
Improve Productivity, After Completion
Capacity (Team Size)
3 Velocity Improve Cost Productivity
Delivered Quantity (User Stories/ Taks)
(Delivered/Capacity)*100 of the Iteration
Effectiveness
% On Going
It contains three components viz
Prevention cost – Training
To meet Delivery, (Product/Technology/Process). Captured as Effort After Completion
Cost Of
4 Quality and Cost of ROI Appraisal Cost * - Testing & Review. Expressed as % of Total % Ongoing of the Iteration
Quality the Project for ROI Failure cost – Rework Effort
2. Improve Cost
Effectiveness by
continuously
reducing the
Overheads/Obstacles
Examples:
1. Business Value
Captured as Effort After Completion
Cost Of Added or Reduce Effort Spent in Handling/Addressing/Solving
5 Expressed as % of Total % Ongoing of the Iteration
NonQuality Nonnegotiable Waste Overheads/Impediments the Overheads and Obstacles
Effort
like Setting Up
System etc and
2. Minimizing Non-
Value Added or pure
waste like waiting
long for
Clarrifications etc
The RTF metric
RTF - shows, at every
moment in the Readiness For a Number Of Features
Running Number of Features/User Stories Passing
6 project, how many Potentially Shippable Passing Tests / Total % Ongoing Daily
Tested the Acceptance Tests For the Iteration
features are passing Product Number Of Features *100
Features all their acceptance
tests.
3. Mode of
Notes Tools
Consolidation
1. Rationale is to measure Commited Vs Delivered
User Story Completeness.
By Iteration /
2. New Enhancements (Demo Commenets) even Excel/Wiki
Phase
though Captured as part of the Iteration Backlogs
shall be Marked Uniquely.
Using Bugzilla - By
Defects that are Injected During the Iteration
By Iteration / Asking Tester to
Development is Key & Defects that are not Related
Phase Use the Keyword
Should be Captured separately at Product level.
against the Build
As On Date
Change in Template
&
Of Burndown Chart
Consolidated By
Capturing Velocity
Iteration / Phase
Consolidated By
Clarity
Iteration / Phase
4. Aggregate of time, money, and opportunity cost
Consolidated By losses resulting from doing things wrong. CONQ is
Excel
Iteration / Phase estimated by time taken to fix a
Defect/Overheads/Obstacles by Total Effort.
The desired software is broken down into named
features which are part of what it means to deliver
As On Date the desired system.
& For each named feature, there are one or more
Excel
Consolidated By automated acceptance tests which, when they
Iteration / Phase work, will show that the feature in question is
implemented.
5. S No Metric Data Notes
Acceptence Criteria Met.
1 Backlogs / Iteration 1
Change Request
2 Defects Injected / Iteration NA Product Testing has not Started
3 Velocity- By Story Points 44 Normalized Story Points
Under Control
4 Cost Of Quality 24%
[Acceptable Range: 20% to 28%]
Closely Monitor
5 Cost Of NonQuality 13%
[Acceptable Range: < 8%]
6 RTF - Running Tested Features 100% JUnit & Manual Integration Tests