Effective Product Ownership is key to enabling an effective agile delivery. What goals should we set? How would we measure them?

1. Metrics for the LEAN/Agile
Product Manager
Yuval Yeret

2. Main things we want to pay attention
to
• Performance of the Production floor
– covered elsewhere (Simple KPIs
Slides)
• Performance of the Product
Management group:
– Business Value
– Wastes related to PM
– Technical Debt

3. Business value
• We care about outcome – features delivered,
adopted, used, paid for
• How can we measure this?
• Manage a kanban at the high level features level,
that tracks when features are adopted, and upon
first paying customer.
• Then see how much WIP of features not yet
adopted we have, LEAd/Cycle time to adoption,
features that we dropped on the way.

4. Debt
• A lot of time debt is taken due to PM
decision
• We want to track how much debt we
have, and take action to minimize it.
• E.g. we need to release ASN1 now, so we don’t
“automate tests”/”code it correctly”, so every
work on ASN1 is slower, until this is fixed

5. Tracking debt in kanban
• Have debt card type that is created
when debt is taken on
• Track amount of debt versus overall
WIP/Backlog
• See whether stable, improving,
worsening trend
• Decide on policy for dealing with debt –
WIP Limit, etc.
• Track the cycle time and WIP for debt
cards to see whether they get the SLA
they deserve

6. Wastes related to PM
• Waiting for PM
• PM related Churn / Context switching
/ Expediting
• Sunk Costs
• Rework due to late feedback by PM

7. Waiting for PM
• Look at the CFD, observe the size of the
PM-related queues over time.
– Especially Pending PM Review which is in the
middle of DEV/Test
– And Ready-MMFs as well as DEV Ready in
some cases which depend on PM approval
– Advanced – in the cycle time performance
report, focus on PM areas
• When looking at exceptions to Cycle
time, participate in the root cause
analysis, and see if interaction with PM
was part of the long cycle time.

8. PM related Churn / Context switching /
Expediting
• Add Expedited class of
service
– Can be used by PM to
override priorities in
DEV WIP – just to top of
queue, don’t override
current WIP
• Add emergency class of
service
– Can also override
current WIP
• Assumption –
– This is value trumps
flow. We give up
efficiency when we use

9. Measuring the effect of value over
flow COS
• Look at cycle times for different
kinds of classes of service
• Look at distribution of different COS
in the WIP

10. Look at amount of changes in scope
• Replace – need visualization that
shows scope changes in content
• Add – can simply look at total scope
for a “Release” and observe whether
its growing

11. Case Study – Typical release behavior
250
Added Growth in
Scope Feature
Cost / Dark
200
Matter
Total Required Work
150
Actual Done
Planned Scope
Planned Done
Original + Dark Matter
100
Linear (Total Required Work)
Linear (Original + Dark Matter)
50
0
1 2 3 4 5 6 7 8 9 10

12. • Dark Matter – Is where we thought a
feature costs X
• But then, during
breakdown, analysis, creating iteration
stories, we understand it actually costs
X+D
• Then, PM decides whether to scope to fit
down to X again, or D is worth it.
• Worthwhile tracking our behaviour on
this, and learning from it.
• What is the right D number/percent?
Good question!
• Can be observed in the CFD for a release.

13. Sunk Costs
• Add a LANE that collects
features/stories that are “ON HOLD” –
the Recycle Bin in the archive area
• The amount of work done on them is
the sunk cost
• Amount of work hard to measure, so
use alternative:
– CYCLE TIME – look at cycle time for end
lane being the recycle bin

14. Rework due to late feedback by PM
• Will appear as high cycle times
• Will appear as moving back cards on
the board (need to find way to
measure)
• Can use special Class of service / card
type to identify these kinds of stories
better for measurement/tracking
purposes

15. Workload compared to DEV
• See how much workload is in PM compared
to DEV
• Look for trends and major changes that
can indicate:
– Bottleneck in PM
– Idle and slack capacity – expect to see PM seeing
clients/customers at those times