This is a presentation and workshop given to the Silicon Valley Engineering Leadership Group, held in Palo Alto, CA. First it shows how metrics related to compensation can really drive bad behavior. Then the presentation turns to product development metrics that can be used in the context of program management consulting, to improve effectiveness. More is available at https://tcgen.com/product-development-metrics, where there are papers on metrics as well as downloadable tools for improving the product development process. Metrics svelg aug16_26

1. 1
1
SILICON VALLEY SOFTWARE LEADERSHIP COMMUNITY
METRICS:
MORE HARM THAN GOOD?
John Carter
August 2018

2. “In order to get results, you
need to measure them”

3. WRONG!

5. $100B Cost of Bad Metrics driven by Bad Management

6. RED, YELLOW, GREEN – REALLY?

8. 8
88

9. 9
99

10. 10
1010

12. “You miss 100 percent
of the shots you never take.”

15. 15
1515
SUCCESS WITH METRICS: APPLE
• Pinpoint a small number of improvement levers
• Organizational changes teased apart and divided into
incremental improvements, sequenced and prioritized
• For each lever, a target metric to seed changes within the
organization
• Only move on to the next transformation when metrics indicate
that an improvement has taken hold

16. 16
1616
METRICS CHECKLIST
Criteria Good Metric
Traceable to results (not measure results)? 
Hard to game (objectively measured)? 
Target curve that varies with time? 
Not related to compensation? 
Culturally aligned? 
Easy to measure? 
Rapidly change? 

17. 17
1717
PREDICTIVE METRICS TREE
Goal
Key Drivers
Initiatives
Predictive Metrics

18. 18
1818
HALF LIFE TABLE
• Complexity drives the time to achieve an objective
• THIS time is PREDICTABLE and should be used to GUIDE PROGRESS
• Based on empirical study of more that 50 improvement programs
• The rate of improvement is related to organizational complexity
• In the absence of a definitive plan this heuristic is much better than a traffic light

19. 19
1919
HOW TO USE METRICS TO MANAGE CHANGE
• Reported to Steering Committee bi-weekly
• Action items were generated based on metrics
• Once goal maintained over time
• Up the ante / raise the bar
• Find a replacement metric to replace
Time (Months)
0 %
20 %
40 %
60 %
80 %
100 %
12/15 1/15 2/15 3/18 4/18 5/2
Percent
Using
Target
Actual
gap
Measure existence Measure coverage Measure quality
• Only one metric at a time
• Inch-wide, Mile-deep

20. 20
2020
WORKSHOP
With the scenario you provide, construct an improvement
approach; suggest a predictive metric and a target curve.
Discuss with your table – PICK A TABLE LEADER!
• Decide on an improvement goal
• List initiatives: pick one that eliminates root causes
• Create a candidate PREDICTIVE metric
• Describe your target curve: dependent on complexity

21. 21
2121
DISCUSSION
• Sharing workshop results?
• Questions, comments?

22. “We are making better
decisions, faster”

23. 23
2323

24. 24
24
TCGen Inc.
Menlo Park
CA, 94025
jcarter@tcgen.com
(650) 733-5310

25. WORKSHOP
With the scenario the table agrees to, construct an initiative that reverses root cause, and suggest a predictive
metric and describe a target curve, inspired by half-life; finally check you work against the checklist
Discuss with your table
• Define your improvement goal: Describe an improvement objective
•
• Discuss initiatives that reverse root causes and pick one
• Create a candidate PREDICTIVE metric: That measures progress
•
• Describe target curve: Dependent on complexity
• Check your work by reviewing CHECKLIST – How did you do?
Write answer here
Write answer here
Write answer here
FIRST - PICK A LEADER FOR YOUR TABLE!
jcarter@tcgen.com Silicon Valley Engineering Leadership Community

26. METRICS CHECKLIST & HALF LIFE TABLE (TIME TO 50% IMPROVEMENT)
Criteria Check if Yes
Traceable to results (but not measure results)? o
Hard to game (objectively measured)? o
Target curve that varies with time? o
Not related to compensation? o
Culturally aligned? o
Easy to measure? o
Rapidly change? o
Typically Half-Lives are 1 – 6
Months for most
transformations that don’t
involve partners or large new IT
systems. Make your best guess,
unless you have a program plan
that defines it.
Good metrics satisfy most to all
of these criteria. The most
important are: Traceable to
results, Having clear targets that
vary over time, avoid
compensation, supported by
culture, and rapidly change.
jcarter@tcgen.com Silicon Valley Engineering Leadership Community