The document discusses measuring software development team performance through metrics like efficiency, effectiveness, employee sentiment, and relative performance compared to other teams. It suggests collecting a variety of data points over multiple stages to get a more holistic view of how fast a team is going, including efficiency metrics like lead time and throughput, effectiveness metrics like customer retention and value delivered, employee sentiment surveys, and finding ways to compare performance to other teams either internally through rotations or externally through collaborations. The goal is to understand if a team is truly fast, above average, or awesome and identify opportunities to accelerate their speed further.

1. 1
Do you know how fast
you are going?

2. 2
At
We are starting to try to collect
the data to find out, and to get
relativity we need others to join
us!
Douglas Talbot
Douglas.Talbot@Ocado.com
@douglastalbot
Head of Organisational Effectiveness
Thought provoker and Catalyst

3. 3
Ocado icons

4. 4
Stage 1 - Evolution
Admit the problem!

5. 5

6. 6
Can we estimate?

7. 7
The goal estimate is
‘‘the most optimistic prediction that has a
non-zero probability of coming true’’
- Tom DeMarco 1982

8. 8
Focal Bias
Anchoring and Adjustment problems
Motivation
Perceived power
Social cues
Temporal perspective
Unpacking
Deliberation
etc etc etc
The team scaling fallacy
The Planning Fallacy

9. 9
Dunning – Kruger effect
Unrealistic optimism
Over estimating desirable effects
Over confidence in judgement
Information deficits
Ill defined nature of competence
Information neglect
Focus on the self
Influenced by external cues
Self Perception Fallacy

10. 10
Flawed self assessment
StanfordEdu - Biased
Self Views.pdf

11. 11
80% of developers think
they are above average
XKCD 715

12. 12
80% of software
teams think they are
above average

13. 13
Stage 1
Shall we admit the problem?

14. 14
Stage 2 - Evolution
Start to measure

15. 15
20

16. 16

17. 17

18. 18

19. 19

20. 20

21. 21

22. 22

23. 23

24. 24
So lets imagine we have all of these

25. 25
Do we know how fast we are
going?

26. 26
We can probably all
make a quick
judgement about
having more data ...

27. 27

28. 28

29. 29
Stage 2 - Efficiency
measurement

30. 30
We can compare ourselves to our last
attempt
- How much did we do?
- Has our quality got better?
- How fast can we deliver change?

31. 31
We have an Efficiency measure
Do you think this team is going fast?:
Lead time: 103 days
Cycle time: 81 days
Efficiency: 49.5 / 81 = 61%
Throughput: 6 Small features
3 Medium features
1 Large feature / 81 days

32. 32
We can probably even answer
more esoteric questions -
How predictable is that?
Can we better use our people?

33. 33
Efficiency measurement
There have been studies considering this in far more detail
looking at best methods for:
TCO - Total Cost of Ownership, CoQ - Cost of quality, Speed1
Factors of success, Factors of value, Project failure rates2
1.InfoQ - Evaluating agile software methodologies
2.http://www.standishgroup.com/Reports2015

34. 34
But are we missing some
important knowledge?

35. 35
So what do we need?
Your turn

36. 36
How many of you measure
value?
Standish 1995 - 80% of
projects cost more than the
value they delivered!

37. 37
Stage 3 - Evolution
Measure more meaningfully

38. 38
Delighted customers

39. 39
Effectiveness
- Customer retention
- Customer acquisition
- Spend / shop
- Social networks
- etc etc etc
- NOT vanity metrics

40. 40
Make money or
Deliver mission

41. 41
Effectiveness
- Share price
- Profit
- Crime lowered
- People saved
- Fewer accidents
- More biodiversity

42. 42
Happy Organisation / Boss

43. 43
Effectiveness
- Was it worth it (ROI)?
- Creativity
- Innovation
- On time
- On budget
- Impressive brand win
- Lower costs to recruit

44. 44
What should we be measuring about
ourselves as a team to be the best we
can be for our businesses and create
trust/belief in our integrity and skills?

45. 45
Stage 3 - Effectiveness
measurement

46. 46
Effectiveness
Do you think this team is going fast?:
Value: + £231,000 sales volume,
+2% customer retention
+1% customer acquisition on
previous rate in the last 30 days

47. 47
Bibliography
 David Anderson
 Don Reinertsen
 Eric Ries
 Mary Poppendieck

48. 48
What other measures do we
need to know how fast we
are going?
Your turn again

49. 49
Stage 4 - People
measurement

50. 50
Engaged team = faster team?

51. 51
Happy organisation?
Ocado Technology Adjusted data for example

52. 52
Fast
team?
Sentiment No. of answers
Overall 0.588 100
Department 1 0.928 2
Department 2 0.785 16
Department 3 0.727 20
Department 4 0.680 10
Department 5 0.591 1
Department 6 0.544 17
Department 7 0.538 15
Department 8 0.516 5
Department 9 0.246 3
Department
10
0.234 1
Department
11
0.124 10
Ocado Technology Adjusted data for example

53. 53
Can we know
what this does
to Efficiency
and
Effectiveness?
Sentiment No. of answers
Overall 0.588 100
Department 1 0.928 2
Department 2 0.785 16
Department 3 0.727 20
Department 4 0.680 10
Department 5 0.591 1
Department 6 0.544 17
Department 7 0.538 15
Department 8 0.516 5
Department 9 0.246 3
Department
10
0.234 1
Department
11
0.124 10
Ocado Technology Adjusted data for example

54. 54
Adam Tomhill – Code as a crime scene

55. 55
But are we missing some
important knowledge?
Do we really know how
fast we are going yet?

56. 56
20
/but What’s the difference?
400

57. 57
20

58. 58
How would you really know you’re
FAST?
or better than average?
or Awesome?

59. 59
Let’s try
Efficiency - 61% Value add, Throughput
– 3 Medium features / 81 days
Value - £231,000 and 2% new
customers in the last 30 days
We have a sentiment score of 0.8

60. 60
Is that fast, > average, or awesome?
How much faster can we get?
How fast learning are we?
Some questions

61. 61
“If you’re not moving at the speed of
the market you’re already dead …
you just haven’t stopped breathing
yet”
- Jack Welch, CEO GE
Are our devs / teams at
market speed?

62. 62
Relative Productivity?
+
Relative Lead time?
+
Relative Speed to pivot capacity?

63. 63
At
We are starting to try to collect the
data to find out. But, to get relativity
we need others to join us!

64. 64
Internal Relativity
Team rotation
Peer feedback
Health checks
Internal code quality
Failure demand ratio

65. 65
University scholars compete
Even lawyers have potential
comparative measures
External Relativity

66. 66
Why would I hire you as a
software developer or
your software development team?
How fast do you go?

67. 67

68. 68
The how discussion!
Coding competition?1
Trust for Productivity study?2
Team rotation?
Common assessment model? But for teams?
Hopefully brainstorm time
1.Peopleware, Timothy Lister & Tom de Marco
2.Teams in the test tube

69. 69
At
We are starting to try to collect
the data to find out, and to get
relativity we need others to join
us!
Please come and chat!
Make contact if you are interested in this
challenge.
Douglas Talbot
Douglas.Talbot@Ocado.com
@douglastalbot
I’m hiring Catalysts / Coaches!