This document summarizes key points from the book "When will it be done?" by Daniel Vacanti. It discusses how traditional software estimating techniques often fail and presents better approaches. These include using data from past cycle times to generate forecasts in the form of percentiles, tracking item ages, and applying Monte Carlo simulations to forecasts for multiple items. Keeping work in progress limited and flows smooth helps improve predictability. Focusing standups and retrospectives on queue times, scatterplots, and histograms can help teams reduce cycle times and make more reliable forecasts. The document emphasizes that reliable forecasts require a predictable process with minimal waste.
2. Why should you read this book?
What is the first question the boss/customer makes when we start a new feature/project?
This book shows how the most popular estimating and forecasting techniques used on software develop
fail, some better (and simple) ways to make such forecasts, and how to introduce practices in how the
team develops software so that forecasts become more reliable.
This book also shows that it’s better to let the data help produce forecasts instead of spending time
doing estimates with meetings etc
“Garbage in, garbage out”: if the process is not predictable, there’s no point making predictions.
3. What’s in it?
● Section I: Forecasting
● Section II: Forecasting for Single Items
● Section III: Forecasting for Multiple Items
● Section IV: How to know if you can trust your forecasts
● Section V: Putting it all together
● Section VI: Case Studies
5. “Note: the cone contains the
probable path of the storm
center but does not show the
size of the storm. Hazardous
conditions can occur outside of
the cone”
“The entire track of the center of
the tropical storm can be
expected to remain within the
cone roughly 60-70% of the
time.”
6.
7. The further out into the future
you get, the more uncertainty
you are likely to encounter
Uncertainty does not imply
Unpredictability
8.
9. The answer to WWIBD should be a forecast not
a date
A forecast is a calculation about the future that includes
both a range and a probability of that range occurring.
“Our data as of today shows that we have 85% chance of
finishing these 20 cards until March 25th”
11. How long does it take for you to go to work?
● How would we come up with an answer?
12. How long does it take for you to go to work?
Date Start End
13/01 08:17 08:43
14/01 08:35 09:03
15/01 08:22 08:44
16/01 07:44 08:58
17/01 09:12 09:37
14. How long does it take for you to go to work?
Date Start End Cycle Time (minutes)
13/01 08:17 08:43 26
14/01 08:35 09:03 28
15/01 08:22 08:44 22
16/01 07:44 08:58 74
17/01 09:12 09:37 25
15. Work item forecasting
“For the purposes of work item forecasting, if you track
nothing else, track these two things: the timestamp for
when work begins on an item and the timestamp for
when work finishes on an item.”
16.
17. 50th percentile: 8 days
95th percentile: 44 days
85th percentile: 22 days
18. Why scatterplots are cool?
● No advanced maths required: just count dots
● It is not influenced by outliers (unlike averages)
● Percentiles are forecasts!
From the scatterplot we saw, we know that we have 85% of chance to find a single work item in 43 days
or less.
19. A few things to consider
● Choose past data wisely: team size changes, project phase (S-curve), holidays..but do not
overcomplicate it either
● Remember to re-forecast often (many teams just do it at project start)
● Or even better, be explicit “this forecast is valid until DD/MM/YYYY”
20. How to improve forecasts for single items?
● How to get the percentile lines “closer together” i.e. more accurate?
● ..And why should we care about making more accurate forecasts?
21. How to improve forecasts for single items?
● How to get the percentile lines “closer together” i.e. more accurate?
● ..And why should we care about making more accurate forecasts?
the practices that will improve your predictions will also to improve
the health and efficiency of your process overall.
● Less stress
● Less “crunch time”
● Happier customers
22. How to improve forecasts for single items?
“The single most important thing that you can do to
improve single item forecasts is be proactive about how
long it takes for items to complete.”
25. Example: for an item above 50th percentile
● What have we found out about this item that might require us to take action on it?
● Do we need to swarm on it?
● Do we need to break it up?
● Do we need to escalate the removal of a blocker?
● When we first pulled the work item into our process it had a 15% chance of violating its forecast
(by definition).
● Now that the item has hit the 50th percentile, the chance of it violating its forecast has doubled
from 15% to 30%.
26. Why we don’t want an item to ever hit the 85th
percentile?
27. Why we don’t want an item to ever hit the 85th
percentile?
28. Why we don’t want an item to ever hit the 85th
percentile?
29. What to do?
● During daily meetings, have a look at the age of the work items, especially those going beyond the
50th percentile (and improved version of the “right to left” principle)
● Focus on flow efficiency (as opposed to ask people to “type faster”)
Flow efficiency = Active time/Total time
● Our current flow efficiency is around 49%
● If we increased it to 75% by focusing on queue time, our 85th percentile would go from 22 days to
around 14 days. Just by reducing wait time, not speeding up how we code
31. What histogram show us
● Cycle time distribution is not a normal distribution, it has a long tail
● Most statistical methods are for normal distributions, so they’re useless for answering WWIBD
● By improving flow, we reduce the “tail” and therefore improve predictability
● Therefore, comparing histograms over time are very useful to measure process improvements
33. How agile people usually do
● CEO: When will it be done?
● Agile dude: Easy! We have 50 cards to do, we’re doing 10 per week, so it will take 5 weeks!
● CEO: Awesome! I’ll schedule the demo to 5 weeks from now, for the whole company
● Agile dude: (f*ck.. I’ll tell the team that the CEO twisted my arm and came up with this date)
36. How to perform MCS
Monte Carlo methods vary, but in general, they tend to follow a typical pattern:
1. Define a probability distribution of possible inputs
2. Randomly select values from the input distribution and perform a computation on the selected
inputs
3. Repeat steps 1-2 an arbitrary number of times and aggregate the results (usually by employing a
Histogram)
4. Repeat steps 1-3 an arbitrary number of times until you have a clear picture of what the result set
looks like.
39. Section IV: How to know you can trust your
forecasts
● Limit WIP to ensure that the arrival rate is close to the finishing rate
● Ensure every item that is started is eventually finished (do not delete or put it back to “to-do”)
● Do not let items age unnecessarily
● Update the board daily to get more realistic data (garbage in, garbage out)
40. Lots of stuff about Little’s Law and CFDs
Too nerdy for a summary
45. How to improve predictability
● Standups for predictability: focus on queue time and item age
● Retrospectives for predictability: focus on scatterplot and histogram evolution. Are we reducing
our 85th percentile for cycle time? How is our average cycle time evolving?
46. Key takeaways
1. If you need to track one thing, track the date that each item starts and finishes
2. The answer to WWIBD is a forecast
3. Single items forecasts are done via cycle time
4. Multiple items forecasts are done via Monte Carlo Simulations
5. To be able to make good forecasts you need to run kanban well (WIP limit, ensure every item
enters and leaves, update the board often)
47. Want to read the book or see more?
Actionable Metrics for Predictability –
Daniel Vacanti at LKCE15
Check the book
Notes de l'éditeur
It looks pretty random? it ‘s because it is!
When a card starts, where will it end? We dont know!
50th percentile..it’s the average. That’s what we usually use for estimation. Would the custoemr be happy with 50% confidence?