2. Advertisement
https://stnektariostakingsteps5k.racesonline.com/
Register for St. Nektarios Taking Steps 5K on Saturday, Dec 8th 2018 @ 8:00 AM
Proceeds benefit Shelter Health Services, the region’s largest free medical clinic caring
for the 425+/night homeless women and children who stay at the Salvation Army
Center of Hope
3. The Agile Forecast: Outline
● Introduction
● Background
● Inspiration
● Innovation
● Nuts & Bolts
● Making it work
● Wrap up
5. About Joe
● Passion for making high quality software on
time that makes users AWESOME at what they
do
● Mechanical Engineering background
● Software Development for 25 years
● Agile for 17
○ Learned XP in 2001 from Joshua Kerievsky
○ CSM from Net Objectives in 2010
○ Brought scrumban to ANSYS shortly thereafter
● Inspiration
○ Poppendieck
○ Modern Agile
○ etc
®
6. Background
● New job!
● Product “promises” & a fixed date
● How are we going to make the date?
○ Cut scope!
○ Still overloaded
○ Make Small Stories!
○ We need to leverage our historical data!
8. Time to Delivery
● Set clear milestones and objectives
● Release often
● Measure cycle time to ensure people are moving their work across the board
● Increase throughput
11. Statistical Methods
“a slight study of statistical methods and
their application to social phenomena affords
one of the simplest examples of the
application of algebraic ideas.”
- Alfred North Whitehead, The Aims of Education
12. What about Little’s Law?
● Average Throughput
● Average Lead Time
● Average Work in Progress
Works great for:
● Assembly Line Production ....
● Little Variance ...
13. JIRA Control Chart
● Cycle Times Do NOT have a normal distribution
● Nor does WIP
● We are not stamping widgets!
YIKES!!!!
16. Research Findings
● Others confirm that software cycle times do not have a normal distribution
● They also found that a naive use of Little’s Law did not fare well in practice
● Troy Magennis and Alexei Zheglov arrived at similar conclusions
○ Use historical data
○ Fit a Probability Distribution Function to your data
○ Both used a Weibull distribution
17. What’s a Weibull distribution
● Weibull distribution is a continuous probability
density function which uses a constant shape
parameters to alter the shape of the distribution
● It is commonly used in reliability failure analysis
of mechanical parts to forecast failure
SO WHAT?
18. Weibull & Software
● Magennis found that as a team becomes more
lean the β becomes smaller
● Smaller stories, smaller cycle times
● The magic is finding β for your team
● Lots of curve fitting
● Still dissatisfied with the hoops to jump through
to estimate β and make an automated process
21. Data Science and the KDE
In need of a better fit, data scientists Mike Korvink and Katie Pratt suggested the use of
a KDE (Kernel Density Estimator) to estimate my distribution in a non parametric way
building it from my historical data
EUREKA!!!!
Perfect Fit
22. Hypothesis
Given KDE Probability Distributions of Cycle Time and WIP along with a release date,
I should be able to determine our chances of meeting said date
BUT HOW?
23. Monte Carlo Analysis
● Sample PDF over and over again and determine probability to meet date
● Additional uses of the data also reveal themselves
26. Pseudocode
Num Buckets = Num
Stories/WIP
Pull from JIRA
Calcs:
Cycle Time/ Story
WIP/Day
Compute KDE for
Cycle Time and WIP
based on lookback
period
Sample WIP KDE
For WIP in Sample
Sample cycle times
For each bucket
For each sample
Total Days += sample
For each sample
End Date = today + Total Days
If End Date <= Release Date
Match++
Probability =
Match / Total Samples
Build CDF of sampled
probabilities
27. Complete Automation
● Using this method with various lookback periods, I can have a completely
automated process that runs everyday without fail.
● Everyone knows where we are, everyday
28. Failsafe
● Use a Fibonacci sequence to
look-back in weeks for samples
○ [1,2,3,5,8,13,21]
● X-Axis: How far do our
samples look back in time
● Y-Axis: What is our probability
● Bubble Size: Number of
Samples
29. Samples and More Samples
● 7 Lookback Samples [1,2,3,5,8,13,21]
● 1000 WIP Samples
● 1000 Cycle Time Samples
● Up to 7 Million Samples
○ Samples are excluded if outside real range of parameters
since KDE’s are not bound to discrete values
33. Bonus!
● When pinpointing our next
release date, we can leverage
the CDF to guide us on picking
a date
● The X- Axis is number of days
out from today
● The Y-Axis is the probability
that we will finish our work by
that date
34. Heuristics
lookback_weeks:<How many wks back>
longest_cycle_allowed:<outliers>
Stories_remaining:<jql or value>
monte_carlo_samples:<# samples>
wip_exclude:<exclude dates>
in/exclude: <exclude people>
blocked_status: <blocked>
start_status: <start timer>
closed_status: <end timer>
status_for_cycle_time:
<working kanban column
holidays: <holidays>
zero_cycle_time:
<time to allot
for stories closed
without being opened>
35. Multiple Development Streams in a Single Team
● Using inclusion and exclusion of people and stories bucketed to those people,
multiple development streams can be tracked and monitored
Production Timeline
Feature 2
Feature 1
37. The Scale of Probability
Unlikely Likely
0% 20% 50% 80% 100%
impossible certain
very likely but not certainvery unlikely but not impossible
38. ● Teams must get into a habit of making very small stories
● Heuristics account for larger stories
● Daily probability tells us when to step on the gas
● Since starting this method in July 2016, the method has predicted release dates
with 95%
○ One of 20+ large releases slipped by 1 week due to Murphy’s law
How does it work in practice
40. Philosophy
● It’s a marathon from release to release
● The sprint is there as a mental time box
along with level setting
● Assess release scope change as it impacts
probability.
○ If you can absorb it, do it
26.2
41. ● Estimates become complexity & doubt estimates,
time is taken into account from results
● No longer compare teams on story points
● Keep your eye on adding value
● We have experimented with no estimation
Stop Point Madness
“I’m not sure if I was the inventor of story
points, but if I am, I’m sorry”
-Ron Jeffries
Complexity
Doubt
Effort
42. What about Points?
● Use Points for High Level, relative, estimation
● Use historical data to guide you along the way
● How does “new work” compare to “old work”?
● Get a feel for what fits
● Use this method to determine date
● Don’t use points for day to day
○ Track throughput
Completed
Stories
Completed
Points
43. What about new teams?
● Start with Weibull, fix your distribution
● After accumulating data start this method
● You can start with less than 100 closed stories but 100 closed stories will get you
into the 90% accuracy range
44. Key Behaviors
● Regardless of story size, team must move stories across the board accurately
● Trust your team, if the numbers don’t sit well talk to the team and investigate
45. Enhancements
● Dependent stories and their placement into the WIP buckets
● Better point heuristics
● Performance improvements
○ Backed by a database to cook less WIP and cycle time numbers
47. Wrap Up
● Ability to leverage wealth of information in JIRA to predict future release dates
and probability of meeting that release date using KDE and Monte Carlo analysis
● Team knows where they are everyday
● Stakeholders are aware of teams confidence in meeting release
● Trust is built between stakeholders and team for consistently meeting
commitments with high quality on time, every time