In today’s business climate, change occurs at an ever-increasing pace. Managers and executives are increasingly challenged to build an organization that is able to respond effectively to this change. This is the essence of agility. This talk will provide the latest thinking on building the agile organization, moving beyond the command and control paradigm to one that balances employee empowerment and business alignment.

1. Choosing
Analytics
Murray
Cantor
PhD.
Cutter
Senior
Consultant
mcantor@cutter.com
©
2015,
Murray
Cantor

2. Things
I
have
heard
from
over
the
years
• “I
have
no
idea.”
– Developers,
when
asked
about
how
long
will
it
take?
• “Measures
are
a
waste,
they
are
costly,
oppressive,
and
interfere
with
the
real
work”
– Some Methodologists
• “Trust
the
(my)
process.
If
the
process
is
not
working
for
you,
you
are
doing
it
wrong.”
– Some
(of
the
same)
Methodologists

3. Each
of
these
have
generated
lots
of
heated
disagreements
3

4. Metrics
are
essential
for
sense
and
respond
loops
to
achieve
goals
When
choosing
measures
consider
whether
– The
measures
let
you
know
how
whether
you
are
achieving
the
goals?
– You
have
a
way
to
respond
to
the
measures?
4
Avoid
building
dashboards
just
to
use
the
data
you
have

5. The
two
key
considerations
to
picking
your
measures:
5
nMixtures
of
work
efforts
nLevel
of
the
organization Work item, artifact
completion
Staff
member Commits
to
Project, product deliveryProject
manager,
team
lead Commits
to
Efficiency, value deliverySenior
manager Commits
to
Profit, return on
investment
Line
of
business
executive Commits
to

6. The
two
key
considerations
to
choosing
your
measures:
6
nMixtures
of
work
efforts
nLevel
of
the
organization Work item, artifact
completion
Staff
member Commits
to
Project, product deliveryProject
manager,
team
lead Commits
to
Efficiency, value deliverySenior
manager Commits
to
Profit, return on
investment
Line
of
business
executive Commits
to

7. Meeting
goals
requires
analytics
7
Work
item,
artifact
completion
Staff
member Commits
to
Project,
product
delivery
Project
manager,
team
lead
Commits
to
Efficiency,
value
delivery
Senior
manager Commits
to
Profit,
return
on
investment,
mission
fulfillment
Line
of
business
executive Commits
to
Before

8. Aligning
goals
• For
each
level
to
meet
its
goal,
the
leader
is
dependent
on
the
lower
level.
• So,
the
leader
seeks
commitments
from
that
layer.
Meeting
those
commitments
becomes
the
goal
of
the
next
layer.
• Hence
the
analytics
serve
to
integrate
the
organization
8
Work item, artifact
completion
Staff member Commits to
Project, product delivery
Project manager, team
lead
Commits to
Efficiency, value deliverySenior manager Commits to
Profit, return on investment,
mission fulfillment
Line of business executive Commits to
Work item, artifact
completion
Staff member Commits to
Project, product delivery
Project manager, team
lead
Commits to
Efficiency, value deliverySenior manager Commits to
Profit, return on investment,
mission fulfillment
Line of business executive Commits to
Commitments
Analytics

9. The
two
key
considerations
to
picking
your
measures:
9
nMixtures
of
work
efforts
nLevel
of
the
organization Work item, artifact
completion
Staff
member Commits
to
Project, product deliveryProject
manager,
team
lead Commits
to
Efficiency, value deliverySenior
manager Commits
to
Profit, return on
investment
Line
of
business
executive Commits
to

10. Kinds
of
Development
Efforts:
What
is
your
mix?
10
1.Low
innovation/high
certainty
–Detailed
understanding
of
the
requirements
–Well
understood
code
2.Some
innovation/
some
uncertainty
–Architecture/Design
in
place
–Some
discovery
required
to
have
confidence
in
requirements
–Some
refactoring/evolution
of
design
might
be
required
3.High
innovation/Low
Uncertainty
– Requirements
not
fully
understood,
some
experimentation
might
be
required
– May
be
alternatives
in
choice
of
technology
– No
initial
design/architecture

11. 1. Low
innovation
-­‐ high
certainty:
Statistics
of
– Cycle,
lead
times
– Backlogs
size,
growth
– Time
in
process
– Utilization
– Non-­‐value
added
effort
11
2. Some
innovation
-­‐
some
uncertainty
– Time,
cost
to
delivery
– Velocity
– Burn
down
– Cumulative
Flow
Diagrams
3. High
innovation:
Low
certainty
– Time
to
pivot
– Value
of
learning
– Business
canvas
– Time,
cost
to
delivery
Apply
measures
in
accord
with
mix
of
work
Descriptive
Predictive/Bayesian

12. Example:
Fitting
analytics
and
practices
to
routine
efforts
• For
low
innovation
efforts
(continuous
delivery,
not
“real”
projects),
pick
product
flow
practices
and
analytics
– Uncertainty
is
low:
you
have
already
carried
out
similar
efforts
many
times
– The
only
thing
that
matters
is
how
quickly
or
efficiently
you
can
carry
out
the
project
• Suitable
for
lean/VSM
measures
• Tradeoff
between
speed/efficiency(utilization)
• The
principles
described
by
Don
Reinertsen in
his
book
Flow apply
in
this
bucket
12

13. The
two
challenges
in
meeting
Bucket
1
goals:
13
1. The
work
requests
flow
is
unsteady
2. Each
work
request
is
different
• The
assumptions
around
6-­‐sigma’s
controllable
processes
are
not
met’
• In
practice,
meeting
bucket
1
goals
takes
constant
feedback
and
response

14. A
work
flow
model
for
routine
efforts:
Focus
on
the
state
transitions
of
the
work
products
14

15. Measure
the
work,
not
the
workers
• Focus
on
describing
how
business
data
is
changed/updated,
by
a
particular
action
or
task,
throughout
the
process.
• Specifically,
in
the
routine
effort
bucket,
flow
measures
to
state
transitions
of
work
product:
– Two
state
types:
• In
process
(undergoing
state
transitions)
• In
backlog
(awaiting
state
transition)
15
In
backlog
In
process

16. Descriptive
example:
Cycle
times
16

17. To
Visualize
the
data,
use
a
histogram
17
80%
point
is
about
105
days

18. Insights
and
Actions
• Insights
– Both
teams
performing
comparably:
Not
obvious
skills
issue
– Backlogs
too
large
– The
teams
seem
to
be
focusing
on
the
easier,
not
the
most
critical
• Actions
– With
team
investigate
reason
for
backlog
size
– Discovered
the
governance
process
(decision
to
update
statuses)
is
overly
cumbersome
leaving
staff
free
to
work
elsewhere
– In
response,
the
governance
process
was:
• Streamlined
(an
approval
eliminated)
• Automated
(less
time
spent
finding
e-­‐mails)
– Work
with
teams
to
set
and
track
cycle
time
80%
goal
by
priority
18

19. This
is
what
improvement
looks
like
19

20. Example
2:
Fitting
analytics
and
practices
to
high
innovation
projects
• For
high
innovation
projects
pick
probabilistic
methods
and
the
corresponding
set
of
practices:
– You
really
do
not
know
what
the
solution
would
look
like
– you
must
experiment
in
order
to
find
it
• Not
knowing
what
the
solution
would
look
like,
your
intuition
is
a
poor
guide
for
estimating
and
scheduling
under
systemic
uncertainty:
– You
must
experiment
in
an
affordable
manner
20

21. Bayes
is
the
way
for
development
teams
and
management
to
deal
with
uncertainties
• In
bucket
2
and
3
development,
quantities
such
as
time,
cost
to
complete,
and
velocity
are
not
known
for
certain.
– There
is
not
enough
known
to
make
exact
predictions
– You
need
to
utilize
the
actual
data
you
produce
sprint
by
sprint
• Bayesian
analysis
is
the
centuries
old
method
for
rigorously
dealing
with
with
uncertain
quantities.
• Bayesian
analytics
allows
everyone
on
the
team
to
learn
together.
21
• Attributes
of
Bayes:
• Uncertain
quantities
are
specified
probabilities
• The
probabilities
capture
both
the
best/worst
estimates
and
the
level
of
uncertainty
• The
probabilities/beliefs
are
updated
as
information,
evidence
comes
in.
• The
probability
distributions
can
be
“added,”
“multiplied,”
etc.

22. Estimating
effort
remaining
22
+
…
+ =
l e h
No
probability
less
than
No
probability
greater
than
Most
probable
value
For
remaining
stories
in
each
epic:
• Estimate
size
with
triangular
distributions
• Sum
using
forward
propagation
(aka
Monte
Carlo)

23. Ongoing
Estimates
of
Weeks
Late

24. Bayesian
Example:
Four
Project
Pattern
24
Summary'Statistics
Mean 11.5377134
Median 2.00294414
Variance 3412.51999
Standard'Deviation58.4167783
Lower'Percentile'[25.0]E1.3278719
Upper'Percentile'[75.0]7.37082892

25. Achieving
goals
requires
sense
and
respond
loops
• Key
principles
– Kelvin’s
Principle:
“To
measure
is
to
know.
If
you
can
not
measure
it,
you
can
not
improve
it”
• Measures
are
part
of
feedback
loops
– The
converse
principle:
“Don’t
bother
to
measure
what
you
do
not
intend
to
improve”
• Find
a
small
set
of
measures,
not
a
long
laundry
list
– Einstein’s
Principle:
“The
best
solution
is
as
simple
as
possible,
but
not
simpler.”
• Pick
the
right,
not
overly
simple,
statistic
25
(re)Set
Goal
Take
action
(practices)
Measure
progress
(analytics)
React

26. Choosing
metrics
big
picture
Agree
on
goals
-­‐ Depends
on
the
levels
and
mixture
of
work
Agree
on
the
how
they
fit
into
the
loop
1.
“How
would
we
know
we
are
achieving
the
goal”
2.”
What
response
we
take?”
Determine
the
measures
needed
to
answer
the
questions
-­‐ Apply
the
Einstein
test
(as
simple
as
possible,
but
no
simpler)
Specify
the
data
needed
to
answer
the
questions
Automate
collection
and
staging
of
the
data
26

27. To
summarize
• There
is
no
one-­‐size
fits
all
choice
of
measures
• Measures
must
be
part
of
some
feedback,
sense
and
respond
loop
• Choice
of
measures
Depends
chiefly
on
– Mixture
of
work
– Level
of
organization
27

28. Murray Cantor
Email: mcantor@cutter.com
www.murraycantor.com
Contact
Me
.

29. Acerca
de
Cutter
Consortium
• Cutter
Consortium
es
una
firma
única
en
su
tipo,
integrada
a
partir
de
una
red
de
colaboración
de
más
de
150
expertos
practicantes,
mundialmente
reconocidos
en
el
ámbito
de
las
Tecnologías
de
Información,
comprometidos
en
la
generación
de
consejos
críticos,
objetivos
y
de
alto
nivel.
• Nuestra
misión
es,
a
través
de
servicios
de
consultoría,
educación
ejecutiva
y
de
acceso
a
nuestra
base
de
conocimiento,
ayudar
a
las
organizaciones
en
el
logro
del
éxito
empresarial,
la
innovación
y
la
generación
de
ventajas
competitivas
a
partir
del
uso
de
las
Tecnologías
de
la
Información.
• Nuestra
propuesta
de
valor
consiste
en
proporcionar
a
nuestros
clientes
Acceso
a
los
Expertos,
los
más
destacados
dentro
de
su
área
de
especialidad
y
que
han
estado
en
campo,
al
frente
de
organizaciones
y/o
proyectos
de
TI.
Su
consejo
deriva
de
la
experiencia
acumulada
durante
décadas
y
de
las
lecciones
aprendidas
al
haber
enfrentado
algunos
de
los
problemas
más
críticos
para
las
TI.
• Cutter
promueve
la
reflexión
sobre
las
TI
alentando
el
debate
y
la
colaboración
entre
líderes
de
diferentes
dominios,
países
y
disciplinas;
los
pensadores
más
destacados
del
binomio
TI-­‐Negocios.

30. Cutter
Consortium
América
Latina
Retorno
30
No.
2
Col.
Avante
Coyoacán,
D.F.
C.P.
04460
Tel.
55-­‐5336-­‐0418
contacto@cutter.com.mx
www.cutter.com.mx
@cuttermexico
cuttermexico