A combination of Evidence Based Management, Scrum.org PAL-E, and new executive leadership content on what to measure, whether to measure at all

1. How to measure success?
John Coleman, PST, LSFT, Tameflow Kanban
Marshall Goldsmith #LEAD60
@JohnColemanIRL

2. What is organizational
agility?

5. What is success?
To measure, it’s more useful to know why we’re changing and what
good looks like

6. Let’s imagine a scenario
For what to measure, whether to measure

9. Let’s consider the quality of
measures
activities, outputs, outcomes, impact
leading indicators, lagging indicators

15. Unintended
consequences
of comparing
teams?
Gaming Measurements lose
their value

16. What matters
most
Getting better Getting worse

17. Are we
missing a
trick?
Mind the Gap
Over-playing

18. Scrum.org’s Evidence Based Management

19. Quantitative
Leading
Number of PBIs
85th percentile Cycle Time
Practices ???
Potential value delivered
Lagging
xxxxxxx
Value confirmed via yyyyyyyyyy
Employee engagement
Customer satisfaction

20. Qualitative
• Interview teams and capture trends in performance & feel
• Interview teams and capture how leaders are showing up
• Star rate the journey, and if less than 5 give specific comments to help
improve without proving any hints of who is making the comments

21. But for many teams outside my sphere of influence the needles didn’t move

22. Team
maturity
Just consider for all teams to
avoid ranking
Scrum Master Product
Owner
Development
Team
LeaderMaturity
Level
1
2
3
4
5
How mature is your Scrum Team?
Content on this poster is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International-License
Leads & stimulates everyone in the organi-
zationvalue stream to success
Enables the organization to continuously
have impact on customers
Helps the organization to understand &
apply Agile principles
Focus on continuous improvement in the
organization
Teaches others how to combine Scrum
with additional practices
Leads & stimulates his entire team to suc-
cess
Enables the Scrum team to continuously
improve and deliver value
Helps stakeholders to understand & apply
Agile principles
Focus on continuous improvement in the
value chain
Extended Scrum experience with Scrum
and additional practices
Stimulates team members in taking re-
sponsibility & become succesfull
Enables the Product Owner to focus on
value delivery and visualize progress
Helps his team to understand & apply the
ScrumAgile values
Focus on continuous improvement in the
team
Extended Scrum experience & experi-
ments with additional practices
Guides team members in understanding
their role & apply Scrum ‘By the Book’
Enables the Development Team to track
progress & visualize work
Facilitates team events using ScrumAgile
values as a guide
Focus on connecting practices with goals
and outcomes
Has situational experience in running
Scrum ‘By the Book’
Executing operational work & searching
for personal development
Tracks progress & visualizes work
Plans & facilitates all Scrum events
Focus on learning new practices and oper-
ational successes
Learns how to apply Scrum ‘by the book’
Is able to lead complex value chains with
multiple teams & stakeholders
Has impact on customer happiness based
on continuous value delivery
Is responsible for planningbudgetprofit
& loss of the products’ value chain
Focus on continuous value optimizing and
customer collaboration
Deep knowledge on onemore product
portfolio(s)
Can lead a value chain with multiple teams
& stakeholders
Has impact on all stakeholders based on
frequent value creation
Has the mandate to make product related
decisions and release plans
Focus on continuous value creation and
stakeholder collaboration
Deep knowledge of the productvalue
chaincustomers
Inspires his team to collaborate with all
stakeholders in the value chain
Has impact on stakeholders through con-
tinuous outcome
Collaborates with stakeholders to make
plans & product related decisions
Focus on creating value & progress
Has overview on all steps in the value cre-
ation process
Is able to collaborate with multiple stake-
holders from a single Product Backlog
Gains more trust by increasing transpar-
ancy progress & output
Has influence on stakeholders in making
plans & product related decisions
Focus on creating sprint goals & product
increments
Owns & manages the Product Backlog
Is able to translate stakeholders wishes
into concrete work items for a team
Satisfies stakeholders by capturing re-
quirements and feedback
Executes plans & product decisions made
by stakeholders
Focus on creating requirements & Product
Backlog items
Analytical knowledge & basic product
knowledge
Team members trust each other blindly
Delivering value on a frequent basis con-
firmed by actual users
Collaborating with everyone to participate
in sharing feedback & learning
Accountable or creating high quality prod-
ucts
Intuitively acting on commonly shared
standards
Trust & respect are the basis of all acting
Working with stakeholders to deliver high
quality product increments
Collaborating with stakeholders to contin-
uously improve
Responsible and committed to frequently
deliver valuable outcome
Standards are continuously usedchal-
lengedupdated
Team members are open
Actively asking for feedback to have more
impact on stakeholders
Using ScrumAgile values as a guideline
for all interaction
Focussed on reaching sprint goals & im-
proving quality
Common standards are discoveredcap-
turednew insights created
Looking for common understanding
Working on explicit measurements of suc-
cess
Discovering differencesconflictshared
values
Ensuring the outcome of all events &
learning the Scrum values
Knowledgepracticesquality standards
are shareddiscussed
Looking for stabilityresta sense of be-
longing
Following processesrulesinstructions
Avoiding conflictpursuing individual tar-
getsgenerating output
Relying on the Scrum Master to facilitate
them
Individual knowledgestandards deter-
mine what tasks people work on
Delegates all decisions & responsibilities
Facilites people & organization to act with
a Greater Goal in mind
Facilitates entrepreneurship & growth for
every employee
Inspiresguards culturestimulates con-
tinuous improvement
Delegates responsibility for the full Value
Chain to the PO & his team
Delegates all but critical decisions & re-
sponsibilities
Provides a vision & mission and that peo-
ple can connect to
Creates an environment where teams can
self-organize & create value
Gives advicecoachesfacilitates
Delegates planning & execution to the
Product Owner
Delegates more important responsibilities
Tracks progress by regularly visiting team
events
Provides boundary conditions for teams
to set their own targets
Ensures consensus on rulesplansquali-
ty standards.
Co-creates plans together with the Prod-
uct Owner
Wants to be in control & delegates less
critical responsibilities
Asks for explicit measurements of prog-
ressoutputquality
Gives team(s) targets and on progressef-
ficiencyqualityoutcome
Arranges buy-in to rulesplansquality
standards.
Creates plans and delegates execution to
the Product Owner
Wants full control & has a directive com-
munication style
Progress is measured by profit & share-
holder happiness
Gives individual targets and on progress
efficiencyquality &outcome
Ensures compliance to rulesplan execu-
tionquality targets.
Creates plansrulescontrols budget

23. For low maturity teams, it’s sensible to measure practices before “needles move”
If you had these stats, hope shoudn’t be the strategy….focus on discipline needed
• 89% used sprints, 61% used events regularly even though it’s complex work
• 53% had a DoD that was respected and continuously improved
• 71% had daily meetings
• 39% used relative sizing or right sizing
• 21% did product backlog refinement,
• 2 teams used Monte Carlo
• 88% were good with time management and didn’t have an unplanned leave problem
• 2 week sprints for many but only 3 teams had 85th percentile of 12 days or less
• Most teams fed a value chain and didn’t generate any real value themselves
• 2 teams doubled throughput and also managed to reduced cycle times
• Most teams have not reduced cycle time significantly in the last 12 months, due to lack of passion, focus & energy for continuous
improvement
• Most “Kanban teams” had neither WIP limits nor workflow policies
• Less than 33% had a solid team launch

24. No declared negative unintended consequences from agility
Markup denotes updates
March 20XX to July 20YY
green shows improvement
red shows disimprovement
Jan-Mar Vs Jan-Jul
red shows dis-improvement
Jan-Mar Vs Jan-Jul

25. green shows improvement
red shows disimprovement

26. Let’s imagine..
You had executive leaders as follows….what would you do?

27. Scrum.org 21st century executive leadershipLicensed under CC
“I agree / strongly agree” qualifies …. “somewhat agree” does not qualify
The journey to enabling success
Step 1 Apprentice exit criteria (motivating people)
• Only 31% stopped irrefutable demands
• Only 32% reviewing performance trends and limits to growth
• 60% aware of the bottlenecks
Step 2 Tiger exit criteria (begin migration to
product/job teams)
• 48% shifting from broadcast to dialogue
• Only 27% keeping long term stable teams
• Only 27% valuing team performance over “star performers”
The journey to long term success
Step 3 Impediment Blaster exit criteria (org redesign)
• Only 25% support bets < 3 month
• 49% short-lived impediments
• 44% value impact over efficiency
• 40% optimising e2e
• 48% embracing uncertainty
• Only 27% blasting bottlenecks
• 33% Moving to long term contracts
Step 4 System Inverter exit criteria (inspiring people)
• Only 31% ruthless with value and compassionate with people
• Only 40% considering values and trends more important than targets
• Only 29% value effectiveness over efficiency
Step 5 Guardian of the Culture

28. Scrum.org 21st century executive leadershipLicensed under CC
“I agree / strongly agree” qualifies …. “somewhat agree” does not qualify
The journey to enabling success
Step 1 Apprentice exit criteria (motivating people)
• Only 29% stopped irrefutable demands
• Only 31% reviewing performance trends and limits to growth
• 65% aware of the bottlenecks
Step 2 Tiger exit criteria (begin migration to
product/job teams)
• 48% shifting from broadcast to dialogue
• Only 27% keeping long term stable teams
• Only 27% valuing team performance over “star performers”
The journey to long term success
Step 3 Impediment Blaster exit criteria (org redesign)
• Only 25% support bets < 3 month
• 49% short-lived impediments
• 44% value impact over efficiency
• 40% optimising e2e
• 48% embracing uncertainty
• Only 27% blasting bottlenecks
• 33% Moving to long term contracts
Step 4 System Inverter exit criteria (inspiring people)
• Only 31% ruthless with value and compassionate with people
• Only 40% considering values and trends more important than targets
• Only 29% value effectiveness over efficiency
Step 5 Guardian of the Culture

29. Scrum.org 21st century executive leadershipLicensed under CC
“I agree / strongly agree” qualifies …. “somewhat agree” does not qualify
The journey to enabling success
Step 1 Apprentice exit criteria (motivating people)
• Only 29% stopped irrefutable demands
• Only 31% reviewing performance trends and limits to growth
• 65% aware of the bottlenecks
Step 2 Tiger exit criteria (begin migration to
product/job teams)
• 47% shifting from broadcast to dialogue
• Only 25% keeping long term stable teams
• Only 26% valuing team performance over “star performers”
The journey to long term success
Step 3 Impediment Blaster exit criteria (org redesign)
• Only 25% support bets < 3 month
• 49% short-lived impediments
• 44% value impact over efficiency
• 40% optimising e2e
• 48% embracing uncertainty
• Only 27% blasting bottlenecks
• 33% Moving to long term contracts
Step 4 System Inverter exit criteria (inspiring people)
• Only 31% ruthless with value and compassionate with people
• Only 40% considering values and trends more important than targets
• Only 29% value effectiveness over efficiency
Step 5 Guardian of the Culture

30. Scrum.org 21st century executive leadershipLicensed under CC
“I agree / strongly agree” qualifies …. “somewhat agree” does not qualify
The journey to enabling success
Step 1 Apprentice exit criteria (motivating people)
• Only 29% stopped irrefutable demands
• Only 31% reviewing performance trends and limits to growth
• 65% aware of the bottlenecks
Step 2 Tiger exit criteria (begin migration to
product/job teams)
• 47% shifting from broadcast to dialogue
• Only 25% keeping long term stable teams
• Only 26% valuing team performance over “star performers”
The journey to long term success
Step 3 Impediment Blaster exit criteria (org redesign)
• Only 24% support bets < 3 month
• 48% short-lived impediments
• 43% value impact over efficiency
• 39% optimising e2e
• 47% embracing uncertainty
• Only 26% blasting bottlenecks
• 32% Moving to long term contracts
Step 4 System Inverter exit criteria (inspiring people)
• Only 31% ruthless with value and compassionate with people
• Only 40% considering values and trends more important than targets
• Only 29% value effectiveness over efficiency
Step 5 Guardian of the Culture

31. Scrum.org 21st century executive leadershipLicensed under CC
“I agree / strongly agree” qualifies …. “somewhat agree” does not qualify
The journey to enabling success
Step 1 Apprentice exit criteria (motivating people)
• Only 29% stopped irrefutable demands
• Only 31% reviewing performance trends and limits to growth
• 65% aware of the bottlenecks
Step 2 Tiger exit criteria (begin migration to
product/job teams)
• 47% shifting from broadcast to dialogue
• Only 25% keeping long term stable teams
• Only 26% valuing team performance over “star performers”
The journey to long term success
Step 3 Impediment Blaster exit criteria (org redesign)
• Only 24% support bets < 3 month
• 48% short-lived impediments
• 43% value impact over efficiency
• 39% optimising e2e
• 47% embracing uncertainty
• Only 26% blasting bottlenecks
• 32% Moving to long term contracts
Step 4 System Inverter exit criteria (inspiring people)
• Only 32% ruthless with value and compassionate with people
• Only 41% considering values and trends more important than targets
• Only 31% value effectiveness over efficiency
Step 5 Guardian of the Culture

32. A wise man told me…
It’s about inspiring people…. even if they leave
Thank you!
https://www.scrum.org/resources/blog?uid=217428
https://Ace.works/blog
https://Valueglide.com/blog
https://ace.works/TrainingEvents
@JohnColemanIRL