The document discusses integrating quality into portfolio management. It describes how adaptive planning is necessary due to inherent uncertainty in software development. Traditional earned value management (EVM) expects full definition upfront and does not assert quality, but AgileEVM combines Agile practices with EVM to measure outcomes rather than outputs and prioritize to maximize business value. The presentation provides examples of how AgileEVM can be used to forecast releases and make decisions when commitments are at risk.

1. Integrating Quality into Portfolio Management
PMI Silicon Valley Chapter
Jan 17, 2011
Brent Barton
1

2. Brent Barton - Sterling Barton, LLC
Partner: Sterling Barton, LLC
and President: AgileEVM, Inc.
Former CTO, Development Manager, PMO Manager,
Agile Coach, Mentor, Certified Scrum Trainer,
ScrumMaster, Product Owner
Active practitioner delivering value using Agile and
helping others do it; from small Product companies to
very large IT organizations
Agile Articles
Email: brent@sterlingbarton.com
“Manage Project Portfolios More Effectively by Including brent@agileevm.com
Software Debt in the Decision Process”, Cutter Journal Web: www.agileevm.com
2010 www.sterlingbarton.com
Blog: gettingagile.com
“AgileEVM – Earned Value Management in Scrum Follow me on Twitter: brentbarton
Projects”, IEEE 2006
“Implementing a Professional Services Organization Using
Type C Scrum”, IEEE
“Establishing and Maintaining Top to Bottom
Transparency Using the Meta-Scrum”, AgileJournal
“All-Out Organizational Scrum as an Innovation Value
Chain”, IEEE
© 2009-2011, 2

3. Two Important Themes for Tonight
Asserting Quality
➡Opportunity from Agile
Leveraging Adaptive Planning
➡Consequence of Agile
© 2009-2011, 3

4. Complexity Requires Adaptive Planning
It is not possible to completely specify an interactive system.
Wegner’s Lemma, 1995
Uncertainty is inherent and inevitable in software development
processes and products.
Ziv’s Uncertainty Principle, 1996
For a new software system the requirements will not be
completely known until after the users have used it.
Humphrey’s Requirements Uncertainty Principle, c.1998
© 2009-2011, 4

5. Meet Earl -
Strategic Planner
Earl just finished the annual
portfolio budgeting process for
the new fiscal year!
© 2009-2011, 5

6. Meet Geoff -
Project Manager
Geoff was a Software
Developer
and is now in charge of Saturn
Saturn is a key part of a
company-wide strategy
© 2009-2011, 6

7. Portfolio is Done for the year!
(This month is the beginning of our new Fiscal Year)
© 2009-2011, 7

8. Three Months Later...
© 2009-2011, 8

9. 3 months later, things aren’t looking as good
© 2009-2011, 9

10. So...what is happening? Who is affected?
© 2009-2011, 10

11. Another Month Later...
© 2009-2011, 11

12. Defect Containment is helping...not solving
© 2009-2011, 12

13. Continuous Integration is “My Friend”
© 2009-2011, 13

14. Meet Huang -
Test Engineer
Huang is a Software
Development Engineer in Test
- SDET
Huang wants to proud of the
quality of every release
How can I help?
© 2009-2011, 14

15. © 2009-2011, 15

16. Meet Sonia -
Program Manager
Sonia is a Program Manager
Uses Scrum well...good
servant leader
How can I help
ensure better
Quality?
© 2009-2010,

17. Definition of Done
Defines the work products that will
be delivered with each item as it is
ready for acceptance
Typical entries in Definition of Done
Code includes unit tests,
reviewed, checked in
Tests described and executed
Build, release notes
Compliance documentation
updated to include current
functionality
What else?
© 2009-2010, 17

18. Definition of Done as a Compliance Checklist
Acceptance defined criteria for each Code checked in with reference to
user story US#/Task#
Unit tests written and passed Tested on FE
Code compiles with no errors and no Integration test written & passes
warnings
Test code reviewed
New code doesn’t break existing code
Environment requirements documented
Test case review (Dev to review test
Interface document updated/added
case written)
and checked in to SVN
Architectural impact assessed and
Acceptance criteria verified complete
artifacts updated if necessary
All P1-P3 bugs for the story are
Comments in code
closed
Error codes added
Test approves user story
Code reviewed by peer
Story demonstrated to product owner
and accepted on Target Platform
© 2009-2010,

19. How does a “Release Definition of Done” help?
Every release should have clear quality criteria
With a “Release Definition of Done” you can understand targets better
Measure the gap between the teams’ Definition of Done and a Release
Definition of Done.
This gap is a source of quality issues and represents significant risk to
schedule
© 2009-2010,

20. [Pause for enlightening discussion]
© 2009-2011, 20

21. Two Important Themes for Tonight
Asserting Quality
Leveraging Adaptive Planning
© 2009-2011, 21

22. Meet Earl -
Strategic Planner
How do I balance
Value and Quality?
© 2009-2011, 22

23. Balancing Signal Indicators
- (some weaker than others)
Value
Quality Constraints
(Schedule, Cost, Scope)
© 2009-2011, Source: Jim Highsmith 23

24. Project Portfolio Defined
A portfolio is a collection of projects or
programs and other work that are
grouped together to facilitate effective
management of that work to meet
strategic business objectives.
source: PMI The Standard for Portfolio Management 24
© 2009-2011,
— Second Edition

25. Project Portfolio Management
We want to measure
outcomes, not outputs
YES: Business Value Business
Value
not so much: Completed
Projects
Time
© 2009-2011, 25

26. Effective Project Portfolio Management
Requires:
Prioritization to maximize Business Value
Effective delivery to minimize costs
Re-allocation of resources when costs are too high
or the benefit is too low
© 2009-2011, source: Cutter Journal 26

27. Agile partially supports
Portfolio Management
• Prioritize to maximize Agile
Business Value
• Effectively deliver to Agile
minimize costs
• Re-allocate resources
when costs are too high or
the benefit is too low

28. Earned Value
Management (EVM)

29. Earned Value
Over EAC
Total Allocated Budget Budget Estimate at Complete
Management Reserve
PMB
Project Management Baseline
Projected Slippage
$
Planned Value
Actual Cost
(PV)
(AC) (EV)
Earned Value
Time
Time Completion
Now Date

30. EVM Performance
Indicators
Cost Performance Index (CPI=EV/AC)
CPI < 1 CPI = 1 CPI > 1
Over Budget On Budget Under Budget
Schedule Performance Index (SPI=EV/PV)
SPI < 1 SPI = 1 SPI > 1
Behind Schedule On Schedule Ahead of Schedule

31. Strengths of EVM
• Integrates cost and schedule management
• Forecasts in financial units based on units
used for actual cost
• Decades of use
• Part of PMBOK (ANSI/PMI 99-001-2008)
• Part of EVMS (ANSI/EIA-748-B-2007)

32. Weaknesses of
Traditional EVM
• Expects everything Ugh!
fully defined up front
• No assertion of quality
• Claiming value is earned
on intermediate
work products

33. Agile + EVM
• We want to measure +
Agile
EVM
outcomes, not outputs
• Prioritize to maximize Agile
Business Value
• Effectively deliver to Agile
minimize costs
• Re-allocate resources
when costs are too high or EVM
the benefit is too low

34. [Pause for Reflection]

35. AgileEVM

36. AgileEVM Background
• Mathematically proven that Release Dates
based on average velocity (story points)≡
estimate at complete (dollars)
• Key Assumption: The ratio of (story points
completed)/(total story points in a release)
is a good measure of Actual Percent
Complete

37. Scenario 1: Commit
37

38. 38

39. Scenario 2: Transform
39

40. 40

41. 41

42. 42

43. Quality Criteria
asserted by team
43

44. Scenario 3: Kill
44

45. 45

46. What are some options?
Saturn RC-1 Forecasts
Mercury Forecasts
46

47. Questions?
© 2009-2011, 47

48. Brent Barton - Sterling Barton, LLC
Partner: Sterling Barton, LLC
and President: AgileEVM, Inc.
Former CTO, Development Manager, PMO Manager,
Agile Coach, Mentor, Certified Scrum Trainer,
ScrumMaster, Product Owner
Active practitioner delivering value using Agile and
helping others do it; from small Product companies to
very large IT organizations
Agile Articles
Email: brent@sterlingbarton.com
“Manage Project Portfolios More Effectively by Including brent@agileevm.com
Software Debt in the Decision Process”, Cutter Journal Web: www.agileevm.com
2010 www.sterlingbarton.com
Blog: gettingagile.com
“AgileEVM – Earned Value Management in Scrum Follow me on Twitter: brentbarton
Projects”, IEEE 2006
“Implementing a Professional Services Organization Using
Type C Scrum”, IEEE
“Establishing and Maintaining Top to Bottom
Transparency Using the Meta-Scrum”, AgileJournal
“All-Out Organizational Scrum as an Innovation Value
Chain”, IEEE
© 2009-2011, 48