Business Value of Agile Methods: Using ROI and REal Options
1. Business Value of
Agile Methods
Using ROI & Real Options
Dr. David F. Rico, PMP, ACP, CSM
Twitter: @dr_david_f_rico
Website: http://www.davidfrico.com
LinkedIn: http://www.linkedin.com/in/davidfrico
Facebook: http://www.facebook.com/profile.php?id=1540017424
2. Author Background
DoD contractor with 28+ years of IT experience
B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys.
Large gov’t projects in U.S., Far/Mid-East, & Europe
Published six books & numerous journal articles
Adjunct at George Washington, UMUC, & Argosy
Agile Program Management & Lean Development
Specializes in metrics, models, & cost engineering
Six Sigma, CMMI, ISO 9001, DoDAF, & DoD 5000
Cloud Computing, SOA, Web Services, FOSS, etc.
2
3. Today’s Whirlwind Environment
Global Reduced
Competition IT Budgets
Work Life Obsolete
Imbalance Technology & Skills
Demanding 81 Month
Overruns Customers Cycle Times Inefficiency
Vague Attrition High O&M Overburdening
Requirements Escalation Lower DoQ Legacy Systems
Runaways Vulnerable
Cancellation N-M Breach
Organization Redundant
Downsizing Data Centers
Technology Poor
Change System Lack of IT Security
Complexity Interoperability
Pine, B. J. (1993). Mass customization: The new frontier in business competition. Boston, MA: Harvard Business School Press.
Pontius, R. W. (2012). Acquisition of IT: Improving efficiency and effectiveness in IT acquisition in the DoD. Second Annual
AFEI/NDIA Conference on Agile in DoD, Springfield, VA, USA. 3
4. Software in U.S. DoD Systems
No. of software-intensive systems is growing
80% of US DoD functions performed in software
Major driver of cost, schedule, & tech. performance
Kennedy, M. P., & Umphress, D. A. (2011). An agile systems engineering process: The missing link. Crosstalk, 24(3), 16-20.
4
5. Traditional Projects
Big projects result in poor quality and scope changes
Productivity declines with long queues/wait times
Large projects are unsuccessful or canceled
Size vs. Quality Size vs. Requirements Growth
16.00 40%
Defect Density
12.80 32%
Percentage
9.60 24%
6.40 16%
3.20 8%
0.00 0%
0 2 6 25 100 400 0 2 6 25 100 400
Lines of Code (Thousands) Lines of Code (Thousands)
Size vs. Productivity Size vs. Success
5.00 60%
Code Production Rate
4.00 48%
Percentage
3.00 36%
2.00 24%
1.00 12%
0.00 0%
0 2 6 25 100 400 0 2 6 25 100 400
Lines of Code (Thousands) Lines of Code (Thousands)
Jones, C. (1991). Applied software measurement: Assuring productivity and quality. New York, NY: McGraw-Hill. 5
6. Global Project Failures
Challenged and failed projects hover at 67%
Big projects fail more often, which is 5% to 10%
Of $1.7T spent on IT projects, over $858B were lost
$1.8
2010 33% 41% 26%
2008 32% 44% 24%
$1.4
Trillions (US Dollars)
2006 35% 46% 19%
2004 29% 53% 18% $1.1
Year
2002 34% 51% 15%
2000 28% 49% 23% $0.7
1998 26% 46% 28%
$0.4
1996 27% 33% 40%
1994 16% 53% 31%
$0.0
0% 20% 40% 60% 80% 100% 2002 2003 2004 2005 2006 2007 2008 2009 2010
Successful Challenged Failed Expenditures Failed Investments
Standish Group. (2010). Chaos summary 2010. Boston, MA: Author.
Sessions, R. (2009). The IT complexity crisis: Danger and opportunity. Houston, TX: Object Watch.
6
7. Requirements Defects & Waste
Requirements defects are #1 reason projects fail
Traditional projects specify too many requirements
More than 65% of requirements are never used at all
Defects Waste
Never
Requirements
45%
47%
Other 7% Always 7%
Implementation
Often 13%
18% Rarely
Design 19%
28% Sometimes
16%
Sheldon, F. T. et al. (1992). Reliability measurement: From theory to practice. IEEE Software, 9(4), 13-20
Johnson, J. (2002). ROI: It's your job. Extreme Programming 2002 Conference, Alghero, Sardinia, Italy.
7
8. What is Agility?
A-gil-i-ty (ə-'ji-lə-tē) Property consisting of quickness,
lightness, and ease of movement; To be very nimble
The ability to create and respond to change in order to
profit in a turbulent global business environment
The ability to quickly reprioritize use of resources when
requirements, technology, and knowledge shift
A very fast response to sudden market changes and
emerging threats by intensive customer interaction
Use of evolutionary, incremental, and iterative delivery
to converge on an optimal customer solution
Maximizing BUSINESS VALUE with right sized, just-
enough, and just-in-time processes and documentation
Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.
8
9. What are Agile Methods?
People-centric way to create innovative solutions
Product-centric alternative to documents/process
Market-centric model to maximize business value
Customer Collaboration Contracts
Frequent comm.
Close proximity
Regular meetings
Multiple comm. channels
Frequent feedback
Relationship strength
valued
more than
Contract compliance
Contract deliverables
Contract change orders
Individuals & Interactions Processes
Leadership Competence valued Lifecycle compliance
Boundaries Structure more than Process Maturity Level
Empowerment Manageability/Motivation
Working Software
Regulatory compliance
Documentation
Clear objectives Timeboxed iterations
Small/feasible scope Valid operational results
Acceptance criteria Regular cadence/intervals
valued
more than
Document deliveries
Document comments
Document compliance
Responding to Change Project Plans
Org. flexibility System flexibility valued Cost Compliance
Mgt. flexibility Technology flexibility more than Scope Compliance
Process flexibility Infrastructure flexibility Schedule Compliance
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
9
10. How Do Agile Methods Work?
Agile methods DON’T mean deliver it now & fix it later
Lightweight, yet disciplined approach to development
Reduced cost, risk, & waste while improving quality
What How Result
Flexibility Use lightweight, yet disciplined processes and artifacts Low work-in-process
Customer Involve customers early and often throughout development Early feedback
Prioritize Identify highest-priority, value-adding business needs Focus resources
Descope Descope complex programs by an order of magnitude Simplify problem
Decompose Divide the remaining scope into smaller batches Manageable pieces
Iterate Implement pieces one at a time over long periods of time Diffuse risk
Leanness Architect and design the system one iteration at a time JIT waste-free design
Swarm Implement each component in small cross-functional teams Knowledge transfer
Collaborate Use frequent informal communications as often as possible Efficient data transfer
Test Early Incrementally test each component as it is developed Early verification
Test Often Perform system-level regression testing every few minutes Early validation
Adapt Frequently identify optimal process and product solutions Improve performance
Rico, D. F. (2012). What’s really happening in agile methods: Its principles revisited? Retrieved June 6, 2012, from http://davidfrico.com/agile-principles.pdf
10
11. Agile World View
“Agility” has many dimensions other than IT
It ranges from leadership to technological agility
The focus of this brief is program management agility
Agile Leaders
Agile Organization Change
Agile Acquisition & Contracting
Agile Strategic Planning
Agile Capability Analysis
Agile Program Management
Agile Project Management
Agile Systems Development
Agile Processes & Practices
Agile Tools
Agile Information Systems
Agile Tech. 11
12. Agile Enterprise Delivery Model
Begins with a high-level product vision/architecture
Continues with needs development/release planning
Includes agile delivery teams to realize business value
Leffingwell, D. (2011). Agile software requirements: Lean requirements practices for teams, programs, and the enterprise. Boston, MA: Pearson Education.
12
13. Agile Methods
VersionOne found 80% using agile methods today
Most are using Scrum with several key XP practices
Number of CSMs have doubled to 200,000 in 2 years
House, D. (2012). Sixth annual state of agile survey: State of agile development. Atlanta, GA: VersionOne. 13
14. Agile Practices
VersionOne found 65% using Scrum practices
55% are using some of top XP technical practices
Lean-Kanban is a rising practice with a 24% adoption
Continuous
Integration
House, D. (2012). Sixth annual state of agile survey: State of agile development. Atlanta, GA: VersionOne.
14
15. Studies of Agile Methods
Dozens of surveys of agile methods since 2003
100s of Agile and CMMI case studies documented
Agile productivity, quality, and cost better than CMMI
Rico, D. F. (2008). What is the return-on-investment of agile methods? Retrieved February 3, 2009, from http://davidfrico.com/rico08a.pdf
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
15
16. Agile Cost of Quality (CoQ)
Agile testing is 10x better than code inspections
Agile testing is 100x better than traditional testing
Agile testing is done earlier “and” 1,000x more often
Rico, D. F. (2012). The Cost of Quality (CoQ) for Agile vs. Traditional Project Management. Fairfax, VA: Gantthead.Com.
16
17. Agile Cost & Benefit Analysis
Costs based on avg. productivity and quality
Productivity ranged from 4.7 to 5.9 LOC an hour
Costs were $588,202 and benefits were $3,930,631
5
i 1
d1 = [ln(Benefits Costs) + (Rate + 0.5 Risk2) Years] Risk Years, d2 = d1 Risk Years
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation.
Ft. Lauderdale, FL: J. Ross Publishing. 17
18. Benefits of Agile Methods
Analysis of 23 agile vs. 7,500 traditional projects
Agile projects are 54% better than traditional ones
Agile has lower costs (61%) and fewer defects (93%)
2.8 18
Before Agile Before Agile
3.00 20
After Agile After Agile
2.25 15 11
1.1
1.50 10
0.75 61% 5 39%
Lower Less
Cost Staff
Project Cost in Millions $ Total Staffing
18 2270
Before Agile Before Agile
20 2500
13.5 After Agile After Agile
15 1875
10 1250
381
5 24% 625 93%
Less
Faster Defects
Delivery Time in Months Cumulative Defects
Mah, M. (2008). Measuring agile in the enterprise: Proceedings of the Agile 2008 Conference, Toronto, Canada.
18
19. Agile vs. Traditional Success
Traditional projects succeed at 50% industry avg.
Traditional projects are challenged 20% more often
Agile projects succeed 3x more and fail 3x less often
Agile Traditional
Success
Success
14%
42%
Challenged
57%
Failed
Failed Challenged 29%
9% 49%
Standish Group. (2012). Chaos manifesto. Boston, MA: Author.
19
20. Agile vs. Traditional Outcomes
Agile requirements implemented in slices vs. layers
User needs with higher business value are done first
Reduces cost & risk while increasing business success
Agile Traditional
Faster 1 2 3 Late
GUI
Early ROI No Value
APIs
Lower Costs Applications Cost Overruns
Fewer Defects Middleware Very Poor Quality
Operating System
Manageable Risk Uncontrollable Risk
Computer
Better Performance Network
Slowest Performance
Smaller Attack Surface Seven Wastes
More Security Incidents
1.Rework
JIT, Just-enough architecture 2.Motion Myth of perfect architecture
Early, in-process system V&V 3.Waiting Late big-bang integration tests
Fast continuous improvement MINIMIZES 4.Inventory MAXIMIZES Year long improvement cycles
Scalable to systems of systems 5.Transportation Breaks down on large projects
Maximizes successful outcomes 6.Overprocessing Undermines business success
7.Overproduction
Shore, J. (2011). Evolutionary design illustrated. Norwegian Developers Conference, Oslo, Norway. 20
21. Benefits of Organizational Agility
Study of 15 agile vs. non-agile Fortune 500 firms
Based on models to measure organizational agility
Agile firms out perform non agile firms by up to 36%
Hoque, F., et al. (2007). Business technology convergence. The role of business technology convergence in innovation
and adaptability and its effect on financial performance. Stamford, CT: BTM Institute.
21
22. Agile Industry Case Studies
80% of worldwide IT projects use agile methods
Includes regulated industries, i.e., DoD, FDA, etc.
Agile now used for safety critical systems, FBI, etc.
Industry Org Project Purpose Size Metrics
20 teams 1,838 User Stories
Electronic
Google Adwords Advertising 140 people 6,250 Function Points
Commerce 5 countries 500,000 Lines of Code
15 teams 26,809 User Stories
Shrink Project
Wrapped
Primavera Primavera 90 people
Management Collocated
91,146 Function Points
7,291,666 Lines of Code
4 teams 1,659 User Stories
Health Blood
Care
FDA m2000
Analysis
20 people
Collocated
5,640 Function Points
451,235 Lines of Code
10 teams 3,947 User Stories
Law Case File
Enforcement
FBI Sentinel
Workflow
50 people
Collocated
13,419 Function Points
1,073,529 Lines of Code
U.S. Knowledge 3 teams 390 User Stories
Stratcom SKIweb 12 people 1,324 Function Points
DoD Management Collocated 105,958 Lines of Code
Rico, D. F. (2010). Lean and agile project management: For large programs and projects. Proceedings of the First International Conference on Lean
Enterprise Software and Systems, Helsinki, Finland, 37-43.
22
23. Perceptions of Agile Methods
Structure, reward, decision, staffing, leadership, etc.
Top-down, individualism, regulation, compliance, etc.
Focus on reforming acquisition & procurement system
Type/Kind Common DoD Agile Perceptions Reality with Respect to Agile Methods
Discipline Undisciplined Cowboy Coding Rigorous process, plans, requirements, QA, CM, testing, documents etc.
Scalability Only Applies Small Projects Used by 100, 500, 1,000, 10,000+ person person projects & organizations
Domain
Management
Only for Protoperational Systems Used in DoD, medical devices, avionics, autos, electronics, etc.
Flexible Scope/Can't Use EVM Lightweight EVM model is used with its release planning methodology
Requirements Doesn't Use Requirements Always begins with valuable, well-defined, & prioritized requirements
Architecture Spaghetti Code from Iterations Begins with lean architecture or create waste-free emergent design
Quality No Documents/Unmaintainable Electronic plans, requirements, designs, tests, manuals, documents, etc.
Inspections High CoQ from No Inspections One or two orders of magnitude more inspections & tests performed
Security Vulnerabilities from Hacking Security practices result in smaller attack surface & fewer vulnerabilities
Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation.
Ft. Lauderdale, FL: J. Ross Publishing.
23
24. Conclusion
Agility is the evolution of management thought
Confluence of traditional and non-traditional ideas
Improve performance by over an order of magnitude
Agile methods are …
Systems development approaches
New product development approaches
Expertly designed to be fast and efficient
Intentionally lean and free of waste (muda)
Systematic highly-disciplined approaches
Capable of producing high quality systems
Right-sized, just-enough, and just-in-time tools
Scalable to large, complex mission-critical systems
Designed to maximize business value for customers
“The world of traditional methods belongs to yesterday”
“Don’t waste your time using traditional methods on 21st century projects”
Wysocki, R.F. (2010). Adaptive project framework: Managing complexity in the face of uncertainty. Boston, MA: Pearson Education. 24
25. Books on ROI of SW Methods
Guides to software methods for business leaders
Communicates business value of software methods
Rosetta stones to unlocking ROI of software methods
http://davidfrico.com/agile-book.htm (Description)
http://davidfrico.com/roi-book.htm (Description)
25