Quantifying DevOps Adoption Empirically for Demonstrable ROI

1. Quantifying DevOps Adoption
Empirically for Demonstrable ROI
DES-1720

2. Insight
Operations
Interaction

3. Insight
Operations
Interaction
Innovation
and Speed
Optimize

4. Friction Reduction with multi-speed IT
Optimize
Innovate
Focus on Operational Costs
• Consolidation and modernization
• Operations automation
• Risk and compliance mgmt
Empowered Process Team
Legacy Infrastructure
Focus on Speed and Agility
• Assemble solutions from verified
components
• Fast development and redeployment
• User first delivery model
Systems of Engagement
Rapid Service Creation Team
PaaS Engine
Insight
Operations
Hybrid
Enterprise
Mobile
Social
Interaction
Service Integration Team
Brokerage Services
DevOps
API Economy
Cloud Service Brokers
Systems of Record
Reduction

5. Why DevOps is Important
Line-of-
business
Customer
1
3
2
1. Get ideas into production fast
2. Get people to use it
3. Get feedback
Adopt DevOps approach to continuously
manage changes, obtain feedback and ,
deliver changes to users
Eliminate any
activity that is not
necessary for
learning what
customers want

6. What DevOps Practices should Address…
Manual Effort
Unproductive Work Poor Visibility
Long Wait Times

7. Lets Walk through a Story…
• Meet Julie
• Julie works in the LOB and has an Idea for a new
solution to address a key business need.
…Julie is daunted by the current process:
Business Need
•Document
•Weeks
Approval
Process
•Meetings
•Weeks
Hardware
Purchase
•Negotiations
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks

8. Traditional Product Development
Months before you find out whether the product meets the need
Business Need
•Document
•Weeks
Approval
Process
•Meetings
•Weeks
Hardware
Purchase
•Negotiations
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks

9. Introducing IaaS
• Months before you find out whether the product meets the need
• Hardware provisioning is undifferentiated heavy lifting
Business Need
•Document
•Weeks
Approval
Process
•Meetings
•Weeks
Hardware
Purchase
•Negotiations
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks
IaaS
Cloud

10. IaaS Summary
• Months before you find out whether the product meets the need
• Hardware provisioning is undifferentiated heavy lifting
Business Need
•Document
•Weeks
Approval
Process
•Meetings
•Weeks
Hardware
Purchase
•Negotiations
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks
IaaS
Cloud
Business Need
•Document
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks
From months to
weeks is good.
Can we do better?

11. IaaS based Product Development
• Weeks before you find out whether the product meets the need
Business Need
•Document
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks

12. IaaS based Product Development with DevOps
• Improved efficiencies (including automation) between development and
deployment has a significant impact in consistency and speed
Business Need
•Document
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks
Dev
Ops

13. IaaS based Product Development with DevOps
Business Need
•Document
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks
Automation is great and
reduces the time to days
But I want more!
Dev
Ops

14. Introducing PaaS for Feature Delivery
Business Need
•Document
•Weeks
Software
Development
•Specifications
•Weeks
Deployment
and Testing
•Reports
•Weeks
Customer
Feedback
•Doesn’t work!
•Weeks
DevOpsPaaS
w/DevOp
s

15. PaaS based Feature Development
Business Need
•Discussion
•Days
• Design
Thinking
Software
Development
•Code
•Weeks-Days
Customer
Feedback
•Fix this Bit!
•Hours
Days!
That is what I’m
talking about!

16. 15

17. “The Technology Tree”

18. DevOps “Technology Tree” (Phase 1)
Phase 1 Goal:
Continuous
Testing
Starting Point
Step #1: Automated
Builds Step #2: Automated Functional Testing
as part of build process
Step #3: Code Scanning and Security
scanning as part of build process Step #4: Shift Left Testing (Integration
Testing before Functional Testing)
Step #5: Automated application
deployment to Test Environment
Step 6: On-Demand Test
Environment Provisioning

19. DevOps “Technology Tree” (Phase 2)
Phase 2 Goal:
Continuous
Delivery
Starting Point
Step #1: Continuous
Testing
Step #2: Automated application deployment to test
environment along with middleware configuration
(standard topologies)
Step #3: Automated application
deployment and middleware
configuration across DEV, QA, and Prod
Application Portfolio Release
Management
Step #5: Self Service build,
provision, and deploy (Cloud)
Step #4: Manage Environments
through automation

20. What do you see?

21. 52% of users
don’t have
confidence
in their information1
42% of managers
use wrong
information
at least once a week2
59% of managers
miss
information
they should have used2
Product
Profitability
Product
Variance
And Health
Workforce
Optimization
Distributed and
Outsourced
Development
Contradictions result from the quest for information

22. Metrics That Matter

23. Metrics That Matter
Category Metrics
Cost / Budget Maintenance/Innovation Ratio (% spent on bug fixes/technical debt vs new capabilities)
Assignment scope (Number of function points supported by person)
Budget underage/overage
Time / Speed Velocity Planned (Throughput of team per iteration/milestone)
Velocity Actual (Historical Velocity)
Feature cycle time (Time / Effort for developing a feature)
Turnaround Time for Changes (CR or Defect -> Production)
On-time delivery % (# of projects delivered on time / Total # of projects)
Quality / Reliability Uptime/MTBF (Mean time between failures)
Defect Backlog (# of defects not fixed in a release)
Total Tests (# of tests for application)
Total % Tests Automated
Test Escapes (# of defects not found in testing)
Support Call Volume
Defect Density (# of defects per Function point or KLOC/KELOC)
Regression (# of defects fixed that have creeped back into system)
Requirements Coverage (Total # of Requirements vs # of Requirements implemented)
Requirements Volatility (Measuring changes in requirements by lifecycle)
System Utilization (Measures Resource Utilization during performance/stress testing)
Test Case Efficiency (Ratio of Defects found in Test Cases vs total Defects)
Size (Underlying) Story Points
KLOC, KELOC
Function Points
Backfiring (KELOC -> Function Point mapping)
Code Complexity (# of methods that have >15 linear independent paths)
Percent Reuse ((Reused code count / (Reused code count + total code count)) * 100)

24. Rational Enterprise DevOps solutions save money and increase resource capacity
• By increasing staff capacity through productivity improvements
– Examples include:
• Faster code development and maintenance
• Faster and improved impact analysis and testing
• Build automation
• Quality Management and Test Automation
• Configuration Management
• Time to market
• By discovering defects earlier in the lifecycle thus reducing the repair costs
– Examples include:
• Improved Requirements Analysis and Management
• Code Analysis
• “Shift Left” strategy. Using transparency, continuous integration and testing to identify issues early
23
Quality
Productivity

25. Monetizing the Value of Productivity Improvement
How do you measure value?
– Lines of Code (LOC) and Function
Point (FP) – a lot of baseline data
available, but poor indicators of value.
– Story Points – used by agile teams,
often no baseline data exists…
– Business Points¹ – few measure it, but
focus on value. Often, no baseline data
exists…
• Solution:
• Measure Velocity, that is, how
much faster is the team working
now relative to solution adoption.
• To calculate velocity of your agile
team, simply add up the estimates
of the features, user stories,
requirements or backlog items
successfully delivered in an
iteration.
24
¹While Story Points reflects the size of the implementation effort, Business Points
reflect the business value associated with each story / epic / scenario / …

26. Monetizing the Value of Quality Improvement
Two key drivers:
1. Fewer defects found post deployment
– A result of better test coverage, automated and frequent testing
2. The average defect is found earlier in the lifecycle, when the
cost of finding them is lower
25

27. Case Study: Enterprise DevOps Cost/Benefit Analysis
26

28. IBM Rational DevOps Business Case Assumptions
•Average staff costs $65/hr.
•400 users
•5-year investment analysis
•Adoption Rates:
–30% after 12 months
–60% after 24 months
–90% after 36 months
–100% after 48 months

29. Business Value Model – Productivity Impact
29
$3.4 million annual savings through combined FTE time
Increase software delivery capacity of 26 FTE from 3.3% productivity improvements

30. Expected Productivity Impact Causation
30

31. Business Value Model – Quality Impact
31
Based upon other Client implementations and recorded results, this initial draft
proposes a left shift in defect detection and resulting $1.8m cost reduction

32. Expected Quality Impact Causation

33. Business Value Model
Cost Benefit Summary

34. Business Value Assessment
Preparing a Business Value Assessment includes the following steps:
1.Initial presentation to get Client participation
2.Interview with Client to capture processes and metrics
3.Presentation slides, analysis document & spreadsheet
4.Review and acceptance with Client / Sponsor
The BVA is a consultative sales method that
documents the business impact with the client of a
proposed solution. Following this, it compares the
cost of a proposed solution to the business value so
that an ROI can be presented.
The BVA communicates and prioritizes the
proposed solution business value to the executive
and financial level just as the technical validation
sells to the practitioner and technology level
.
34

35. System Z DevOps Workshop Description
The System Z DevOps Workshop is a no cost ‘discovery’ workshop to identify gaps
in the client’s current System Z software delivery capabilities and provide them
with a customized, prioritized list of DevOps practices and a roadmap for their
adoption to support faster and more efficient enterprise software delivery.
The target audience for the workshop are executives and senior development and
operations managers. The workshop is designed to last 4 to 6 hours. It is preceded
by a 30 minute pre-call and followed up with a 2 hour presentation of results and
recommendations.
The workshop is meant to be a strategic initiative to help the customer develop a
pragmatic approach for adoption of Enterprise DevOps best practices building on
initiatives and capabilities already in place

36. Next Steps / Resources
• DES-4657: Enterprise DevOps Cost/Benefit Analysis Workshop at InterConnect
• Take the DevOps Practices Self Assessment
http://www.ibm.com/ibm/devops/us/en/
• Additional Resources
http://www.ibm.com/devops
https://www.ibm.com/developerworks/devops/
36

37. Notices and Disclaimers
37
Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission
from IBM.
U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of
initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS
DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE
USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY.
IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.
Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.
Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers
have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in
which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials
and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or
their specific situation.
It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and
interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such
laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

38. Notices and Disclaimers Con’t.
38
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not
tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products.
Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the
ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT
NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual
property right.
IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®,
FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG,
Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®,
PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®,
StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business
Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM
trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

39. Thank You
Your Feedback is Important!
Access the InterConnect 2016 Conference Attendee
Portal to complete your session surveys from your
smartphone,
laptop or conference kiosk.