The document provides an overview of steps for determining a Value Stream Management (VSM) solution for an organization. It begins with an introduction of the speakers and outlines the webinar goals of explaining the comprehensive process for selecting a VSM solution. The webinar then details each step, including understanding why the steps are important, how solutions are determined through activities like future state mapping and return on investment analysis, and the expected outcome of team alignment around a recommended solution.
Critical steps in Determining Your Value Stream Management Solution
1. Jeff Keyes
Director of Product Marketing
Greater Seattle Area
Marc Hornbeek
Independent Consultant
Author – Engineering DevOps
Determine a VSM Solution
2. Value Stream
Management
Webinar Series
I. Why is Value Stream Management
Essential for Effective DevOps?
II. Conducting Value Stream Management
Assessments
III. What to Look for in a VSM Solution?
IV. How to Implement VSM?
V. Operationalizing VSM within Your
Organization
3. Webinar Goals
Explain the steps of a comprehensive
process for determining a VSM
solution for your organization.
1 What you will learn
The steps to determine a VSM solution
Why following these steps is important
What is the expected outcome
Next steps
2
3
4
5
4. 1. WHY – is following steps for determining a VSM solution important?
2. HOW – are VSM solutions determined?
3. WHAT – is the expected outcome of a Value Stream Management
solution recommendation?
What you will learn:
5. Solution matched to vision and goals
1. WHY –
is following steps
for determining
a VSM solution
important?
Team alignment around future state solution
Avoid lost business opportunity costs caused
from delays
Return on Investment with solution that
optimizes labor and systems costs
Morale improved when the team is
aligned to the solution
6. 2. HOW –
are Value Stream
Management
solutions
determined?
1. Review of Inputs from VSM
Assessment
2. Future-State Value Stream Mapping
3. Tools Recommendations
4. Road-mapping DevOps
Transformation
5. Build a backlog of Themes, Epics and
User-stories
6. Estimate Return on Investment
7. Solution Recommendation Alignment
8. Next steps
Aligned Solution
Requirements
Current-State
Value-Stream
Map
Future-State
Value-Stream Map
DevOps
Transformation
Roadmap
Backlog of Themes,
Epics, User Stories
Solution
Recommendation
Alignment
Tools
Recommendations
Return on
Investment Estimate
7. To what extent is
your organization
using value stream
management
today?
Survey
Question #1
Don’t know.A
Just getting familiar with VSMB
Decided to implement VSM but not yet implemented.C
VSM is being implemented but not yet part of our production
environment.
D
VSM is part of our production environment.E
Decided not to use VSM.F
8. Review
Inputs from
Assessment
Current state
of tools
Current state of
Gaps in current
practices
Current-state
Value Stream
identifying
bottlenecks
Model
application
selected
Aligned VSM
Business
Goals
Current state of
organization
9. Goal Alignment Tool
* For a specific model application and timeframe determine goals, current state, desired
future state, Gap% and Rank.
Goal Metric Current Future Gap% Rank
Agility
Duration,
Frequency
10 2 500% 1
Stability MTTR 10 5 200% 3
Efficiency % waste 20 17 118% 6
Quality Remediations 20 5 400% 2
Security Events 5 3 167% 4
Satisfaction Referrals 35 50 142% 5
10. Stage Solution Tactics Expected Improvement Draft Rank
Plan Release Codify backlog 6 days 3
Plan download Codify download 8 days 2
Approvals Automate CAB 9 days 1
Engineer a Future State Map that attacks
the primary bottlenecks
DeployApproval
Download
& Test
Release Preparation
2.5d
0d
System
Acceptance
4d 2.5d 1d
0d 0w 0d
0.1d
Future State Value Stream Map – 2 weeks
11. When engineering
a solution for any
stage in the value
stream consider all
three dimensions:
people, process
and technology.
Changes to any of
these dimensions
may affect timings
and quality.
Backlog Dev Integration Deliver Deploy Operate
PEOPLEPROCESSTECH
L/P L/P L/P L/P L/P L/P
NVT NVT NVT NVT NVT
%C/A %C/A %C/A %C/A %C/A
12. Consider pros and cons of
available sources against
general tools selection
factors and VSM specific
tool selection factors
VSM Tool
Selection
Source of
Tools
VSM-
Specific
Selection
Factors
• Open-Source
• Freemium
• Do-It-Yourself (DIY) or
“Home-Grown”
• Enterprise License
Tools
Selection
Factors
• Open-Source
• Freemium
• Do-It-Yourself (DIY) or
“Home-Grown”
• Enterprise License
• Per Gartner
2018
14. Tool Alternative 2 Tool Alternative 3Tool Alternative 1
Tools
Comparison
Matrix
• List tool requirements
side-by-side the
characteristics of
alternative tools.
• If there are multiple tools
that are equally well
suitable then a deeper
dive or Proof-of-Concept
(POC) bake-off may be
required
Comparison Category Requirement
Weight
(W)(1-5)
Description
Score
(S) (1-5)
Description
Score
(S)(1-5)
Description
Score
(S) (1-5)
Source: Open-Source, Freemium, DIY, Enterprise
Initial Cost: the licensing and labor cost for an initial
configuration including costs internal to the organization.
Total Cost of Ownership: the licensing and labor cost to the
organization at scale, including costs internal to the
organization.
Compatibility: operating systems, ecosystems, cloud-
native, DevOps frameworks, APIs
Ease of Use: intuitive user interface, efficient controls,
efficient outputs
Administration Capabilities: installation support, diagnostic
support, built-in metrics to track performance
Functional Requirements: features and capabilities
Non-Functional Requirements: Performance, reliability,
stability, scalability
Roadmap: planned enhancements for future solution
requirements
Support: professional services for installation, proof-of-
concept, configuration, training, and bug fixes
Weighted Score W x S
15. What is your highest priority
use case for Value Stream
Management?
Survey
Question #2
Don’t know.A
Remove bottlenecks in our end-to-end value stream.B
Reduce the effort and cost of orchestrating tasks such as test
environment set-up.
C
Automate the collection of data for governance and compliance
audits.
D
Automate the collection of data needed for process or capacity
planning.
E
I don’t see a compelling use case for Value Stream Management.F
16. Preparations
• Non-disclosure Agreement
• Business goals
• Test plan
• Test lab requirements (infrastructure, tools,
licenses)
• Test data requirements
• POC schedule of events
• Vendor support requirements
• Documenting results plan.
• Communication plan
• Assure confidentiality between competing
vendors.
• Evaluation decision criterion
1. Define requirements for all
stakeholders
2. Prepare for the POC
3. Bring vendor products in-house.
4. Document everything
5. Validate results with each vendor
6. Compare results
7. Decide which tool to buy
8. Determine follow-up actions
9. Communication the decision
10. Set follow-up actions dates
Proof of Concept Bake-off
17. • Starting with the highest
priorities break the user stories
into Themes and Epics.
• For each Epic include tasks for
Definition, Implementation,
Validation, Training and
Operations.
• If there are multiple Themes
identify dependencies.
Solution
RoadMap
18. • Define and load user stories,
and Epics into a backlog
management tool (e.g. Jira,
ServiceNow, etc.)
Build the
Backlog
19. • Use tangible OpEx and CapEx
costs and savings that are
acceptable to Finance such as
labor and CapEx
• Typical 3 year perspective
Return-on-
Investment
20. • Meeting of sponsors and
stakeholders
• Seek consensus around
Solution and next steps
Solution
Recommendation
Alignment
VSM Solution
Recommendation Meeting
Current State
Solution
Requirements
Future State
Value Stream
Map
Solution
RoadMap
Solution
Recommendation
Summary
Return on
Investment
Next Steps
21. • Meeting of sponsors and
stakeholders
• Seek consensus around
Solution and next steps
Next Steps
Specific
Measurable
Responsibilities
Identified
22. What is the primary
impediment for investing in
Value Stream Management
solutions in your
environment?
Survey
Question #3
Don’t know.A
Lack of understanding of use cases that would justify investment
in VSM.
B
Unsure how to create a Return – On – Investment (ROI) business
case for VSM.
C
Higher priority problems not addressed by VSM.D
Alternative Do-It-Yourself (DIY) solutions can substitute for VSM
solutions.
E
I don’t see a compelling case for Value Stream Management
investment.
F
23. 2. WHAT –
is the expected
outcome of
determining steps
for a Value
Stream
Management
Solution?
Team and sponsors alignment
around solution and clear next
steps.
24. Reserve your copy today on
www.EngineeringDevOps.com
Book price: 6”x9” Softcover print $19.95, Downloadable eBook $9.95
Engineering DevOps is scheduled to release October 2019.
“Enterprises are brushing off the sleep of being industrial companies and
awaking as software and analytics companies utilizing Agile and DevOps
practices become central to their success. In the fray of books
operationalizing DevOps, Engineering DevOps stands apart. It alone
provides valuable engineering blueprints and a comprehensive
collection of engineering practices. The seven-step engineering
transformation blueprint is a helpful new approach for organizations
needing to transform their delivery pipelines and processes. Marc’s Nine
Pillars approach to describing DevOps is easily consumable, describing
key elements that enable teams to begin their DevOps approach
regardless of their current maturity. As a developer-turned-marketeer, I
applaud this book for its message and guidance to the industry, and
useful guidebook in the journey to improve software value streams.”—
Jeff Keyes, Product Marketing, Plutora
25. Plutora is the most complete Value Stream
Management platform
FOUNDED IN 2011
26. COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
PAGE 26
Plutora
Platform
DECISION-MAKING & ANALYTICS
MANAGEMENT & ORCHESTRATION
INTEGRATION & COMMON DATA MODEL
Value Stream Mapping
Deep Analytics &
Comparative Metrics
AI-Powered Predictive Insights
VA L U E S T R E A M M A N A G E M E N T
Plan
Code /
Build
Verify
Package &
Deploy
Configure
Manage &
Monitor
Audit & Governance Pipeline Oversight & Traceability Real-Time Collaboration
Release Management &
Pipeline Orchestration
Hybrid Environment
Management
Deployment Management &
Orchestration
Tool Integrations Normalized Data Model Converged Toolchains
27. Jeff Keyes
Director of Product Marketing
jeff.keyes@plutora.com
Marc Hornbeek
Principal Consultant - DevOps
mhornbeek@engineeringdevops.com
Q & A
Notes de l'éditeur
We’ve been here before
Goals for today?
What we learn today then?
Q: Why? Do we need steps? (why not just design it)
Q: Do you have experiences where gone right or wrong?
Q: Fallacies of the “Guru” approach
Q: How to determine?
Q: Tools recommendations is nice to say, but how do you really find a good tool?
15 after
Q: So we need to review the assessment
Q:Where do these inputs come from?
Q: What are common mistakes of these inputs?
Q: How do you know that the information is current?
Q: What kind of tool do we use to prioritize our work?
Q: How do you align the assessment with the business goals?
Q: What kind of weighting should be applied to these goals – like…where to start?
Q: What is the activity to fix a problem area?
Q: How do you create the solution of future state?
Q: Customer story where people have been surprised? Vulnerability & Security
Q: How do you identify tactics to support those improvements?
Q: How to evaluate these value stream maps?
Q: What are the kinds of tools available to you?
Sources of Tools
• Open-Source: unlimited distribution without cost with support and evolution provided by the open-source community
• Freemium: limited distribution with minimal or no cost with support and evolution provided by a commercial vendor
• Do-It-Yourself (DIY) or “Home-Grown”: tools solutions unlimited distribution within the source organization with support and evolution provided by the source company.
• Enterprise License: limited distribution under commercial license with support and evolution provided a commercial vendor
Tools Requirements factors:
•Cost: the licensing and labour cost for an initial configuration and total cost of ownership at scale, Include costs internal to the organization.
•Compatibility: operating systems, ecosystems, cloud-native, DevOps frameworks, APIs
•Ease of Use: intuitive user interface, efficient controls, efficient outputs
•Administration Capabilities: installation support, diagnostic support, build-in metrics to track performance
•Functional Requirements: features and capabilities
•Non-Functional Requirements: Performance, reliability, stability, scalability
•Roadmap: planned enhancements for future solution requirements
•Support: Professional services for installation, proof-of-concept, configuration, training, and bug fixes
Q: What are key capabilities of a VSM tool?
Q: Where would I read more about this?
Q: How do you do a thorough job of comparing tools
40 after
Q: What happens AFTER you’ve selected one or two tools?
1. Define a list of requirements that cover the priority interests of all stakeholders
Ask all stakeholders to define their top two to three highest priority tool requirements from their own point-of-view. Do not suppose you already know! By asking them you will get their buy-in to the POC process and most importantly they are more likely to accept the results. Ask them to quantify the expected benefit as much as possible. Their answers will determine POC test cases and evaluation criterion.
Business /executives: What features are the executives prepare to invest in? E.g. Reduce time to deploy, reduce wasted time setting up configurations, improve compliance audits, reduce maintenance costs, reduce security risks.
Managers: What features will improve the job of the manager? E.g. simplify work assignments, make work more predictable, easy to document, easier to train, establish best practices, etc.
Release Managers: What features will help release managers the most? E.g. easy to document the configuration options have been tests and are supported by a release.
QA staff: What features will help QA staff the most? E.g. Easy to validate all the configuration that are to be supported. Easier to orchestrate Green-Blue and A/B test strategies.
Developers: What features will help developers the most? E.g. simplify setting up a dev sandbox test environment, simplify pre-flight regression testing, being able to track configurations post integration
Operations staff: What features will help operations staff the most? E.g. reduce the effort to create and maintain configuration data. , easy of automation and orchestration.
Infra/tools staff: What features will help Infrastructure and tools staff the most? E.g. reduce the drudgery and time to respond to requests to stand-up new configurations?
Customers: what features will help customers the most? E.g. Reduced number of field failure events.
2. Prepare for the POC
Validate there is a valid non-disclosure agreement in place with each vendor.
Define the POC plan and review it with each Vendor.
Business goals
Test plan
Test lab requirements (infrastructure, tools, licenses)
Test data requirements
POC schedule of events
Vendor support requirements
Documenting results plan.
Communication plan
Explain how confidentiality will be assured between competing vendors.
Evaluation decision criterion
Ask each vendor if you missed anything in the plan that the vendor has seen before.
Define the test strategy, test cases, success criterion and importance weights that will be used to verify each tool against it goals and to compare alternatives.
Example test cases:
• Documentation ease of use
• Ease of installation
• Support services
• Ability to replicate configurations
• Ability to audit configurations
• Ability to monitor configuration changes
• Ease of creating configuration recipes
• Roles and role-based controls
• Performance and capacity for deployment of large configurations
• High availability / fault tolerance
• Ability to test configurations before deployment
• Support for Green-blue and A/B deployment strategies
• Ability to integrate with Application Release Automation tools such as ElectricFlow, Jenkins, etc.
Define a dedicated and equivalent test lab environment (systems and resources) that will be required for each tool. The lab doesn't have to be a full-blown lab; it can be a scaled-down environment but there need to be tools or ways to test performance and capacity. One way is to use simulation tools or leverage temporary Cloud resources for those portions of the tests that require large number of nodes.
If the POCs for the tools are conducted in parallel, make sure confidential information between competing vendors is protected.
Setup the POC lab in such a way that it will be available and usable for the real project if the tool is chosen. For example, it could be left in place as a training environment after the POC. This was you will have a head start on the actual implementation.
Review the test cases, weights and test lab resources with each vendor.
Procure and prepare the lab infrastructure.
3. Bring the product in-house.
Ask the vendor to assign an engineer to help with the POC. On-site is strongly preferred because it increases the focus of the effort and reduces delays.
Assign a dedicated resource to install and test the POC software for each tool. Ask the vendor to review and validate the installation before conducting tests.
4. Document everything
The documentation should not be limited to the lab and how it was configured. It should also include the steps of installation, results of tests that were performed, and final findings and recommendations.
It's important to document the POC environment so it can be easily replicated for the teams supporting your product. Well-written documentation will also ensure that other teams within the organization understand the POC steps and why certain decisions were made.
5. Validate results with each vendor
As tests are run for each tool, report the results to the tool vendor. If there are any discrepancies try to resolve them with the vendor as you go. Don’t wait till the end. Document clearly any resolutions or revisions to the plan or workarounds that were used to resolve a discrepancy.
6. Compare results
Once all tests have been completed, prepare a comparison chart and call a meeting with internal stakeholders. This will provide data to drive towards a consensus recommendation.
7. Decide which tool to buy
This is critical. Make sure the business leaders are onboard with the decision.
8. Determine follow-up actions
Prior to discussion with vendors list out what you will expect from the winning vendor going forward. Be sure to document a positive letter to both vendors and a high-level statement of the rationale for the decision. It is important to provide detailed feedback to the losing vendor regarding specific reasons they were not chosen. This must be done without disclosing confidential information about the competing vendor.
9. Communication the decision
A meeting or detailed letter should be shared with internal staff immediately following the meetings with the vendors.
A meeting with each vendor is recommended. Thank the vendor and present the letter. Go through the expected follow-up actions with the winning vendor.
10. Set follow-up actions dates
Agree with the winning vendor what the next steps schedules are.
Q: How do you implement the solution?
Q: How do you turn the roadmap into real work?
Q: How do you build a business case?
Q: What is reasonable for the business case? (15:1?)
Q: How do you get consensus on the final solution?
Q: How to move forward after the meeting?
Q: What would be some sample goals here?