Thoughts on Lean Product Development at CAMUG, YYC Nov 2014

Lean Product Management
The Value of Experiments
!
!
!
@DaveSharrock
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013

compliance international B2B MBA English IPO
agile husband start-up technology
newly-minted Canadian
executive leanstartup outsourcing father
enterprise transitions
B2C data analysis kanban seismology PhD
scrum
organizational excellence
Dave Sharrock
dave.sharrock@agile42.com
twitter: @davesharrock
Certified Scrum Coach (CSC)
Certified Scrum Trainer™ (CST)

The Sky is Falling! The Sky is Falling!
Applying the Scientific Method to
Software Development
Incorporating Learning into Product
Development
A Last Note on Shorter Feedback
Loops

“No matter how good the team or how
efficient the methodology, if we’re not
solving the right problem, the project fails.”
!
Woody Williams

Assembly Line Manufacturing
has a high cost of change
http://www.archives.gov.on.ca/english/on-line-exhibits/d-day/big/big_03_airline_assembly.aspx
Archives of Ontario, Reference Code: C 190-5-0-0-21

Inclusive thinking
• In software development,
(presumed) high cost of
change leads to long delivery
cycles
• Any and every idea has to be
captured in the first version of a
requirements specification
• This creates waste - bloated
documents, unwanted features
and entitlement thinking
Feature usage in
enterprise software
Never
Rarely
Sometimes
Often
Always
Standish Group - CHAOS Report, 1995, 2002

Cost of
Change
Hypothesized
Requirements
Capture all
possible needs
Detail of
Requirements

Software Development has
commoditized cost of change
DevOps
Object-oriented
languages
Ruby on Rails
Continuous
Delivery
jquery
Coffeescript
Automated
Tests

Cost of
Change
Hypothesized
Requirements
Capture all
possible needs
Detail of
Requirements
Evolving
Requirements
Emerging needs
as development
progresses

Usability Testing - c.2000

Usability Testing - today
http://welovelean.wordpress.com/2012/01/29/15-essential-tools-every-lean-startup-cant-li/

Changing paradigms
• If change is cheap, requirements
can change continuously
• We can evolve our thinking as
we learn more about the product
we are building
• Maximizing Return-on-
Investment requires validating
new features early, before the
total investment cost has been
spent

Cost of
Change
Hypothesized
Requirements
Capture all
possible needs
Detail of
Requirements
Validated
Requirements
Evolving
Requirements
Lean Startup
experiments
Emerging needs
as development
progresses

Designed by KISSmetrics

Theory
Use the theory
to make a
prediction
Prediction
The Scientific Method
Modify or change
your theory
Run the prediction
experiment
Experiment
Observe
Design an
experiment to
test the

The Theory of General Relativity
Einstein’s Theory of
General Relativity
Einstein’s
prediction
(1907)
Light bends in a
gravitational field
Gradual acceptance of
General Relativity over
Newtonian Mechanics
Observations validated
General Relativity
Solar Eclipse of 1919
Wait for 1919 solar
eclipse to observe
whether or not light
bends around the sun
Arthur Eddington
observed that Light did
bend around the sun
http://thethoughtstash.wordpress.com/2011/01/03/how-eddington-demonstrated-that-einstein-was-right/

Applied to an Online Experience
User Model
Use the model
to make a
prediction
Prediction
Modify or change
your user model
Run the prediction
experiment
Experiment
Validate
Design an
experiment to
test the

Five Levels of Planning
Vision
Planning
Roadmap
Planning
Release
Planning
Vision
Planning
Roadmap
Planning
ReleaItseera
P3loann
ning
Planning
Daily
Planning
Annually Define a vision
Bi-‐annually Rank features in order of
delivery
Quarterly Plan stories to be delivered
during next release cycle
Bi-‐weekly Sprint planning, focussing on
delivery of sprint goal
Daily Focus on daily planning of
team activities
5 Levels of Agile Planning: From Enterprise Product Vision to Team Stand-up, Hubert Smits (2006)

Learning
Vision
Planning
Roadmap
Planning
Release
Planning
Vision
Planning
Roadmap
Planning
ReleaItseera
P3loann
ning
Planning
Daily
Planning
Long-‐term
Define & refine behavioural
models we are testing?
Pivots happen here.
Medium-‐term
Validate understanding with
Lean (or Customer
Experience) Canvas
Regularly
Minimum Viable Product.
Validate ideas before fully
committing investment
Bi-‐weekly Product Owners, Delivery
Team
Daily Product Owners, Delivery
Team

Known Unknowns
Recognize there are gaps in your
vision/business model, and look to
fill those gaps in as quickly as
possible
Map of Africa, 1824
http://culturalstudieslectures.blogspot.com/2012/01/
lecture-one-on-question-what-is.html

“In a nutshell, the job of the product
organization is to make that vision a
reality.
So we don't work for a few weeks or even a
few months and then just declare it's not
happening and we should do a pivot.
We will typically work hard for many
months on a product vision and not even
entertain the possibility of giving up.”
!
Marty Cagan
http://www.infoq.com/news/2013/07/pivoting-product-development

Learning
Vision
Planning
Roadmap
Planning
Release
Planning
Vision
Planning
Roadmap
Planning
ReleaItseera
P3loann
ning
Planning
Daily
Planning
Long-‐term
Define & refine behavioural
models we are testing?
Pivots happen here.
Medium-‐term
Validate understanding with
User Story Mapping and
Lean Canvases
Regularly
Minimum Viable Product.
Validate ideas before fully
committing investment
Bi-‐weekly Product Owners, Delivery
Team
Daily Product Owners, Delivery
Team

User Story Mapping is a
powerful tool for understanding
your product from the
perspective of the user

Validated Learning
• Understand the problem a
feature will solve
• Decide how to ask if your
customer cares
• Validate before investing
complete cost

1.
Problem
top
3
problems
!!!!
Exis3ng
alterna3ves
Lean Canvas: Ash Maurya
4.
Solu3on
top
3
features
8.
Key
metrics
e.g.
AARRR
3.
Unique
value
proposi3on
Clear
compelling
message.
Why
makes
you
different?
2.
Customer
segments
Target
customers
!!!
Early
adopters?
9.
Unfair
advantage
Not
easily
copied
or
bought
5.
Channels
Path
to
customers
7.
Cost
Structure
Customer
acquisi3on,
distribu3on,
people,
etc.
6.
Revenue
streams
Revenue
model,
life
3me
value,
revenue,
gross
margin
Product Market
Lean
Canvas,
from
Ash
Maurya:
Running
Lean.
Licensed
under
Crea3ve
Commons-‐Aribu3on-‐Share
Alike
3.0

<Title
/
headline>
Problem
Top
3
problems
!!!!
Exis3ng
/
alterna3ve
solu3ons
Lean Experience Canvas
Customer
segments
!
Early
adopters?
Success
metrics
!
Leading
and
lagging
indicators
Possible
solu3ons
!
Key
features
&
benefits
for
customers
&
business
Business
readiness
What
is
needed
to
deploy
and
support
the
solu3on?
!
(Staff,
budget,
training,
systems,
support,
opera3ons
dependencies
on
projects/
vendors)
Channels
Path
to
customers
Costs
People,
training,
capital,
marke3ng,
etc.
+
Objec3ve
jus3fica3on
Business
value
1.
User
&
business
value
2.
Cost
of
Delay
(urgency)
3.
risk
reduc3on/opportunity
enablement
+
Objec3ve
jus3fica3on
Cost
of
delay
linear
1,2,3,5,8,13,21 1,2,3,5,8,13,21
time
delay cost
Market Product

Zappos famously launched
without stock, selling shoes from
their local specialist show store

Minimum Viable Product
A Minimum Viable Product may be an entire product or a sub-set of product (such
as a feature):
• Product - The canonical MVP strategy for a web application is to create a mock
website for the product and purchase online advertising to direct traffic to the
site. The mock website may consist of a marketing landing page with a link for
more information or purchase. The link is not connected to a purchasing system,
instead clicks are recorded and measure customer interest.
• Feature - A link to a new feature in a web application might be shown in a
prominent location on the website. The feature is not implemented, rather an
apology, mock-up, or marketing page is provided. Clicks of the link are recorded
and provide an indication as to the demand for the feature in the customer base.
It is ALWAYS smaller than you think!

How Much Effort on Validation
• In 2009, Google ran
approximately 12,000
experiments, of which about
10% led to business changes
• Linus Pauling famously said,
"The best way to have a good
idea is to have a lot of ideas."
• Focus on minimizing the
incremental cost of testing ideas
with your customers/users
Feature usage in
enterprise software
Never
Rarely
Sometimes
Often
Always
Standish Group - CHAOS Report, 1995, 2002

One test is worth a
thousand expert
opinions.
!
Wernher Von Braun

Interaction Model Backlog Validation
Execution
To Do Observation
Doing Doing Deployed
For <target group>
Who <has a specific need>
The <product name>
Is A <product category>
That <key benefit>
Unlike <primary competitors>
Our Product <additional
advantages>
Pathway
#1
Pathway
#2
•Visualize your User Interaction Model
and adjust it as you learn
•Come up with ideas for each pathway
•Prioritize experimental testing
•Spend time on measurement
infrastructure (a test without ability to
measure = an unvalidated feature)
Idea
#1.1
Validated?
Idea
#1.2
Level of Detail / Close to done
•Always revisit validation tests for size •The execution board is equivalent to
the team’s task board, but at the Idea
Card level
Idea
#1.3
Idea
#2.1
Idea
#2.2
Idea
#2.3
Idea
#2.4
Idea
#1.4
Idea
#1.6
Idea
#1.5
http://www.agile42.com/en/blog/2013/06/17/product-owner-board/

Economists
In November 2007,
economists in the
Survey of Professional
Forecasters —
examining some 45,000
economic-data series —
foresaw less than a 1-
in-500 chance of an
economic meltdown as
severe as the one that
would begin one month
later.

vs Weatherman
In 1940, the chance of
an American being
killed by lightning was
about 1 in 400,000.
Today, it’s 1 in 11
million.
http://www.nytimes.com/2012/09/09/
magazine/the-weatherman-is-not-a-moron.
html

Shifting Paradigms
B.G.|A.G.
“I know that half of
my advertising
budget is wasted,
but I’m not sure
which half”
Lord Leverhulme
http://www.marketingprofs.com/charts/2009/3161/more-smbs-use-online-media-than-traditional
http://marketing.blogs.ie.edu/archives/2011/01/trends-in-online-advertising-in-2011-in-the-united-states.php

Thank you
@davesharrock
dave.sharrock@agile42.com
Lean Product Management
The Value of Experiments
!
• Product Management is becoming data-driven
• Drive down your incremental cost of validation
• Validated requirements trump everything
• Feedback without a model means little
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright © 2007 - 2009.

Thoughts on Lean Product Development at CAMUG, YYC Nov 2014

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (14)

Similaire à Thoughts on Lean Product Development at CAMUG, YYC Nov 2014

Similaire à Thoughts on Lean Product Development at CAMUG, YYC Nov 2014 (20)

Plus de Dave Sharrock

Plus de Dave Sharrock (18)

Dernier

Dernier (20)

Thoughts on Lean Product Development at CAMUG, YYC Nov 2014