Contenu connexe Similaire à Simple Measurements #2 (20) Plus de Schalk Cronjé (20) Simple Measurements #21. Simple Measurements
Schalk W. Cronjé
ysb33r @ gmail.com
@ysb33r
CT SPIN 2012 © Schalk W. Cronjé
3. “What is Agile?” - Responses
● “Continuous development done in small iterations”
● “Adapting to rapid change, predictable periodic
delivery of usable software”
● “Developers and testers work together from
beginning to end”
● “You'd only get a cynical answer from me”
CT SPIN 2012 © Schalk W. Cronjé
4. Being an Agile Fundamentalist
● Increased revenue
● Reduction in costs
● Faster response times
● Less rework
Agile is the continuous delivering of value
to stakeholders
● Customers
● Users
● Developers & Testers
● Governments & Legislators
● Societal Institutions
CT SPIN 2012 © Schalk W. Cronjé
5. "The engineering method is the use of
heuristics to cause the best change in a
poorly understood situation within the
available resources."
- Billy V Koen
CT SPIN 2012 © Schalk W. Cronjé
6. Accuracy is to precision
as
engineering is to mathematics
CT SPIN 2012 © Schalk W. Cronjé
7. Accuracy is to precision
as
engineering is to mathematics
Be accurate enough
to get there, but no more accurate
CT SPIN 2012 © Schalk W. Cronjé
8. Software Engineering
● Implies measurement
● Implies Kolb / Shewhart / Deming Cycles
● Is not code craftsmanship
● SE receives bad press
– Perceived difficulty in measurement
– Lack of measurement in S/W dev
CT SPIN 2012 © Schalk W. Cronjé
9. Simple Measurements:
A foundation of software engineering,
NOT
a tool for measuring humans
CT SPIN 2012 © Schalk W. Cronjé
10. Why are you not measuring
your process today?
CT SPIN 2012 © Schalk W. Cronjé
11. How do you know when your
delivery was successful?
CT SPIN 2012 © Schalk W. Cronjé
12. Signature Skills
● Three parts
– Preferred Task
– Preferred Cognitive Approach
– Preferred Technology
● This may affect how/what you measure!
● Combat with
– Appropriate team composition
– Reviews / quality measures
CT SPIN 2012 © Schalk W. Cronjé
13. Batch Development
Feature 1 Dev Feature 1 QA
Feature 2 Dev Feature 2 QA
Feature 3 Dev Feature 3 QA
Feature 4 Dev Feature 4 QA
delivered
build
CT SPIN 2012 © Schalk W. Cronjé
14. Batch Development
Total Time = devt1 .. devt4 + qat1 …qat4 + fixdelay + fixtime + retest time
Feature 1 Dev Feature 1 QA
Feature 2 Dev Feature 2 QA
Feature 3 Dev Feature 3 QA
Feature 4 Dev Feature 4 QA
delivered
build
defect trickle feed
Bug Feature 5+ QA
DB Retest fixes QA
delivered
build
Feature 5+ Dev
CT SPIN 2012 © Schalk W. Cronjé
15. Time Factors
Building test
understanding infrastructure
specs
Feature 1 Dev Feature 1 QA
Feature 2 Dev Feature 2 QA
Feature 3 Dev Feature 3 QA
Feature 4 Dev Feature 4 QA
basic build
verification
delivered
build
time from raising defect until it is
available for testing
defect trickle feed time to
re-test
Bug Feature 5+ QA
DB Retest fixes QA
delivered
build
Feature 5+ Dev
CT SPIN 2012 © Schalk W. Cronjé
16. Taking an Economic View
● It helps to quantify the effects of multiple interacting
variables
● It helps us to understand that the customer is not the only
judge of value
● By using an economic framework it can allow us to
maximise value, including
– cycle time
– product cost
– development expense
● It helps to communicate with non-technical decision
makers
CT SPIN 2012 © Schalk W. Cronjé
17. "If you only quantify one thing,
quantify the cost of delay."
- Don Reinertsen
CT SPIN 2012 © Schalk W. Cronjé
18. Cost of Delay – Quality Issue
● One small feature, not properly tested
● Found post-release
● Time wasting by users having to manual work
quantified => £35,000
TotalManualTimeSpent x AvgCostToCompanyPerUser +
TimeToRework x AvgCostToCompanyPerDev
CT SPIN 2012 © Schalk W. Cronjé
19. "A pilot who sees from afar
will not make his boat a wreck."
- Amen-en-apt
(Egyptian philosopher, ~700BC)
CT SPIN 2012 © Schalk W. Cronjé
20. David Anderson's Six Steps
● Focus on quality
● Reduce Work-in-Progress
● Deliver often
● Balance demand against throughput
● Prioritise
● Attack sources of variability to improve
predictability
CT SPIN 2012 © Schalk W. Cronjé
21. Business Value Increments
● Delivery is in business value increments
(BVIs)
● Only “done” when the increment is delivered
end-to-end
● No burning down of tasks ! (false economy)
● Each increment must have at least one metric
of the value it will deliver
CT SPIN 2012 © Schalk W. Cronjé
23. Visualisation
● Honesty!
● Makes it transparent to everyone that is going
on
● Visual cue for bottlenecks
● Manual board and easy-to-use online system
CT SPIN 2012 © Schalk W. Cronjé
27. Limit Work-in-Progress
R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d
C o m p le te C o m p le te C o m p l e te
● Use a pull system
● Managing queues are easier than managing time lines
● Allows for better throughput utilisation
● Identifies bottlenecks in the system
● Measuring cycle time allows better prediction than traditional
estimation
CT SPIN 2012 © Schalk W. Cronjé
28. Policies
R e a d y S p e c S p e c D e v D e v Q A Q A R e le a s e d
C o m p le te C o m p le te C o m p l e te
● Each stage has a policy
● Describes generic activities required
● “Definition of Done” for each stage
CT SPIN 2012 © Schalk W. Cronjé
29. Feedback & Batch Size
● Faster feedback makes learning faster and
more efficient
● Faster feedback provides a sense of control
● Large batches leads to slower feedback
CT SPIN 2012 © Schalk W. Cronjé
30. Measuring Cycle Time
● Measure average time in system of each BVI
● Less time is spent on estimating future delivery times
● Historic data automatically takes into account any
disruptions such as
– Operational issues
– Days off sick
AvgTimeInSystem x NoOfBVIs
Time = ---------------------------------
Concurrency
CT SPIN 2012 © Schalk W. Cronjé
32. Cadences
● Delivery is done at regular cadences.
● What is ready, is delivered, what is not, is left
out
● No artificial time-boxing or break points as in
SCRUM-like approaches
● Cycle-time leads to better predictability of what
can actually be delivered
CT SPIN 2012 © Schalk W. Cronjé
33. Case study
3 Teams – 3 Locations – 18 months
CT SPIN 2012 © Schalk W. Cronjé
34. Simple Flow Model
Specify Develop QA
Kanban
Board
Ready Spec Spec Dev Dev Q A Q A R e le a s e d
C o m p l e te C o m p le te C o m p le te
CT SPIN 2012 © Schalk W. Cronjé
35. Reduced Learning Time
● Writing test specifications after the
development is costly in time
– Knowledge decay
– Leads to incomplete designs
● Move the test specification up-front before
any development starts
– This is part of requirements discovery
– It broadens the perspective of the programmer
– Allows us to distinguish between automated checks
and human testing
CT SPIN 2012 © Schalk W. Cronjé
36. Reduced Learning Time
Specify Develop QA
Test specification
+Time to specify -Time to re-work
+Time to develop -Time to re-test
CT SPIN 2012 © Schalk W. Cronjé
37. Changed the Flow Model
Specify Develop & Check Verify
● Ensure the all automated checks are included
in the development stage
● Free up a Verification stage for exploratory
testing and validating that process has been
fulfilled.
CT SPIN 2012 © Schalk W. Cronjé
38. Real-world measurements
Specify Develop & Check Verify
S1 S2 S3 S4
READY 4 6 5 1h
SPEC 1 0.5 6 0.5
SPEC/Completed 2 0.5 1 0.5
DEV + Check 2 1 13 4
DEV + Check / 3 6 4 18
Completed
VERIFY 3 7 8 1
VERIFY/Completed 2 5 6 18
Blockages 0.5 1h 2 1h
Total 17 26 43 42
Average cycle days per feature
CT SPIN 2012 © Schalk W. Cronjé
39. Pathology
S1 S2 S3 S4
READY 4 6 5 1h
SPEC 1 0.5 6 0.5
SPEC/Completed 2 0.5 1 0.5
DEV + Check 2 1 13 4
DEV + Check / 3 6 4 18
Completed
VERIFY 3 7 8 1
VERIFY/Completed 2 5 6 18
Blockages 0.5 1h 2 1h
Total 17 26 43 42
Average cycle days per feature
CT SPIN 2012 © Schalk W. Cronjé
40. Pathology
Excessive long time between feature availability and testing. Is there enough
testing bandwidth? Or is this team just cheating?
S1 S2 S3 S4
READY 4 6 5 1h
SPEC 1 0.5 6 0.5
SPEC/Completed 2 0.5 1 0.5
DEV + Check 2 1 13 4
DEV + Check / 3 6 4 18
Completed
VERIFY 3 7 8 1
VERIFY/Completed 2 5 6 18
Blockages 0.5 1h 2 1h
Total 17 26 43 42
Average cycle days per feature
CT SPIN 2012 © Schalk W. Cronjé
41. Kanban fails when people
don’t want to face the truth.
- Hillel Glazer
CT SPIN 2012 © Schalk W. Cronjé
42. It all fell apart ...
● Restructure – 3 teams + 3 managers
● 2 managers rejected Kanban
– 1 manager wanted ScrumWorks
– 1 manager used Microsoft Project
● 1 manager was forced not to use Kanban
– Switched to iteration-based deliveries
● All teams forced back to time-based
estimation
CT SPIN 2012 © Schalk W. Cronjé
43. It fell apart even more ...
● 1 team decided on 4 wks dev + 4 wks testing
– Quality was worse than before
– Cycle time was at least 8 calendar weeks per BVI
● 1 team did not deliver for over 5 months
– Finally delivered something the user did not want
● None of teams have any measurements on
cycle time; cannot predict how long delivery
will take
CT SPIN 2012 © Schalk W. Cronjé
44. What did the team members
say?
● “I have lost visualisation. I have no idea what is going
on”
● “We are not using Kanban, because the company forced
us to SCRUM”
● “If I was given a choice, my the team and I would
definitely go with Kanban”
● “It is not the company 'standard' as such you need to
'sell' the process frequently. “
CT SPIN 2012 © Schalk W. Cronjé
46. Quantifying BVIs
● Know what you want to achieve
● Establish a scale for measurement
● Decide how to measure
● Make constraints explicit
● Communicate
CT SPIN 2012 © Schalk W. Cronjé
47. Quantify BVIs - Example
For all customer submissions that meets
the following criteria, 80% should be
handled by an automation system instead
of a human and resolution achieved within
30 minutes.
Criteria A
Criteria B
etc.
CT SPIN 2012 © Schalk W. Cronjé
48. Planguage
● Created by Tom Gilb
● Provides a structured way of quantifying
requirements
● Allows for reuse of specifications
CT SPIN 2012 © Schalk W. Cronjé
49. Quantify BVI - Planguage
Name: Customer submissions resolved by automation systems
Scale: Percentage of submissions resolved by automation in
relation to all submissions
Meter: Query on database using the following statement:
SELECT … FRO M… WHERE . . .
Goal: 80% [Criteria A, Criteria B]
CT SPIN 2012 © Schalk W. Cronjé
50. Quantified BVI - Applying the
Economic Model
● Compare to average human time to respond
and volume covered
● Extrapolate to time savings or cost savings
● This allows for two measures of success
– Technical
– Financial
CT SPIN 2012 © Schalk W. Cronjé
52. Apply Lean Principles
to your Team
Specify Develop & Check Verify
● Start from where you are; model the workflow
● Make the system pull-based
● Reduce batch size; limit the work-in-progress
● One feature end-to-end
● If the flow breaks, fix it immediately
CT SPIN 2012 © Schalk W. Cronjé
53. Apply Lean Principles
to your Team
● Pair up people with different skills to work on
same feature
– Feature teams are not a new concept
– Reduces mind-set trap, creative abrasion
● Could be perceived to have higher person
cost per feature
– Instead of distributing the person cost over
multiple queues, the cost is combined into a
single queue
– Can actually lead to reduced cycle time.
CT SPIN 2012 © Schalk W. Cronjé
54. Apply Lean Principles
Simple Measurements
● Measure cycle time (now predict delivery)
● Quantify requirements (know that you did deliver)
● Use “Cost of Delay” (prioritise, show impact/risk)
CT SPIN 2012 © Schalk W. Cronjé
Notes de l'éditeur Many of things discussed has roots in Knowledge Management and ancient philosophy, beliefs and faiths. It is not new, but are generally ignored due to the strong influence of Taylorism especially in western contexts. This is an overview of things that work, that will need adoption in your context and are simple guidelines showing you how to do things. Cynical answers are usually indicative of organisational pathology You are not here to produce software, you are here to provide value to the business Also refer to Dave Nicolette ( http://davenicolette.wordpress.com/2012/08/14/how-to-avoid-the-local-optimization-problem-when-coaching-at-the-team-level-2/ ) “The goal is delivery effectiveness” Some still preach the Agile Manifesto, Some show you methods that will work Credit to Bob Marshall for introducing me to the work of Billy Koen (Texas University) From the work of Dorothy Leonard. Read “Wellsprings of Knowledge” One always has to revisit classic batch development, as it is where most organisations are still stuck. Read Don’s book: The Principles of Product Development. FLOW. Second Generation Lean Product Development Attention Principle: Time counts more than money Use of an online system provides automated measurement. It is important not to negflect having a BIG VISIBLE BOARD From “Quotable Kanban” Eclesiastes 4:9-10 Two are better than one; because they have a good reward for their labour. For if they fall, the one will lift up his fellow: but woe to him that is alone when he falleth; for he hath not another to help him up.