SlideShare une entreprise Scribd logo
1  sur  27
Software Quality Lab
Markus Unterauer
Consultant and trainer
Lessons Learned from
measuring software development
processes
www.software-quality-lab.com | improve your quality
At the end of this session you know …
Agenda
 Introduction to process measurement
 Experience from practise
 Lessons learned from the project
Slide 2
www.software-quality-lab.com | improve your quality
The Why? and first ideas on the How?
Introduction to process measurement
Slide 3
www.software-quality-lab.com | improve your quality
Measurement as basis for all improvements
Why measure processes
Slide 4
Define goal and
path to reach it
Discover, where we are
Identify gap, to where
we should be
Find corrective measures
Go next step Goal reached 
www.software-quality-lab.com | improve your quality
Proper metrics show the way to our goal
Why measure processes
Slide 5
Source:DchicviaFlickr,https://www.flickr.com/photos/agenciamodels/6118171675
www.software-quality-lab.com | improve your quality
Boundaries show, when to act
Why measure processes
Slide 6
Upper boundary
Lower boundary
Target value
Measured value
Time
www.software-quality-lab.com | improve your quality
Define proper metrics with G-Q-M
GQM for defining metrics
1. Goal
 What should be done for whom?
 Which aspects do we want to change? What is the context?
e.g. „Improve the quality for all new projects“
2. Question
 Which questions guide us to the measurement goal?
e.g. „Are we finding the right bugs?“
3. Metric
 How can we answer the questions using metrics?
e.g. „Ratio of bugs found per severity class“
Slide 7
Define goals
Find
questions
Answer
questions
www.software-quality-lab.com | improve your quality
Example
GQM for defining metrics
Slide 8
Metric
Question
Goal We want to improve the
quality of our products!
Are we finding
the right bugs?
Ratio of
bugs per severity
No. of bugs
found after release
Do we find the bugs
instead of the users?
Ratio of
bugs found by QA
vs. Users
www.software-quality-lab.com | improve your quality
Question: Are we finding the right bugs?
Goal: Improve quality of our products
 Ratio of bugs per severity class
 Number of bugs found after release
 Per severity class
 Ratio of recurring bugs
 Average number of bugs found per test case
 Number of bugs found by test level
 Unit tests
 Integration tests
 System tests
 User acceptance tests
 Bugs found by type
 Coding bug
 Bad requirements
 UI bug
 …
Slide 9
www.software-quality-lab.com | improve your quality
So much for the theory, let‘s turn to practise …
Experience from practise
Slide 10
www.software-quality-lab.com | improve your quality
Situation at our customer
Experience from practise
Slide 11
http://pixabay.com/en/desert-road-sand-forward-winds-609316/
www.software-quality-lab.com | improve your quality
Goals for our customer
Experience from practise
 How bad is it really?
 What should we do?
 Set up an improvement cycle
Slide 12
http://en.wikipedia.org/wiki/PDCA#/media/File:PDCA_Cycle.svg
www.software-quality-lab.com | improve your quality
First attempt …
Experience from practise
 Our customer wanted to measure …
 ~120 metrics
 in 12 process fields with 21 processes
 on 4 levels
 from 14 different sources and source system
 But …
 for more than half of all metrics we could not get values
 no one really knew, what to do with the metrics
 management did not even look at it
 effort for getting numbers was extremly high
Slide 13
www.software-quality-lab.com | improve your quality
Some sources of values
Experience from practise
Slide 14
• Different levels of detail
• Errors and missing data
• Bookings on different categories
Time recording
• Each project with different structure
• Not comparable
• Incomplete
Project
planning
• Only for parts of all sourcecode
• Difficult to query and analyze
Source
Control
• Manually built Excel sheets by Project Managers
• Each project with different structure
• Not comparable
Controlling
sheets
• Very high effort
• Ambiguous questions
Questionaires
and interviews
Good at first glance,
but bad in detail …
www.software-quality-lab.com | improve your quality
Everything very difficult
Experience from practise
Slide 15
http://pixabay.com/en/squirrel-reading-books-surprise-304021/
www.software-quality-lab.com | improve your quality
New approach GQM
Experience from practise
1. Define two goals
 Ask management, what they would like to achieve
 Define reporting
2. Ask questions
3. Find metrics to answer question
 Only automatically measurable ones
 Only one per question
4. Define target values and boundaries
 Get target values from industry benchmarks
5. Try out and validate
6. Report to management
Slide 16
http://pixabay.com/en/shield-directory-panic-fear-trust-492992/
www.software-quality-lab.com | improve your quality
Step 1: Define Goals
Experience from practise
1. We want to improve quality to the customer
2. We want to increase developer productivity
Slide 17
Images from www.pixabay.com
www.software-quality-lab.com | improve your quality
Step 2: Ask questions
Experience from practise
1. We want to improve quality to the customer
 Do we find bugs instead of the customer?
 Are we testing enough?
 Are we testing effectively?
2. We want to increase developer productivity
 Do our developers spend too much time on bug fixing?
 Are we developing fast enough?
 Is the quality of our source code high enough?
Slide 18
Images from www.pixabay.com
www.software-quality-lab.com | improve your quality
Step 3: Find metrics
Experience from practise
1. We want to improve quality to the customer
 Do we find bugs instead of the customer?
• Ratio of bugs found by us to bugs found by customer
 Are we testing enough?
• Number of test cases tested per developer day
 Are we testing effectively?
• Number of bugs found per tester day
2. We want to increase developer productivity
 Do our developers spend too much time on bug fixing?
• Effort-ratio for bug fixing per developer month
 Are we developing fast enough?
• Number of changed LoC per developer day
 Is the quality of our source code high enough?
• Number of bugs found by customer per developer month
Slide 19
Images from www.pixabay.com
www.software-quality-lab.com | improve your quality
Step 4: Define target values and boundaries
Experience from practise
Slide 20
Metric Target Value Current value Conclusion
Changed LoC per
developer day
30 – 40 LoC per day 123 LoC per Day Too high. Results in
poor quality.
Effort-ratio for bug
fixing
25 - 35% 115% Far too high. Low
productivity.
Ratio of bugs found
by customer
1 – 3% of all bugs 68% of all bugs Far too high. Testing
not effective.
… … … …
Target range taken
from literature and
publications
www.software-quality-lab.com | improve your quality
Step 5: Try out and validate
Experience from practise
Slide 21
http://pixabay.com/en/calculator-calculation-insurance-385506/
www.software-quality-lab.com | improve your quality
Step 6: Reporting
Experience from practise
Slide 22
Trend for past thee
months
Textual comment
from Quality
Manager
www.software-quality-lab.com | improve your quality
Document metrics in a metric handbook
Experience from practise
Slide 23
Metric Effort-ratio of bug fixing per developer month
Goal Developers should produce high quality code and spend as much time
for implementing new features as possible
Benefit Fast development of new features
High quality
Formula Ratiobug fixing = Effortbug fixing / Total Effortdevelopment
Method Automatic calculation from timesheets
Datasource Time recording system
Interval Monthly
Responsible Process Manager
www.software-quality-lab.com | improve your quality
From practise
Lessons Learned
Slide 24
www.software-quality-lab.com | improve your quality
Tips for starting your quality metric system
Lessons Learned
 Define metrics top down, not bottom up
 Goal > Question > Metric
 Not: „Hey, look how many things we can measure!“
 Concentrate on 5 - 10 metrics, e.g. the one presented before
 Define lower and upper boundary together with your team
 … or measure without boundaries for some iterations and define boundaries then
 Start with automatically measurable metrics
 When defining metrics, always have in mind, if you have all data necessary in the quality needed
 Define in advance, what to do with the metrics
Slide 25
Image:http://pixabay.com/en/bulb-light-lamp-electric-160207/
www.software-quality-lab.com | improve your quality
Questions?
Lessons Learned from measuring software development processes
Slide 26
Your Partner in Software Quality and Testing
Software Quality Lab GmbH
www.software-quality-lab.com
Consulting | Service | Academy | Tool Expertise
Markus Unterauer
markus.unterauer@software-quality-lab.com

Contenu connexe

Tendances

www.tutorialsbook.com presents Manual testing
www.tutorialsbook.com presents Manual testingwww.tutorialsbook.com presents Manual testing
www.tutorialsbook.com presents Manual testingTutorials Book
 
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinarsoftware testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI WebinarXBOSoft
 
Software Testing Interview Q&A – part 1
Software Testing Interview Q&A – part 1Software Testing Interview Q&A – part 1
Software Testing Interview Q&A – part 1Khoa Bui
 
How to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at OnceHow to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at OnceCentercode
 
Otto Vinter - Analysing Your Defect Data for Improvement Potential
Otto Vinter - Analysing Your Defect Data for Improvement PotentialOtto Vinter - Analysing Your Defect Data for Improvement Potential
Otto Vinter - Analysing Your Defect Data for Improvement PotentialTEST Huddle
 
InCycle Software presents: Quality enablement using agile practices with TFS ...
InCycle Software presents: Quality enablement using agile practices with TFS ...InCycle Software presents: Quality enablement using agile practices with TFS ...
InCycle Software presents: Quality enablement using agile practices with TFS ...InCycle Software
 
Orientation Program on Automated Software testing Powered by Infaum Education...
Orientation Program on Automated Software testing Powered by Infaum Education...Orientation Program on Automated Software testing Powered by Infaum Education...
Orientation Program on Automated Software testing Powered by Infaum Education...Anju ML
 
Software Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour Presentation
Software Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour PresentationSoftware Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour Presentation
Software Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour PresentationXBOSoft
 
Testing Attributes
Testing AttributesTesting Attributes
Testing AttributesAbiha Naqvi
 
Qfactory - A software testing service
 Qfactory - A software testing service Qfactory - A software testing service
Qfactory - A software testing serviceAbout the Code Inc.
 
Risk Mitigation Using Exploratory and Technical Testing | QASymphony Webinar
Risk Mitigation Using Exploratory and Technical Testing | QASymphony WebinarRisk Mitigation Using Exploratory and Technical Testing | QASymphony Webinar
Risk Mitigation Using Exploratory and Technical Testing | QASymphony WebinarQASymphony
 
The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy Brian Estep
 
Software testing interview Q&A – Part 2
Software testing interview Q&A – Part 2Software testing interview Q&A – Part 2
Software testing interview Q&A – Part 2Khoa Bui
 
Enhancing Software Quality
Enhancing Software QualityEnhancing Software Quality
Enhancing Software QualityAnand Prabhala
 

Tendances (19)

QAAgility Trainings
QAAgility TrainingsQAAgility Trainings
QAAgility Trainings
 
www.tutorialsbook.com presents Manual testing
www.tutorialsbook.com presents Manual testingwww.tutorialsbook.com presents Manual testing
www.tutorialsbook.com presents Manual testing
 
Agile Testing
Agile Testing Agile Testing
Agile Testing
 
CAREER ROADMAP FOR GREEN BELTS
CAREER ROADMAP FOR GREEN BELTSCAREER ROADMAP FOR GREEN BELTS
CAREER ROADMAP FOR GREEN BELTS
 
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinarsoftware testing metrics do's - don'ts-XBOSoft-QAI Webinar
software testing metrics do's - don'ts-XBOSoft-QAI Webinar
 
Software Testing Interview Q&A – part 1
Software Testing Interview Q&A – part 1Software Testing Interview Q&A – part 1
Software Testing Interview Q&A – part 1
 
How to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at OnceHow to Juggle Multiple Beta Tests at Once
How to Juggle Multiple Beta Tests at Once
 
Otto Vinter - Analysing Your Defect Data for Improvement Potential
Otto Vinter - Analysing Your Defect Data for Improvement PotentialOtto Vinter - Analysing Your Defect Data for Improvement Potential
Otto Vinter - Analysing Your Defect Data for Improvement Potential
 
InCycle Software presents: Quality enablement using agile practices with TFS ...
InCycle Software presents: Quality enablement using agile practices with TFS ...InCycle Software presents: Quality enablement using agile practices with TFS ...
InCycle Software presents: Quality enablement using agile practices with TFS ...
 
Orientation Program on Automated Software testing Powered by Infaum Education...
Orientation Program on Automated Software testing Powered by Infaum Education...Orientation Program on Automated Software testing Powered by Infaum Education...
Orientation Program on Automated Software testing Powered by Infaum Education...
 
Software Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour Presentation
Software Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour PresentationSoftware Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour Presentation
Software Quality Metrics Do's and Don'ts - QAI-Quest 1 Hour Presentation
 
Testing Attributes
Testing AttributesTesting Attributes
Testing Attributes
 
Qfactory - A software testing service
 Qfactory - A software testing service Qfactory - A software testing service
Qfactory - A software testing service
 
Req2014_Fall-Final
Req2014_Fall-FinalReq2014_Fall-Final
Req2014_Fall-Final
 
Risk Mitigation Using Exploratory and Technical Testing | QASymphony Webinar
Risk Mitigation Using Exploratory and Technical Testing | QASymphony WebinarRisk Mitigation Using Exploratory and Technical Testing | QASymphony Webinar
Risk Mitigation Using Exploratory and Technical Testing | QASymphony Webinar
 
The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy The Three Pillars Approach to Your Agile Test Strategy
The Three Pillars Approach to Your Agile Test Strategy
 
Software testing interview Q&A – Part 2
Software testing interview Q&A – Part 2Software testing interview Q&A – Part 2
Software testing interview Q&A – Part 2
 
Enhancing Software Quality
Enhancing Software QualityEnhancing Software Quality
Enhancing Software Quality
 
Future of QA
Future of QAFuture of QA
Future of QA
 

Similaire à Lessons learned from measuring software development processes

You cant control what you cant measure - Measuring requirements quality
You cant control what you cant measure - Measuring requirements qualityYou cant control what you cant measure - Measuring requirements quality
You cant control what you cant measure - Measuring requirements qualityMarkus Unterauer
 
Setting the right goals and objectives for your beta test
Setting the right goals and objectives for your beta testSetting the right goals and objectives for your beta test
Setting the right goals and objectives for your beta testCentercode
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Fabricio Epaminondas
 
Project Management Tips to Improve Test Planning
Project Management Tips to Improve Test PlanningProject Management Tips to Improve Test Planning
Project Management Tips to Improve Test PlanningTechWell
 
Beyond Simple A/B testing
Beyond Simple A/B testingBeyond Simple A/B testing
Beyond Simple A/B testingRatio
 
Start with Quality - an Agile Tester's Case Study
Start with Quality - an Agile Tester's Case StudyStart with Quality - an Agile Tester's Case Study
Start with Quality - an Agile Tester's Case StudyNick Zdunić
 
Introduction to Software Review
Introduction to Software ReviewIntroduction to Software Review
Introduction to Software ReviewPhilip Johnson
 
Measuring roi of training ppt slides
Measuring roi of training ppt slidesMeasuring roi of training ppt slides
Measuring roi of training ppt slidesYodhia Antariksa
 
Testing Metrics: Project, Product, Process
Testing Metrics: Project, Product, ProcessTesting Metrics: Project, Product, Process
Testing Metrics: Project, Product, ProcessTechWell
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...Seapine Software
 
Creating Irresistible Products in 4 Steps by Google Product Leader
Creating Irresistible Products in 4 Steps by Google Product LeaderCreating Irresistible Products in 4 Steps by Google Product Leader
Creating Irresistible Products in 4 Steps by Google Product LeaderProduct School
 
Assessing Your Agility: Introducing the Comparative Agility Assessment
Assessing Your Agility: Introducing the Comparative Agility AssessmentAssessing Your Agility: Introducing the Comparative Agility Assessment
Assessing Your Agility: Introducing the Comparative Agility AssessmentMike Cohn
 
4S Akademi - QAI - Software Testing Principles and Concepts
4S Akademi - QAI - Software Testing Principles and Concepts4S Akademi - QAI - Software Testing Principles and Concepts
4S Akademi - QAI - Software Testing Principles and ConceptsSerdar Zeybek
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The MetricsTeamQualityPro
 
Day 2 meet shilpa - measuring software quality-are you up-to-date on what an...
Day 2 meet shilpa  - measuring software quality-are you up-to-date on what an...Day 2 meet shilpa  - measuring software quality-are you up-to-date on what an...
Day 2 meet shilpa - measuring software quality-are you up-to-date on what an...XBOSoft
 
Software Testing Process & Trend
Software Testing Process & TrendSoftware Testing Process & Trend
Software Testing Process & TrendKMS Technology
 
Data-Driven Product Management by Shutterfly Director of Product
Data-Driven Product Management by Shutterfly Director of ProductData-Driven Product Management by Shutterfly Director of Product
Data-Driven Product Management by Shutterfly Director of ProductProduct School
 

Similaire à Lessons learned from measuring software development processes (20)

You cant control what you cant measure - Measuring requirements quality
You cant control what you cant measure - Measuring requirements qualityYou cant control what you cant measure - Measuring requirements quality
You cant control what you cant measure - Measuring requirements quality
 
Setting the right goals and objectives for your beta test
Setting the right goals and objectives for your beta testSetting the right goals and objectives for your beta test
Setting the right goals and objectives for your beta test
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
 
Project Management Tips to Improve Test Planning
Project Management Tips to Improve Test PlanningProject Management Tips to Improve Test Planning
Project Management Tips to Improve Test Planning
 
Beyond Simple A/B testing
Beyond Simple A/B testingBeyond Simple A/B testing
Beyond Simple A/B testing
 
System Development Life Cycle (SDLC) - Part I
System Development Life Cycle (SDLC)  - Part ISystem Development Life Cycle (SDLC)  - Part I
System Development Life Cycle (SDLC) - Part I
 
Start with Quality - an Agile Tester's Case Study
Start with Quality - an Agile Tester's Case StudyStart with Quality - an Agile Tester's Case Study
Start with Quality - an Agile Tester's Case Study
 
Introduction to Software Review
Introduction to Software ReviewIntroduction to Software Review
Introduction to Software Review
 
Measuring roi of training ppt slides
Measuring roi of training ppt slidesMeasuring roi of training ppt slides
Measuring roi of training ppt slides
 
Measuring roi of training
Measuring roi of trainingMeasuring roi of training
Measuring roi of training
 
Testing Metrics: Project, Product, Process
Testing Metrics: Project, Product, ProcessTesting Metrics: Project, Product, Process
Testing Metrics: Project, Product, Process
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 
Creating Irresistible Products in 4 Steps by Google Product Leader
Creating Irresistible Products in 4 Steps by Google Product LeaderCreating Irresistible Products in 4 Steps by Google Product Leader
Creating Irresistible Products in 4 Steps by Google Product Leader
 
Assessing Your Agility: Introducing the Comparative Agility Assessment
Assessing Your Agility: Introducing the Comparative Agility AssessmentAssessing Your Agility: Introducing the Comparative Agility Assessment
Assessing Your Agility: Introducing the Comparative Agility Assessment
 
4S Akademi - QAI - Software Testing Principles and Concepts
4S Akademi - QAI - Software Testing Principles and Concepts4S Akademi - QAI - Software Testing Principles and Concepts
4S Akademi - QAI - Software Testing Principles and Concepts
 
The Good, The Bad, and The Metrics
 The Good, The Bad, and The Metrics The Good, The Bad, and The Metrics
The Good, The Bad, and The Metrics
 
Day 2 meet shilpa - measuring software quality-are you up-to-date on what an...
Day 2 meet shilpa  - measuring software quality-are you up-to-date on what an...Day 2 meet shilpa  - measuring software quality-are you up-to-date on what an...
Day 2 meet shilpa - measuring software quality-are you up-to-date on what an...
 
Software Testing Process & Trend
Software Testing Process & TrendSoftware Testing Process & Trend
Software Testing Process & Trend
 
Data-Driven Product Management by Shutterfly Director of Product
Data-Driven Product Management by Shutterfly Director of ProductData-Driven Product Management by Shutterfly Director of Product
Data-Driven Product Management by Shutterfly Director of Product
 
Sanitized tb swstmppp1516july
Sanitized tb swstmppp1516julySanitized tb swstmppp1516july
Sanitized tb swstmppp1516july
 

Plus de Markus Unterauer

Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...
Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...
Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...Markus Unterauer
 
Notfallvorsorge in der Stadt - Staatliche Notfallvorsorge in Österreich
Notfallvorsorge in der Stadt - Staatliche Notfallvorsorge in ÖsterreichNotfallvorsorge in der Stadt - Staatliche Notfallvorsorge in Österreich
Notfallvorsorge in der Stadt - Staatliche Notfallvorsorge in ÖsterreichMarkus Unterauer
 
Man kann nicht nicht planen
Man kann nicht nicht planenMan kann nicht nicht planen
Man kann nicht nicht planenMarkus Unterauer
 
Was machen unsere Anwender so? Prozessmodellierung mit BPMN
Was machen unsere Anwender so? Prozessmodellierung mit BPMNWas machen unsere Anwender so? Prozessmodellierung mit BPMN
Was machen unsere Anwender so? Prozessmodellierung mit BPMNMarkus Unterauer
 
"Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ...
"Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ..."Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ...
"Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ...Markus Unterauer
 
Tipps für Requirements Management Tools
Tipps für Requirements Management ToolsTipps für Requirements Management Tools
Tipps für Requirements Management ToolsMarkus Unterauer
 
Erfolgsfaktoren im Requirements Engineering
Erfolgsfaktoren im Requirements EngineeringErfolgsfaktoren im Requirements Engineering
Erfolgsfaktoren im Requirements EngineeringMarkus Unterauer
 
Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...
Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...
Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...Markus Unterauer
 
Traceability von Software Anforderungen
Traceability von Software AnforderungenTraceability von Software Anforderungen
Traceability von Software AnforderungenMarkus Unterauer
 

Plus de Markus Unterauer (12)

Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...
Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...
Rückwärts denken vorwärts handeln - Requirements Reverse Engineering bei Syst...
 
Notfallvorsorge in der Stadt - Staatliche Notfallvorsorge in Österreich
Notfallvorsorge in der Stadt - Staatliche Notfallvorsorge in ÖsterreichNotfallvorsorge in der Stadt - Staatliche Notfallvorsorge in Österreich
Notfallvorsorge in der Stadt - Staatliche Notfallvorsorge in Österreich
 
Specification by example
Specification by exampleSpecification by example
Specification by example
 
Definition of Ready
Definition of ReadyDefinition of Ready
Definition of Ready
 
Man kann nicht nicht planen
Man kann nicht nicht planenMan kann nicht nicht planen
Man kann nicht nicht planen
 
Was machen unsere Anwender so? Prozessmodellierung mit BPMN
Was machen unsere Anwender so? Prozessmodellierung mit BPMNWas machen unsere Anwender so? Prozessmodellierung mit BPMN
Was machen unsere Anwender so? Prozessmodellierung mit BPMN
 
Risikobasiertes testen
Risikobasiertes testenRisikobasiertes testen
Risikobasiertes testen
 
"Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ...
"Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ..."Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ...
"Das Reiten eines hässlichen Pferdes ist verboten" - Gesetze aus Sicht eines ...
 
Tipps für Requirements Management Tools
Tipps für Requirements Management ToolsTipps für Requirements Management Tools
Tipps für Requirements Management Tools
 
Erfolgsfaktoren im Requirements Engineering
Erfolgsfaktoren im Requirements EngineeringErfolgsfaktoren im Requirements Engineering
Erfolgsfaktoren im Requirements Engineering
 
Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...
Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...
Software Quality Lab - Beratung und Training für mehr Qualität und Effizienz ...
 
Traceability von Software Anforderungen
Traceability von Software AnforderungenTraceability von Software Anforderungen
Traceability von Software Anforderungen
 

Dernier

Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 

Dernier (20)

Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 

Lessons learned from measuring software development processes

  • 1. Software Quality Lab Markus Unterauer Consultant and trainer Lessons Learned from measuring software development processes
  • 2. www.software-quality-lab.com | improve your quality At the end of this session you know … Agenda  Introduction to process measurement  Experience from practise  Lessons learned from the project Slide 2
  • 3. www.software-quality-lab.com | improve your quality The Why? and first ideas on the How? Introduction to process measurement Slide 3
  • 4. www.software-quality-lab.com | improve your quality Measurement as basis for all improvements Why measure processes Slide 4 Define goal and path to reach it Discover, where we are Identify gap, to where we should be Find corrective measures Go next step Goal reached 
  • 5. www.software-quality-lab.com | improve your quality Proper metrics show the way to our goal Why measure processes Slide 5 Source:DchicviaFlickr,https://www.flickr.com/photos/agenciamodels/6118171675
  • 6. www.software-quality-lab.com | improve your quality Boundaries show, when to act Why measure processes Slide 6 Upper boundary Lower boundary Target value Measured value Time
  • 7. www.software-quality-lab.com | improve your quality Define proper metrics with G-Q-M GQM for defining metrics 1. Goal  What should be done for whom?  Which aspects do we want to change? What is the context? e.g. „Improve the quality for all new projects“ 2. Question  Which questions guide us to the measurement goal? e.g. „Are we finding the right bugs?“ 3. Metric  How can we answer the questions using metrics? e.g. „Ratio of bugs found per severity class“ Slide 7 Define goals Find questions Answer questions
  • 8. www.software-quality-lab.com | improve your quality Example GQM for defining metrics Slide 8 Metric Question Goal We want to improve the quality of our products! Are we finding the right bugs? Ratio of bugs per severity No. of bugs found after release Do we find the bugs instead of the users? Ratio of bugs found by QA vs. Users
  • 9. www.software-quality-lab.com | improve your quality Question: Are we finding the right bugs? Goal: Improve quality of our products  Ratio of bugs per severity class  Number of bugs found after release  Per severity class  Ratio of recurring bugs  Average number of bugs found per test case  Number of bugs found by test level  Unit tests  Integration tests  System tests  User acceptance tests  Bugs found by type  Coding bug  Bad requirements  UI bug  … Slide 9
  • 10. www.software-quality-lab.com | improve your quality So much for the theory, let‘s turn to practise … Experience from practise Slide 10
  • 11. www.software-quality-lab.com | improve your quality Situation at our customer Experience from practise Slide 11 http://pixabay.com/en/desert-road-sand-forward-winds-609316/
  • 12. www.software-quality-lab.com | improve your quality Goals for our customer Experience from practise  How bad is it really?  What should we do?  Set up an improvement cycle Slide 12 http://en.wikipedia.org/wiki/PDCA#/media/File:PDCA_Cycle.svg
  • 13. www.software-quality-lab.com | improve your quality First attempt … Experience from practise  Our customer wanted to measure …  ~120 metrics  in 12 process fields with 21 processes  on 4 levels  from 14 different sources and source system  But …  for more than half of all metrics we could not get values  no one really knew, what to do with the metrics  management did not even look at it  effort for getting numbers was extremly high Slide 13
  • 14. www.software-quality-lab.com | improve your quality Some sources of values Experience from practise Slide 14 • Different levels of detail • Errors and missing data • Bookings on different categories Time recording • Each project with different structure • Not comparable • Incomplete Project planning • Only for parts of all sourcecode • Difficult to query and analyze Source Control • Manually built Excel sheets by Project Managers • Each project with different structure • Not comparable Controlling sheets • Very high effort • Ambiguous questions Questionaires and interviews Good at first glance, but bad in detail …
  • 15. www.software-quality-lab.com | improve your quality Everything very difficult Experience from practise Slide 15 http://pixabay.com/en/squirrel-reading-books-surprise-304021/
  • 16. www.software-quality-lab.com | improve your quality New approach GQM Experience from practise 1. Define two goals  Ask management, what they would like to achieve  Define reporting 2. Ask questions 3. Find metrics to answer question  Only automatically measurable ones  Only one per question 4. Define target values and boundaries  Get target values from industry benchmarks 5. Try out and validate 6. Report to management Slide 16 http://pixabay.com/en/shield-directory-panic-fear-trust-492992/
  • 17. www.software-quality-lab.com | improve your quality Step 1: Define Goals Experience from practise 1. We want to improve quality to the customer 2. We want to increase developer productivity Slide 17 Images from www.pixabay.com
  • 18. www.software-quality-lab.com | improve your quality Step 2: Ask questions Experience from practise 1. We want to improve quality to the customer  Do we find bugs instead of the customer?  Are we testing enough?  Are we testing effectively? 2. We want to increase developer productivity  Do our developers spend too much time on bug fixing?  Are we developing fast enough?  Is the quality of our source code high enough? Slide 18 Images from www.pixabay.com
  • 19. www.software-quality-lab.com | improve your quality Step 3: Find metrics Experience from practise 1. We want to improve quality to the customer  Do we find bugs instead of the customer? • Ratio of bugs found by us to bugs found by customer  Are we testing enough? • Number of test cases tested per developer day  Are we testing effectively? • Number of bugs found per tester day 2. We want to increase developer productivity  Do our developers spend too much time on bug fixing? • Effort-ratio for bug fixing per developer month  Are we developing fast enough? • Number of changed LoC per developer day  Is the quality of our source code high enough? • Number of bugs found by customer per developer month Slide 19 Images from www.pixabay.com
  • 20. www.software-quality-lab.com | improve your quality Step 4: Define target values and boundaries Experience from practise Slide 20 Metric Target Value Current value Conclusion Changed LoC per developer day 30 – 40 LoC per day 123 LoC per Day Too high. Results in poor quality. Effort-ratio for bug fixing 25 - 35% 115% Far too high. Low productivity. Ratio of bugs found by customer 1 – 3% of all bugs 68% of all bugs Far too high. Testing not effective. … … … … Target range taken from literature and publications
  • 21. www.software-quality-lab.com | improve your quality Step 5: Try out and validate Experience from practise Slide 21 http://pixabay.com/en/calculator-calculation-insurance-385506/
  • 22. www.software-quality-lab.com | improve your quality Step 6: Reporting Experience from practise Slide 22 Trend for past thee months Textual comment from Quality Manager
  • 23. www.software-quality-lab.com | improve your quality Document metrics in a metric handbook Experience from practise Slide 23 Metric Effort-ratio of bug fixing per developer month Goal Developers should produce high quality code and spend as much time for implementing new features as possible Benefit Fast development of new features High quality Formula Ratiobug fixing = Effortbug fixing / Total Effortdevelopment Method Automatic calculation from timesheets Datasource Time recording system Interval Monthly Responsible Process Manager
  • 24. www.software-quality-lab.com | improve your quality From practise Lessons Learned Slide 24
  • 25. www.software-quality-lab.com | improve your quality Tips for starting your quality metric system Lessons Learned  Define metrics top down, not bottom up  Goal > Question > Metric  Not: „Hey, look how many things we can measure!“  Concentrate on 5 - 10 metrics, e.g. the one presented before  Define lower and upper boundary together with your team  … or measure without boundaries for some iterations and define boundaries then  Start with automatically measurable metrics  When defining metrics, always have in mind, if you have all data necessary in the quality needed  Define in advance, what to do with the metrics Slide 25 Image:http://pixabay.com/en/bulb-light-lamp-electric-160207/
  • 26. www.software-quality-lab.com | improve your quality Questions? Lessons Learned from measuring software development processes Slide 26
  • 27. Your Partner in Software Quality and Testing Software Quality Lab GmbH www.software-quality-lab.com Consulting | Service | Academy | Tool Expertise Markus Unterauer markus.unterauer@software-quality-lab.com

Notes de l'éditeur

  1. My name is Markus Unterauer I come from Software quality lab in austria Out mission is software quality Quality for us does not start at testing, but with the first line of business goals and requirements This vision of software quality was also the starting point of this talk
  2. Let‘s have a look at what you can expect today And what you should take home …
  3. Why is requirements quality that important? QUESTION TO THE AUDIENCE Who of you has problems with poor or bad requirements? With misunderstandings? Let‘s take a closer look, to how practical work in the field of requirements quality really looks like today …
  4. ANIMATION Every improvement starts with defining the goal … At „Discover where we are“ and „Identify gap“ we need to measure
  5. ANIMATION At first I have to decide what to measure A metric must fulfill some criteria It must represent the current situation right It must allow me to derive measures
  6. For each metric we have to define an ideal value Around this ideal value, there is a range, where things are ok If our metrics show, that we leave the green zone, we need to act
  7. One approach to find the proper metrics to measure requirements quality is the GQM approach …
  8. Here we have another example for GQM in requirements quality measurement
  9. We know now Requirements are important Often they are done bad That leads defects, high costs and missed expectations So, we need better requirements
  10. At the moment it is ok, but first dark clouds on horizon Storm is coming First signs are already visible They tried to start measiring so that they get info, where tehy sould go
  11. without us ;-)
  12. For each metric, we took a look in advance, whether we have the data!
  13. Perform measurement, get values Reports from existing systems LoC -> Source-Control Bug numbers -> Bugtracking tool
  14. We know now Requirements are important Often they are done bad That leads defects, high costs and missed expectations So, we need better requirements