Contenu connexe Similaire à Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP (20) Plus de Luigi Buglione (20) Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP1. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
1
Improving Measurement Plans from multiple dimensions:
Exercising with Balancing Multiple Dimensions (BMP)
11stst
Workshop onWorkshop on «« MethodsMethods forfor Learning MetricsLearning Metrics »»
19 September 2005, Como (Italy)19 September 2005, Como (Italy)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE – MONTRÉAL - CANADA
Luigi Buglione &Luigi Buglione & AlainAlain AbranAbran
2. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
2
• Introduction
• Multidimensional Analysis in Project Management
ü Why is it needed?
ü Some Multidimensional Models
ü What should be measured and analysed?
ü Which set of indicators to select?
• BMP: Balancing Multiple Perspectives
ü The proposed measurement procedure
ü An example using 4 dimensions
ü Measuring project performances from multiple views
• Applying BMP
ü BMP-Q: the questionnaire
ü BMP-Q: the metrics
ü First Results & Feedbacks
• Conclusions & Prospects
Agenda
3. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
3
• Introduction
• Multidimensional Analysis in Project Management
ü Why is it needed?
ü Some Multidimensional Models
ü What should be measured and analysed?
ü Which set of indicators to select?
• BMP: Balancing Multiple Perspectives
ü The proposed measurement procedure
ü An example using 4 dimensions
ü Measuring project performances from multiple views
• Applying BMP
ü BMP-Q: the questionnaire
ü BMP-Q: the metrics
ü First Results & Feedbacks
• Conclusions & Prospects
Agenda
4. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
4
Introduction
• Growing interest in Integrated Software
Measurement…
ü E.g. BSC, EFQM, MBQA
• …but still few documented industrial implementations
ü Usually Time and Cost dimensions used
• Other possible dimensions of analysis (eg: Quality,
Risk, …) are not often taken into account
ü Q: how much does it cost project monitoring & control?
ü Q: how many measures/indicators are usually tracked during
the project lifecycle? And from which perspectives?
• Objective:
• to optimize the cost for tracking & control projects, balancing
the number of measures/indicators used by each perspective
of analysis useful to the project
5. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
5
• Introduction
• Multidimensional Analysis in Project Management
ü Why is it needed?
ü Some Multidimensional Models
ü What should be measured and analysed?
ü Which set of indicators to select?
• BMP: Balancing Multiple Perspectives
ü The proposed measurement procedure
ü An example using 4 dimensions
ü Measuring project performances from multiple views
• Applying BMP
ü BMP-Q: the questionnaire
ü BMP-Q: the metrics
ü First Results & Feedbacks
• Conclusions & Prospects
Agenda
6. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
6
Multidimensional Analysis in PM
Why is it needed?
• Loss of project control is one of the most frequent
causes of failure in Project Management
ü Prevention: detailed analysis of content and quality of project
tracking
• Some basic questions:
ü Right number of perspectives?
ü Right number of indicators?
ü What about hypothesis of relationships among processes?
At least, 3 dimensions:
ü Management
ü Users
ü Technical
7. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
7
Multidimensional Analysis in PM
Some examples
8. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
8
Multidimensional Analysis in PM
What should be measured and analysed?
STARSTAR taxonomy: a broader view on Measurement & Analysis
9. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
9
Multidimensional Analysis in PM
Which set of indicators to select?
• Q: What is the right number of indicators to use?
– The Miller’s “magic number” 7 ± 2?
• General suggestions to avoid the misbalance in
selecting the measures critical to success, whatever
the number
Ø“Select a small suite of key measures that will help
you to understand your group’s work better, and
begin collecting them right away, measuring
several complementary aspects of your work, such
as quality, complexity, and schedule.”
(Karl Wiegers)
10. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
10
• Introduction
• Multidimensional Analysis in Project Management
ü Why is it needed?
ü Some Multidimensional Models
ü What should be measured and analysed?
ü Which set of indicators to select?
• BMP: Balancing Multiple Perspectives
ü The proposed measurement procedure
ü An example using 4 dimensions
ü Measuring project performances from multiple views
• Applying BMP
ü BMP-Q: the questionnaire
ü BMP-Q: the metrics
ü First Results & Feedbacks
• Conclusions & Prospects
Agenda
11. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
11
BMP: Balancing Multiple Perspectives
General issue
• Q: how can a proper balance of perspectives and
indicators be selected when managing a portfolio of
projects?
PEANUTS © United Feature Syndicate, Inc.
The problem is not to reduce the cost of measurement,
but optimising it against the informative value provided by the number
of measures/indicators balancing them by each perspective of analysis
12. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
12
BMP: Balancing Multiple Perspectives
The proposed Measurement Procedure
Œ Determine the dimensions of interest in the projectdimensions of interest in the project
• Determine the list of the most representative measuresmeasures
associated with each dimensionassociated with each dimension
Ž For each of the measures selected, identify which other
control variables might be impacted negatively
• Figure out the best combination of indicators and thebest combination of indicators and the
causal relations between themcausal relations between them in order to build abuild a
measurement planmeasurement plan for the project
13. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
13
BMP: Balancing Multiple Perspectives
A Generic four-dimensional BMP
14. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
14
BMP: Balancing Multiple Perspectives
An example with 4 dimensions
Œ Determine the dimensions of interest in the projectdimensions of interest in the project
A: Time, Cost, Quality & Risk
15. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
15
BMP: Balancing Multiple Perspectives
An example with 4 dimensions
• Determine the list of the most representative measures associatedmeasures associated
with each dimensionwith each dimension – Note: excerpt from the PSM Guide
16. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
16
BMP: Balancing Multiple Perspectives
An example with 4 dimensions
• Determine the list of the most representative measures associatedmeasures associated
with each dimensionwith each dimension – Note: excerpt from the PSM Guide
17. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
17
BMP: Balancing Multiple Perspectives
An example with 4 dimensions
Ž For each of the measures selected, identify which other control
variables might be impacted negatively
18. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
18
BMP: Balancing Multiple Perspectives
An example with 4 dimensions
• Figure out the best combination of indicators and the causalbest combination of indicators and the causal
relations between themrelations between them in order to build a measurement planbuild a measurement plan for
the project
19. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
19
BMP: Balancing Multiple Perspectives
An example with 4 dimensions
• Figure out the best combination of indicators and the causalbest combination of indicators and the causal
relations between themrelations between them in order to build a measurement planbuild a measurement plan for
the project
20. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
20
BMP: Balancing Multiple Perspectives
Measuring projects’ performances from multiple views
• Q: What about a possible usage of BMP for measuring
the overall project value within a BSC logic?
– Pros: BSC helps in managing multiple perspectives
– Cons: BSC does not provide the integrated measurement
results
• A: a joint usage of QEST nD model with BSC
framework
• A: BMP can help as a tool for considering the counter-
productive impacts of a possible control action in a
project by each BSC perspective
SourceSource: A.Abran & L.Buglione, A Multidimensional Performance Model for Consolidating Balanced Scorecards, International Journal of Advances in
Engineering Software, Elsevier Science Publisher, Vol. 34, No. 6, June 2003, pp.339-349
21. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
21
• Introduction
• Multidimensional Analysis in Project Management
ü Why is it needed?
ü Some Multidimensional Models
ü What should be measured and analysed?
ü Which set of indicators to select?
• BMP: Balancing Multiple Perspectives
ü The proposed measurement procedure
ü An example using 4 dimensions
ü Measuring project performances from multiple views
• Applying BMP
ü BMP-Q: the questionnaire
ü BMP-Q: the metrics
ü First Results & Feedbacks
• Conclusions & Prospects
Agenda
22. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
22
Applying BMP
BMP-Q: the Questionnaire http://www.geocities.com/lbu_measure/qestlime/bmp.htm
23. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
23
1a 1 Respondents profile by project role (# and %)
2 Experience profiles for current project role (# and %)
1
b
3 # analysis viewpoints (OLD)
1c 4 # analysis viewpoints (NEW)
2 1 # selected measures (OLD)
2 # selected metrics (NEW)
3 # affected viewpoints (NEW)
4 Avg of measures by viewpoint (# and %)
5 Ranking of selected measures by: abs value, respondent project role, analysis viewpoint
3a 1 List of causal relationships among measures
2 Ranking of relationships by: abs value, respondent project role, analysis viewpoint
4a 1 % respondents knowing amount of costs for m&c (monitoring & control) activities
4
b
1 Max, Min, Avg and Med for the returned values (%) – OLD
4c 1 Max, Min, Avg and Med for the returned values (%) – NEW
Applying BMP
BMP-Q: the Metrics http://www.geocities.com/lbu_measure/qestlime/bmp.htm
24. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
24
Applying BMP
First Results & Feedbacks
Teaching viewpoint:
• simplify the way for explaining the ISO/IEC 15939:2002 Information Model
and introducing the cause-effect linkage among development processes, as a
as a useful foundation for ICT Balanced Scorecards (BSC)
• help in pointing out to practitioners and students that there are plenty of well-
proven and established measures within the Software Engineering domain,
but also that each Measurement Program must be built having in mind that
unique situation and that every measure should be designed and verified to
be effectively goal-driven
Industrial viewpoint:
• increase awareness about cost-benefit analysis in implementing (or not) a
certain measure/indicator within their current QMSs.
ü Stimulus for industry people to know more and more about the effort/cost splitting of
their projects by SDLC phase (in many cases the answer is: don’t know)
ü Proposed the Test Selection Algorithm: each measure has its own cost from its
definition till the project closure, within a budget constraint
ü Key point: balancing the current available amount of budget and not to simply cut
controls. And in that eventuality, the point to face was: but which drawbacks of deleting
or adding some controls?
• Proposed to re-label such measures and indicators as project “controls”, to
provide a better match with managers’ responsibilities
25. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
25
• Introduction
• Multidimensional Analysis in Project Management
ü Why is it needed?
ü Some Multidimensional Models
ü What should be measured and analysed?
ü Which set of indicators to select?
• BMP: Balancing Multiple Perspectives
ü The proposed measurement procedure
ü An example using 4 dimensions
ü Measuring project performances from multiple views
• Applying BMP
ü BMP-Q: the questionnaire
ü BMP-Q: the metrics
ü First Results & Feedbacks
• Conclusions & Prospects
Agenda
26. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
26
Conclusions && Prospects
• Project managers often considers only two dimensions for tracking & control
of their projects (Time, Cost)
• At least, the Quality perspective should be also taken into account; further
perspectives (eg: Risk) could be also be useful if considered from the
planning phase on. Even challenging, a multi-perspective approach – as in
the BSC – is suggested
• It does not exist a “magic number” of indicators to track, but the goal is to
optimize costs and informative value derived from that amount of indicators,
establishing also the causal relationships among their related goals
• BMP (Balancing Multiple Perspectives) proposes a 4-step procedure to
select an appropriate balance of indicators from the various perspectives
taken into account (e.g. Time, Cost, Risk and Quality) and focus on the core
indicators from each of them, thereby helping the project manager in
tracking and control activities
• Due to its inner multidimensional nature, future joint usages with methods,
tools and frameworks taking into account concurrent dimensions (eg:
QEST/LIME) will be investigated
27. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
27
Q && A
28. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
28
luigi.buglione@computer.org , aabran@ele.etsmtl.ca
Thank you!