This presentation discusses the need for tailoring known SPI models by a SLR analysis and proposes a 5-step process and related techniques to adopt

1. PROFES 2011
Torre Canne (BA) – Italy, June 21 2011
Proposing an ISO/IEC 15504 Compliant Method for Process
Capability / Maturity Models Customization
Jean C.R. Hauck
Christiane Gresse von Wangenheim
Fergal McCaffery
Luigi Buglione
www.eng.it

2. UFSC At a glance
Federal University of Santa Catarina
Florianópolis/Brazil [http://www.ufsc.br]
• 48 Master courses
• 38 PhD courses
• 62 Undergraduate courses
• EUR 268 million annual budget
• 25,737 Undergraduate students
• 8,543 Graduate students
• 34,280 Students
[http://www.youtube.com/watch?v=V6E1Z5DEuvk]
INCoD is to be a reference institute for excellence in research, validation
and dissemination to support digital convergence. [http://www.incod.ufsc.br]
The Software Quality Group focuses on scientific research, development and
transfer of SE models, methods and tools. [http://www.gqs.ufsc.br]
www.eng.it

3. DKIT At a glance
Dundalk Institute of
Technology is a 90 acre
campus situated between
Dublin and Belfast (each
approximately 50 miles
away).
The Institute consists of 4
Schools:
1. Business & Humanities
2. Informatics & Creative
Arts
3. Engineering
The Regulated Software Research Group which is 4. Health & Science
part of LERO (the Irish Software Engineering
Research Centre) is part of the School of Informatics
& Creative Media
www.eng.it

4. Engineering At a glance
_ The first Italian ICT player
_ more than 730 M/€ revenues Research and PA & HC Finance Industry TELCO Utilities
Development
_ 1000 clients
_ 6,300 IT specialists System Int. &
Consultancy
% 46 70 54 80 80
Outsourcing % 35 10 27 10
Software
% 19 20 19 10 20
ERP IT Security ECM
Plant Management
Managed Operations Broadband & Media
System
www.eng.it
www.eng.it

5. Customizing SPCMMs Goals of the presentation
 G1. Discuss the daily need to tailor, applied to the SPI
world
 G2. Analyze the state-of-the-art about SPCMMs
customizing
 G3. Propose a 5-step process and related basic
techniques from a SLR analysis
5 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

6. Customizing SPCMMs Agenda
• Introduction
– Customize in the ‘real’ world
– …and in the SPI world
• Requirements for SPCMMs
– Criteria and Characteristics
• Existing Methods for SPCMMs Customization
– Methods for Developing SPCMMs
– Development Process for Standards
– Developing a SPCMM in Practic
• A Proposal for a Method for Customizing SPCMMs
– Elements to consider
– Main phases and Basic Techniques
– First results & Discussion
• Conclusions & Prospects
• Q&A
6 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

7. Introduction ‘Natural’ Process Improvement
Natural
• Stalactites – 1 cm
each 50-60 years
• Stalagmites – 1 cm
each 100-120 years
• …have our companies also a (mid-)long term vision
for process improvement?
7 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

8. Introduction Customize in the ‘real’ world...
8 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

9. Introduction Customize in the SPI world...
... ...
• How to customize a SPCMM and which elements to
take into account?
9 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

10. Customizing SPCMMs Agenda
• Introduction
– Customize in the ‘real’ world
– …and in the SPI world
• Requirements for SPCMMs
– Criteria and Characteristics
• Existing Methods for SPCMMs Customization
– Methods for Developing SPCMMs
– Development Process for Standards
– Developing a SPCMM in Practic
• A Proposal for a Method for Customizing SPCMMs
– Elements to consider
– Main phases and Basic Techniques
– First results & Discussion
• Conclusions & Prospects
• Q&A
10 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

11. Requirements for... Criteria & Characteristics (1)
• Several classification for requirements expressing the capability/maturity
of an organization
 Becker et al. (2009)  7 criteria
o (i) comparison with existing models, (ii) iterative development, (iii) model; evaluation, (iv)
multi-methodological procedure, (v) identification of the relevance of problem, (vi) problem
definition, (vii) published results and (viii) scientific documentation;
 Matook & Indulska (2009)  5 criteria
o generality, flexibility, completeness, usability and comprehensibility
 Lahrmann & Marx (2010)  4 criteria, as follows
Criteria Characteristics
Dimensions 1D 2D Hierarchic
Representation Continuous Staged
Audience Unique Multiple
Assessment Approach Qualitative Quantitative
11 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

12. Requirements for... Criteria & Characteristics (2)
• ISO/IEC 15504-2 proposes specific criteria for PRM and PAM
• R1 - Declaration of the specific domain and community of interest, including
aspects of consensus achievement;
• R2 - Description of processes including: unique title, purpose and outcomes;
• R3 - Presentation of the existent relationships between processes;
• PAM
• R4 - Statement of scope and coverage of the model;
• R5 - Indication of the capability levels selected from a measurement
framework for the processes, starting at level 1;
• R6 - Mapping for the selected processes of the chosen PRM(s) ;
• R7 - Details of performance indicators of the processes, mapped to the
purposes and outcomes of selected the processes of PRM(s);
• R8 - Detailed process attributes of measurement framework;
• R9 - Objective evidence that the requirements are fulfilled.
12 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

13. Customizing SPCMMs Agenda
• Introduction
– Customize in the ‘real’ world
– …and in the SPI world
• Requirements for SPCMMs
– Criteria and Characteristics
• Existing Methods for SPCMMs Customization
– Methods for Developing SPCMMs
– Development Process for Standards
– Developing a SPCMM in Practic
• A Proposal for a Method for Customizing SPCMMs
– Elements to consider
– Main phases and Basic Techniques
– First results & Discussion
• Conclusions & Prospects
• Q&A
13 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

14. Existing methods... Methods for developing SPCMMs
• Several methods proposed during the past years:
• de Bruin et al. (2005) – 6 steps sequence for developing a Maturity
Assessment Model, not addressed in detail the customization issue
• Mettler et al. (2009) – 6 main steps from a model user perspective
• Maier, Moultrie & Clarkson (2009) – guide for developing maturity grids
but not focusing on SPCMMs
• Salviano et al. (2009) – PRO2PI framework, by a 7-step process, but not
details about how to customize SPCMMs
• Matook & Indulska (2009) – QFD-based approach for keeping the Voice of
the Customer (VoC), with a quality measure as outcome
• Becker et al. (2009) – Set of defined requirement for development but no
details about the evolution of such models
• Note: none of them is specifically targeted to meet ISO/IEC 15504-2
requirements!
14 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

15. Existing methods... Development Process for Standards (1)
www.iso.org/iso/stage_codes.pdf
15 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

16. Existing methods... Development Process for Standards (2)
www.iso.org/iso/stage_codes.pdf
16 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

17. Existing methods... Developing a SPCMM in Practice
• 52 SPCMMs analyzed
• Only 21% of those models (n=11) presented
detailed info about their development, 27%
(n=14) superficial ones, 52% (n=27) none
Source: Gresse von Wangenheim C., Hauck J.C.R., Zoucas A. Salviano C.F., McCaffery F., Shull F., Creating Software Process Capability/Maturity
Source
Models. IEEE Software, vol. 27 no. 4, pages 92 -94, July/August 2010.
17 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

18. Customizing SPCMMs Agenda
• Introduction
– Customize in the ‘real’ world
– …and in the SPI world
• Requirements for SPCMMs
– Criteria and Characteristics
• Existing Methods for SPCMMs Customization
– Methods for Developing SPCMMs
– Development Process for Standards
– Developing a SPCMM in Practic
• A Proposal for a Method for Customizing SPCMMs
– Elements to consider
– Main phases and Basic Techniques
– First results & Discussion
• Conclusions & Prospects
• Q&A
18 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

19. A proposal for... Elements to take into account
1. Standards development 2. Existing methods for
procedures developing SPCMM/grids
4. KE techniques 3. Current way to customize
SPCMMs
19 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

20. A proposal for... Main Phases & Basic Techniques
1. 2. 3. 4. 5.
Knowledge Knowledge Knowledge Knowledge Knowledge
Identification Specification Refinement Usage Evolution
Method Phase Basic Technique(s)
1. Knowledge Identification Ontology Development; Glossary Development; Literature Review;
Systematic Literature Review; Goal Question Metric; Expert
Selecting; Delphi; Focus Groups
2. Knowledge Specification Delphi; Perspective-Based Reading; Checklist-based Reading;
Semantic Mapping; Domain quality requirements elicitation; Focus
groups; Structured Interview; Nominal group; Software Process
QFD; Process Selection
3. Knowledge Refinement Expert Selecting; Delphi; Guidelines of Modeling; Behavior
Engineering; Interrater Agreement; Checklist-based Reading
4. Knowledge Usage Goal Question Metric; Practical Software and Systems
Measurement
5. Knowledge Evolution Model change request management
20 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

21. A proposal for... First Results & Discussion
• Some application of those concepts, by Explanatory Case Studies
SPCMM Comments/Notes
1. SaaS domain  Software solution offered as a service, developed using SOA
 SaaS scenario requires specific Non-Functional Reqs (e.g.
security, availability, service continuity…)  needs a SPCMMs
customization
 SaaS SPCMM [74] developed at UFSC adopting steps #1-3, not
yet performed steps #4-5
2. Medi SPICE –  customization of ISO/IEC 15504 for software development of
http://medispice.ning.com medical devices
 Particularities due to regulatories approvals within the region(s)
in which the medical device will be marketed (e.g. AAMI/IEC
62304, FDA, etc...)
 Performed steps #1-3 from Jan-Dec 2010
21 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

22. Customizing SPCMMs Agenda
• Introduction
– Customize in the ‘real’ world
– …and in the SPI world
• Requirements for SPCMMs
– Criteria and Characteristics
• Existing Methods for SPCMMs Customization
– Methods for Developing SPCMMs
– Development Process for Standards
– Developing a SPCMM in Practic
• A Proposal for a Method for Customizing SPCMMs
– Elements to consider
– Main phases and Basic Techniques
– First results & Discussion
• Conclusions & Prospects
• Q&A
22 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

23. Customizing SPCMMs Conclusions & Perspectives
• Tailoring, Customizing and SPI
 Continuous need to find for the ‘best fit’, both in the real world as well as in SPI
 ISO/IEC 15504 family of models is an example, contextualizing SPICE to several
domains
• State-of-the-art and main requirements
 Several studies for determining the typical requirements for developing a SPCMM...
 ...but few information about how to customize them
 A proposal for customizing SPCMMs
 Four elements to take into account, jointly with a Knowledge Engineering (KE)
approach
 5 steps (identification; specification; refinement; usage; evolution)
 Some lessons learned
 It doesn’t exist an ‘absolute’ model, each model has a logical boundary as well as its
pros & cons to know for been selected and chosen for building a new model
 A combination of 2+ SPCMMs is better than running 2+ models in parallel (less effort
and costs)  the reference point is your management system, not the SPCMM
If you are out to describe the truth, leave elegance to the
tailor. (Albert Einstein)
23 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

24. Customizing SPCMMs Q&A
Grazie per l’attenzione!
Thanks for your attention!
24 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione

25. Customizing SPCMMs Contacts
Jean Carlo R. C. Gresse von Fergal Luigi
Hauck Wangenheim McCaffery Buglione
UFSC UFSC DKIT Engineering.IT/ETS
jeanhauck@gmail.com gresse@gmail.com fergal.mccaffery@dkit.ie luigi.buglione@eng.it
25 PROFES 2011 – Torre Canne (BA), June 21, 2011– © 2011 L.Buglione