1. Lessons learned from an ISO/IEC
15504 SPI Programme in a Company
Antonia Mas, Bartomeu Fluxà, Esperança Amengual
University of the Balearic Islands
Ctra. de Valldemossa, Km. 7.5. 07122 - Palma de Mallorca, Spain
{antonia.mas, eamengual}@uib.es
Brújula Tecnologías de la Información S.A
Edificio U-Parc BIT Ctra. Palma-Valldemossa, Km 7,4 - Palma de Mallorca, Spain
tfluxa@brujula.es
Abstract
This paper describes a first hand experience in Software Process Improvement within a Span-
ish Company which, since 2002, has been involved in an improvement program led by our re-
search group. We discuss the experience and the results of this improvement programme, as
well as the lessons learned to deal with new future improvement initiatives in other companies
of our environment.
Keywords
SPI (Software Process Improvement), SPICE (ISO/IEC 15504)
EuroSPI 2003 − 1

2. Session I: will be adapted later by the editor
1 Introduction
One of the main interests of our research group is to promote software process improvement within
the companies of our environment. We think that experimentation in research is crucial since it allows
demonstrating the applicability of the proposed solutions.
With this interest in mind, in 2002 we led a SPI project in eight small companies in the Balearic Is-
lands. The QuaSAR project [1] was an initiative that allowed us to analyse the software development
sector in our autonomous region, and it provided guidance to the companies for the improvement of
their software processes and for the ISO 9001:2000 certification. The QuaSAR project was a success-
ful initiative which motivated the companies for continuous process improvement.
Nowadays some of the QuaSAR participants still continue working for the improvement of their soft-
ware processes. In this article we expose the experiences and the results of a SPI programme in one
of these companies. This improvement programme has been placed into a grouped action with other
six companies with a dual objective. On the one hand, the main goal of the improvement initiative is
the identification of synergies among participants to share and spread the knowledge and project ex-
periences. This is an ambitious aspect of the programme since it implies a cultural change in a group
of small enterprises towards a collaborative association, further than the individual benefits of each
company, which has induced to the formation of a group with a common objective: the technological
excellence. On the other hand, the second goal is the dissemination of the improvement programme
results as a public awareness and incentive for other companies and institutions to motivate them to
form groups in future software process improvement initiatives.
2 Company background
Brújula is a Spanish company which began its activity in 2000. Nowadays, Brújula has 125 employees
dedicated to the development of Internet-based applications and the implementation of the infrastruc-
ture which supports them.
The implementation of a quality management system and the ISO 9000 certification obtained by the
company in 2002 initiated what has become one of the main identity insignias of Brújula as a com-
pany: the quality as a management strategy. In 2005 the company introduced the EFQM Excellence
Model to its management system and at the end of 2007 began the adaptation of its processes to the
ISO/IEC 15504 international standard for software process assessment and improvement.
Brújula’s vision is to be a large company, recognized at a national level for providing integral solutions
to its clients through the continuous improvement of the quality of its products. For the company, qual-
ity is the manner of achieving its business goals, satisfying the requirements and expectations of its
clients.
3 Steps of process improvement
The process improvement programme has followed the steps of process improvement utilizing a con-
formant process assessment according to the requirements of ISO/IEC 15504 described in ISO/IEC
155045-2 [2] and in ISO/IEC 15504-3[3]. In this section, the application of these steps, as detailed in
ISO/IEC 15504-4[4], is described.
2 − EuroSPI 2009

3. Session I: will be adapted later by the editor
3.1 Initiating process improvement
From an analysis of the company’s business goals and the existing stimulus to improvement, the fol-
lowing improvement objectives were set:
• Achievement of quality and productivity standards to allow the company technological innovation
in order to gain visibility and recognition to facilitate collaboration with partners and also
participation in large projects.
• Provision of high value to the clients through project management, incidence management and
release of defect-free software.
Initiation of a changing process aimed to improve the position of the company both at national and
European IT sectors.
To address these business goals, the Process Reference Model provided in ISO/IEC 15504-5[5] was
selected as a framework for the improvement programme. This model was the basis for the choice of
the processes to be assessed and the setting of the improvement targets. Moreover, the processes
referenced in ISO/IEC 15504-7[6] (Assessment of Organisational Maturity) were also considered in
order to select the necessary processes to reach a maturity level 2.
As recommended by the standard, the process improvement programme was implemented as a pro-
ject in its own right with defined project management, budget, milestones and accountability. There-
fore, a Process Improvement Plan was produced.
The following sections describe each one of the tasks performed through the improvement pro-
gramme.
3.2 Assessing current capability
In any improvement programme an initial process assessment is undertaken to understand the capa-
bility of the processes which have been previously selected for the improvement.
This assessment was conducted by an independent assessment team of two competent assessors. A
result of this assessment was the identification of a set of strengths and weaknesses that were used
as a basis for the development of an action plan.
The identified strengths were the following:
• The availability of a quality report which considers different organizational and management
aspects, as well as some product development aspects.
• The performance of project management, problem resolution and non-conformity management
activities with the support of a CASE tool.
• The identification and assignment of human resources to the different project processes based on
established profiles.
• The professionalism of the employees, the individual experience of the people in previous
projects, the positive work atmosphere and the fluent communication among team members.
The list of improvement actions compiled from the identified weaknesses was the following:
• Tailoring the company’s standard processes to each particular project.
• Individual management of project processes.
• Explicit adjustment of plans when deviations occur.
• Management of process work products.
EuroSPI 2003 − 3

4. Session I: will be adapted later by the editor
• Management of data about the implementation of the standard processes to demonstrate their
suitability and effectiveness, as well as to identify a continuous improvement of these standard
processes.
Planning improvements
3.2.1 Participants
For the implementation of the improvement project, the following roles were defined:
• Sponsor. The sponsor had the responsibility and the authority to make sure that adequate
resource and competencies were made available. This role was performed by the Service and
Project Manager of the company.
• Project Manager. The project manager should manage the project, validate achieved
improvements, coordinate and control current projects. This role was performed by the Quality
Manager.
• Organizer. The purpose of the organizer was to simplify the work of the teams and identify
common objectives. Tasks of the organizer were the organization of work sessions, supporting
process improvement and controlling projects. This role was performed by an assessor with
experience in teamwork.
• Quality specialist. The quality specialist gave support to the definition of the processes and
participated in the inspection and control of the currents projects.
3.2.2 Work groups
For the improvement of the different identified improvement areas, five work groups were formed.
Each work group was responsible for a set of processes. The leader of each work group had to de-
velop the plan and assign tasks to reach the identified objectives. Each group was formed by people
with different skills and abilities to assure that they would have distinct points of view of the processes
to improve. All of the members of the development area were part of one of the five groups. The fol-
lowing table shows these work groups.
Table 1. Work groups and improvement areas
Work group Processes
Management MAN.3 Project management
MAN.5 Risk management
SUP.4 Joint review
SUP.9 Problem resolution manage-
ment
ENG.12 Software and system mainte-
nance
Analysis ENG.1 Requirements elicitation
ENG.4 Software requirements analysis
ENG.8. Software testing
System ENG.2 System requirements analysis
ENG.3 System architectural design
ENG.9 System integration
ENG.10 System testing
Design- ENG.5 Software design
Construction- ENG.6 Software construction
Integration- ENG.7 Software integration
Installation- ENG.11 Software installation
Release SPL.2 Product release
Quality Assur- SUP.1 Quality assurance
4 − EuroSPI 2009

5. Session I: will be adapted later by the editor
Work group Processes
ance SUP.2 Verification
SUP.7 Documentation
SUP.8 Configuration management
SUP.10 Change request management
3.2.3 Action plan
The quality manager and the assessor analyzed the strengths and weaknesses identified in the previ-
ous assessment to identify: strong processes, improvement areas, and evidences of corrective ac-
tions. It was decided that the actions to be performed should obtain the planned results in nine
months.
Each work group had full autonomy to approach the improvement of the processes. Different commu-
nication channels were established in order to facilitate information interchange among groups. Firstly,
the intranet of the company would be used to spread the implemented improvements and to inform
about their achievement level. Secondly, the corporative blog would be used to inform about technical
improvements. Finally, workshops would be delivered every fifteen days.
The organizer had to give support to the different work groups by focusing the work to the achieve-
ment of the result through the definition of the processes, their documentation, the management of the
tasks and the adaptation of the processes to the standard.
The quality manager had to review the results of each work group to validate if the standard require-
ments were satisfied. If so, the process could be published and made accessible to the organization.
Otherwise, the deficiency in the process could be identified and the process would be adjusted to be
adapted to the standard.
Implementing improvement actions
The implementation of the improvement actions in every project in the company was performed
through joint reviews, quality controls and training actions.
The quality team coordinated and participated in periodical joint reviews with the project managers
and the different specialists assigned to the projects. The main goal of these meetings was to commu-
nicate and support the understanding of the performed improvement actions. During these reviews the
work products of the different processes were adjusted. It is important to notice that some of the im-
provements provoked and important change in the organization processes and also in the manner of
the employees approached their tasks.
In parallel with the joint reviews, the quality team performed quality controls by randomly inspecting
current projects, examining in each control a previously established set of processes. As a result of
these controls, preventive and corrective actions where defined both at a project level and at a proc-
ess level.
Throughout the project, training sessions where prepared and executed. Some of these sessions had
a general interest, for instance the session to introduce the ISO/IEC 15504 standard. Other more spe-
cific sessions focussed in particular processes.
3.2.4 Internal workshops
During the first months of the project, workshops were delivered every fifteen days. The assistants to
these workshops were the project sponsor, the leader of each work group, and the whole quality team:
the manager, the assessor, and the specialist.
The objectives of the workshops were:
• The revision of global achievements and team progression.
EuroSPI 2003 − 5

6. Session I: will be adapted later by the editor
• The alignment of the effort of the different work groups.
• The identification of transversal improvement areas and the decision of the optimal solution.
• The information about decisions that could affect the project advancement.
The order of the day was opened one week before the workshop allowing the inclusion of the different
points to consider by any one of the assistants. During the workshop the points before established
were addressed in rigorous inscription order.
As the project was progressing, the workshops were not delivered periodically. Instead, they were
summoned by any one of the interested parties.
3.2.5 External checkpoints
The improvement plan included two assessments, an initial diagnosis assessment and a final certifica-
tion assessment. Moreover, three external monitoring checkpoints were planned. These checkpoints
allowed evaluating the level of achievement of the implemented improvements and controlling if the
improved processes were aligned with the standard best practices.
From the results of these checkpoints the performed changes were validated and adjusted, as well as
new corrective actions were proposed.
3.2.6 Working day with the client
With the intention of aligning the improvement actions, in the middle of the project it was decided to
organize a working day with the client. A set of clients were selected, using criteria such as represen-
tative and experience, and they were invited to participate in a working day.
This working day was approached as a team dynamics in which the clients could present their prob-
lems, experiences, and the improvement areas that they had identified in their projects. The leaders of
each team assisted to this session as responsible of the projects progress.
The analysis of the results of this session confirmed that the effort made and the approach of the im-
provement project were appropriate, since the improvement areas identified by the clients were
aligned with the improvement actions that were being performed internally in Brújula. Moreover, from
the ideas provided by the clients, it was possible to refine and adjust some of the resultant work prod-
ucts.
Final assessment
At the moment of planning the final assessment, it was decided to reduce from twelve to eight the
number of processes to be evaluated, and plan a checkpoint for the other four processes included in
the improvement project. The main reason for this decision was that some of the improvement actions
to perform were still in an implementation phase in some of the projects. Moreover, they depended on
new software tools that had been acquired recently at that moment. Therefore, since these improve-
ment actions had not been performed in all of the projects, there was no sense in assessing these
processes.
The result of the final assessment was satisfactory because, although the established goal was not
reached, four of the eight assessed processes achieved a capability level 2. The other four processes
reach a capability level 1 with a largely or fully achievement of the 95 percent of the level 2 attributes.
With some more adjustments in the capability level 1 base practices for these processes, it would be
possible to reach a capability level 2.
Regarding the checkpoint results, it is important to highlight that the improvement actions performed in
three of the four processes achieved the necessary quality level to reach the objective. Therefore, it
was confirmed that the considered improvement actions were appropriate.
6 − EuroSPI 2009

7. Session I: will be adapted later by the editor
4 Lessons learned
Throughout the performance of the improvement actions a set of weaknesses have directly influenced
the project schedule and resources. These weak points are the following:
• Lack of knowledge of the standard. At the end of the improvement project it has confirmed that the
importance of understanding the standard and interpreting its best practices for each one of the
software life cycle processes was underestimated.
• Higher effort than expected. The effort for adapting the project tasks to the best practices defined
by the standard was not appropriately estimated.
• Implementation of changes in the processes and information about them. Information about new
established tasks, concept assimilation and the deployment of new manners of working were
hindered by an aggressive planned schedule.
• Project support tools. The implementation of a set of the best practices recommended in the
standard has shown that it is essential to have case tools to support the work.
On the other hand, the strengths that have been the key for the success of the improvement project
are the following:
• The active participation, consciousness and motivation of all the participants in the project.
• The role of the organizer. This was a key role since it gave continuous support to each team,
helping them to approach their objectives and acting as an interface to achieve coherence among
common improvement areas.
• Periodical monitoring of the improvements in the current projects. The improvement programme
had a very ambitious schedule and the number of improvement actions was important. Without the
joint reviews and the understanding of the improvement actions by the team, the cost of the
implementation of the improvement would have been higher.
• Support tools. The availability of a case tool for project management and change control make
easier the achievement of some process base practices. The acquisition of a case tool as
Enterprise Architect also facilitated traceability, configuration management and verifications.
5 Conclusions and further work
The improvement programme presented in this article has allowed the achievement of a capability
level 2 in a set of processes which were considered crucial for the company.
It is not ventured to affirm that this experience has provoked a continuous improvement feeling in the
company. At this moment, the head offices are not the only who promote a change, instead this
change is also requested by the different departments and people in the company. However, for a
successful Plan-Do-Check-Act cycle there are other two fundamental circumstances. On the one
hand, the established processes need to be consolidated and implemented in all of the projects in the
company, without any exception. On the other hand, the acquisition of new case tools is necessary to
speed up the implementation and management of the processes.
It also important to highlight that the company considered in this article has been pioneer in the Bale-
aric Islands in the achievement of a capability level 2 in some of the software life cycle processes. This
satisfactory experience has motivated other companies from turisTEC, an association of software
companies specialized in IT products and services for the tourism sector, to participate in a similar
initiative. Nowadays seven companies are participating in a new software process improvement pro-
gramme.
All of the companies from turisTEC consider that the implementation of a common set of best prac-
tices means competitive advantage. Companies which take part in this initiative will be individually
better situated in the market, and as the number of participants increases the association will also
EuroSPI 2003 − 7

8. Session I: will be adapted later by the editor
have competitive advantages as a technological excellence reference, trademark of the Balearic Is-
lands. Moreover, this standardization of the knowledge supports employee’s mobilization (with its posi-
tive and negative aspects) and allows the collaboration among companies in global, ambitious and
competitive projects.
Acknowledgements. This research has been supported by CICYT TIN2007-67843- TIN2007-67843-
C06-04 “Modelos de simulación basados en ontologías y mejora de procesos para arquitecturas ori-
entadas a servicios”, SOAQSim.
8 − EuroSPI 2009

9. Session I: will be adapted later by the editor
6 Literature
[1] Amengual, E.; Mas, A.: "Software Process Improvement in Small Companies: An Experience".
Industrial Proceedings of the European Software Process Improvement Conference, (Septem-
ber 2007), pp. 11.11-11.17
[2] ISO/IEC 15504-2:2003. Information technology -- Process assessment -- Part 2: Performing an
assessment. International Organization for Standardization (2003)
[3] ISO/IEC 15504-3:2004. Information technology -- Process Assessment -- Part 3: Guidance on
performing an assessment (2004)
[4] ISO/IEC 15504-4:2007. Information technology -- Process Assessment -- Part 4: Guidance on
use for process improvement and process capability determination (2007)
[5] ISO/IEC 15504-5:2006. Information technology -- Process Assessment -- Part 5: An exemplar
Process Assessment Model. International Organization for Standardization (2006)
[6] ISO/IEC 15504-7:2008. Information technology -- Process Assessment -- Part 5: Assessment of
organizational maturity. International Organization for Standardization (2008)
EuroSPI 2003 − 9

10. Session I: will be adapted later by the editor
7 Author CVs
Antònia Mas
Antònia Mas Pichaco is an assistant professor of software engineering and project management at the
University of the Balearic Islands. Her research interests include software process improvement, team
management and service management. She has promoted and coordinated the QuaSAR Project, a
software process improvement initiate in software small companies at the Balearic Islands. She
received her degree in computer science at UAB (Catalonia, Spain) and her PhD in computer science
at the University of the Balearic Islands. She is a member of the Software Quality Group of ATI Spain
from 1998 and she acts as editorial board member in the REICIS journal. She has also served as
program committee chair and program committee member of scientific conferences and workshops
related to software quality. She is an ISO/IEC 15504 assessor and she is focussed in assessing small
companies. Contact her at Campus UIB, Edifici A. Turmeda. Ctra. Valldemossa, Km. 7,5. 07122 Pal-
ma de Mallorca, Spain. antonia.mas@uib.es.
Bartomeu Fluxà
Bartomeu Fluxà Cabrer is the quality manager of Brújula Tecnologías de la Información. He has coor-
dinated and participated in different process improvement projects for the implementation of interna-
tional quality models such as EFQM, ISO 9001, ISO 14001, ISO/IEC 15504 and ISO/IEC 27001. He
received his degree in computer science at the University of the Balearic Islands. Contact him at
Brújula Tecnologías de la Información, Edifici U, ParcBit. Ctra. Valldemossa, km 7,4. 07121. Palma de
Mallorca, Spain. tfluxa@brujula.es.
Esperança Amengual
Esperança Amengual Alcover is a lecturer in computer science at the University of the Balearic Is-
lands. Her research interests include software process improvement and teamwork. She has partici-
pated in the QuaSAR Project, a software process improvement initiate in software small companies at
the Balearic Islands. She is a member of the Software Quality Group of ATI Spain and an ISO/IEC
15504 assessor. She received her degree in computer science at the University of the Balearic Is-
lands. Contact her at Campus UIB, Edifici A. Turmeda. Ctra. Valldemossa, Km. 7,5. 07122 Palma de
Mallorca, Spain. eamengual@uib.es.
10 − EuroSPI 2009

11. EuroSPI 2003 − 11