Paper : The Evolution of Information Systems. A Case Study on Document Management Authors : Paolo Salvaneschi Session: Industry Track Session 3: Evolution and migration

1. 27th IEEE International Conference on Software Maintenance
September 25 – 30, 2011
The Evolution of Information Systems
A Case Study on Document Management
Paolo Salvaneschi
University of Bergamo
Faculty of Engineering
ICSM 2011 Paolo Salvaneschi 1

2. Summary
• The problem
• Aims of the study
• The Information System
• The study
• Results
• Validity and Scope
• Conclusions and future work
ICSM 2011 Paolo Salvaneschi 2

3. The problem
• The management of knowledge required to
support the evolution of large information
systems
• A well-known and difficult problem for IT
departments of organizations
• Many existing applications evolving for a long
period of time (e.g. 10 years).
• New projects :
- a change of one or more applications ( evolution step)
- or the development of a new application
ICSM 2011 Paolo Salvaneschi 3

4. The problem
• Although the initial development was based on
the best Software Engineering practices, the
delivered documents become quickly outdated
and no longer useful
• The cost of maintaining them during the
evolution phase is too high.
• Analysts and programmers do not have time to
update the documents
• Turnover causes loss of knowledge
ICSM 2011 Paolo Salvaneschi 4

5. Aims of the study
• To verify a set of hypotheses
- The evolution can be supported by a minimal
set of “good enough” documents
- The set of evolution documents derives
from a selections of contents of the
development documents
- It is based on a cost-benefit analysis
• To define an engineering guideline (types and
patterns of documents)
• To develop a tool for managing the documents
ICSM 2011 Paolo Salvaneschi 5

6. Aims of the study
• Minimal set of “good enough” documents
- Knowledge is a property emerging from a system
composed by analysts/programmers and documents
- Documents do not require being “complete” It is
sufficient that they be “good enough”
- The analysts/programmers may use a minimal set of
document as a “landscape” for understanding where
to read the details in the code
Reference: the theory of minimalist instruction
founded in the psychology of learning and problem
solving [Carroll, J. M. 1990]
ICSM 2011 Paolo Salvaneschi 6

7. Aims of the study
• Questions
- Are the actors involved in the evolution able to
identify a minimal set (based on a cost-benefit
analysis) of evolution documents they consider
important to understand the software to be
changed?
- What is the content of these documents?
- What is the relationship between these evolution
documents and the development documents?
- Are there recurrent sets of evolution documents
for classes of applications?
ICSM 2011 Paolo Salvaneschi 7

8. The Information System
• The information system of a large retail company
- 100 stores in Italy, 2 billion Euro turnover,
more than 8000 employees (year end 2009)
- IT department: 70 employees,150 servers,
4000 clients, about 100 software applications,
5000 users.
- Internally developed software (external
suppliers) and tailored market-available
packages
- AS400 and Web-based environments
ICSM 2011 Paolo Salvaneschi 8

9. The study
• 21 interviewed people (analysts/programmers,
project managers and system managers) and an
external consultant having the role of researcher
• 89 applications
• About 400.000 Java-JSP LOC, about 1.100.000
RPG LOC and tailored COTS
ICSM 2011 Paolo Salvaneschi 9

10. The study
Number, role and experience of interviewed people
ICSM 2011 Paolo Salvaneschi 10

11. The study
Applications classified by technology
Software applications classified by evolution life
ICSM 2011 Paolo Salvaneschi 11

12. The study
• Information sources:
- Development document stored on a project
basis into a database under version control
- Applications owners
• We study the documents that can help the
analyst/programmer to understand the code
during software evolution
• We do not examine the source code
ICSM 2011 Paolo Salvaneschi 12

13. The study
• Meetings between the researcher and each
application owner
- The researcher explains the hypotheses and
requests the application owner to identify a
minimum set of evolution documents
• An evolution document may be:
- A development document stored in the
database
- A part of a development document
(a document whose quality is poor may
require a re-documentation process)
ICSM 2011 Paolo Salvaneschi 13

14. The study
• Result: a set of candidate documents
• Selection through a cost benefit analysis
Selection rule
ICSM 2011 Paolo Salvaneschi 14

15. The study
• Rcost Cost spent by the new
analyst/programmer for reconstructing the
content of the evolution document, when
needed, from code, data structures and program
behavior
• Mcost Maintenance cost of the evolution
document, estimated by the number of pages
Low < 5 pages
Medium 5 to 10 pages
High > 10 pages
ICSM 2011 Paolo Salvaneschi 15

16. The study
• Example 1:
- Specification document : a detailed
specification of interactive screens.
- The reconstruction cost is low: the
programmer may run the product and use the
person machine interface as a living
specification for understanding code
- discard
ICSM 2011 Paolo Salvaneschi 16

17. The study
• Example 2:
- Design document : includes a five-page model
of software architecture
- The maintenance cost is low
- The cost of understanding and reconstructing
the architecture from code is high
- select
ICSM 2011 Paolo Salvaneschi 17

18. The study
• Example 3:
- The design document includes a detailed
specification (a dozen pages) of an algorithm
implementing a set of very complex business
rules
- The reconstruction cost is high
- Also the maintenance cost is high
- The owner notes that it is
very difficult to understand
the algorithm by reading code.
- After a discussion: select
ICSM 2011 Paolo Salvaneschi 18

19. Results
• Example: the application "Supply Chain"
Development
documents
Evolution
Documents
The minimal set
The E-R model of the central RDBMS is not considered
ICSM 2011 Paolo Salvaneschi 19

20. Results
• The minimalist approach: an average reduction
factor of 10 (number of pages of development
documents / number of pages of evolution
documents)
• Types of evolution documents:
- Landscapes (e.g. Software architecture)
- Aspects (e.g. Use case)
- Critical points (e.g. Complex algorithm)
ICSM 2011 Paolo Salvaneschi 20

21. Results
• Development documents vs evolution documents
- Some contents of the development documents
have been discarded or significantly downsized
- The development documents tend to be
complete and layered according to abstraction
levels. The evolution documents include only
the most important information and collect
parts of the development documents at very
different levels of abstraction
ICSM 2011 Paolo Salvaneschi 21

22. Results
• From the analysis of the
meeting reports:
Patterns of evolution documents
ICSM 2011 Paolo Salvaneschi 22

23. Results
• Classes of applications
ICSM 2011 Paolo Salvaneschi 23

24. Results
• Types
of documents
• Content
of each type
ICSM 2011 Paolo Salvaneschi 24

25. Results
• The wiki tool
- MediaWiki environment
- Now in service
- Loaded with all the 89 applications examined
in the study
- 500 pages and links to three data models
generated by an external CASE
- 60 registered users
- A standard process to upgrade the wiki at the
end of each development step
ICSM 2011 Paolo Salvaneschi 25

26. Results
ICSM 2011 Paolo Salvaneschi 26

27. Validity and Scope
• Threats of validity
• Representativeness of the results
- People and applications biased by specific skills or
technologies
- Owners with different roles and experience.
Applications covering different technologies and
development /acquisition processes
• Reliability of classification
- Subjective assignment of RCost and MCost.
- Reduced using an evaluation based on the number of
pages
ICSM 2011 Paolo Salvaneschi 27

28. Validity and Scope
• Scope
• Study related to just one information system
Generalization of the results to other information
systems should be done with caution
• A contribution that can be replicated on other
information systems
ICSM 2011 Paolo Salvaneschi 28

29. Validity and Scope
• Scope
• The study may be generalized to other commercial
information systems?
- The studied information system is representative
of many others commercial systems
- The types and patterns of evolution document do
not depend on specific technologies or business
processes
They essentially depend on:
The architectural characterization of the application
The degree of transparency (in house developed vs COTS)
ICSM 2011 Paolo Salvaneschi 29

30. Conclusions and future work
• The study questions have been positively answered
• The guideline and the tool are used in the specific
information system
• Replicate the study for other commercial systems
• Other types of systems?
ICSM 2011 Paolo Salvaneschi 30