Application of Method Engineering Principles in Practice

1. Marko Bajec University of Ljubljana Faculty of Computer & Information Science Slovenia Application of ME principles in practiceLessons Learned and Prospects for the Future IFIP 8.1 Working Conference on ME Paris,France, April20-22, 2011

2. Agenda - 2 - Intro Application of ME principles to improve software development practice in Slovenian software companies. What we did What we learned What we intend to do next IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

3. Introduction University of Ljubljana The biggest of 5 SLO universities >40.000 students; 27 member institutions Faculty of Computer & Information Science >1200 students Undergraduate and postgraduate study programmes Laboratory for Data Technologies One of 20 labs Data Representation Networks, Graphs, Ontologies, Data streams, etc. Data Analysis Data Mining, Knowledge Disc., Information Retrieval Data Visualizationand Soundification Network Visualization and Soundification, Data stream Visualization, Visualisation ofsound. Data Management Data fusion and integration, Business Intelligence, etc. - 3 - IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

4. Longitudinal Study(1) - 4 - IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France T i m e l i n e 2010 2011 2006 2003 2004 2005 2007 2008 2009 1 Survey Unsupervised usage MasterProc Future work Check Point

5. Survey(1) - 5 - Objectives: to analyse existing practice: To what extent companies employ formalised methods in ISD? What kind of methods companies use in their everyday working? How technically and social sound are the methods companies use in their everyday working? Do companies lack higher levels of formalisation of their ways of working. Participants: Slovenian IT companies and companies with larger IT departments. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France 2003-2004

6. Survey(2) - 6 - Sample: Invitation sent to 200 companies (>3 emp) Respondents: 70 companies Small: 50 Medium: 13 Large: 7 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France Company Size Size of IT Department

7. Two types of questionnaires: A questioner for IT managers (decision makers), A questioner for developers (individuals). Questions for each discipline: Business modelling, Requirements acquisition, Analysis, Design, Implementation, Testing, Integration, Maintenance, Project management, Change and version management. Survey(3) - 7 - IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France R D A B In T Technical suitability I P M C Social suitability

8. Survey - Results 8 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France SDM underused in practice In-house SDM Social inappropriateness KeyObstacles Additional formalisation Key Findings Inflexibility Testing SDM not documented Change Mngm Maintenance Design Implementation SDM – Software development method(s)

9. Longitudinal Study(2) - 9 - IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France 2004-2007 T i m e l i n e 2010 2011 2006 2003 2004 2005 2007 2008 2009 2 Survey Unsupervised usage MasterProc Future work Check Point

10. Research goals and motivation 10 Motivation: Low usage of documented SDM in Slovenian companies; Lessons learned on projects not captured; reinventing the wheel. Awareness of their importance for QAin software development; Goal: To help companies to improve their software development practice (and results) by providing a framework and tools for reengineering their SDM, focusing on methods formalisation, their continuous improvement, and their adaptability to characteristics of a particular project or team.… IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

11. The MasterProc research project 11 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France MasterProc project: R&D project under the umbrella of the centre of excellence for information and communication technologies; Started in 2004, finished in 2007 Participating partners: Faculty of Computer & Information Science – responsible for research and development; Five software companies – interested parties; The Institute for Economic Research – responsible for market research Co-founded by the Slovenian Ministry of Higher Education, Science and Technology, European Commission and the participating Software Companies

12. Research method 12 Organised as a collaborative practice research. Interviews and surveys used to evaluate existing ways of working in the participating companies. Complemented by action research. For each of the participating companies a working team was set up comprising two researchers and two practitioners. The responsibility of the team was to take part in real projects to get firsthand information. Literature review used to study state of the art in the field of method construction. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

15. Conduct a research to understand how to introduce appropriate level of flexibility.

16. Conduct a research on improvement strategies;

17. identify appropriate strategy to make improvements;

18. Improve practice.Literature review Experiments and field studies Literature review Survey Intervention Design

19. A Framework for Method Reengineering 14 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France Project characteristics Method Configuration Documentedmethod Tailoredmethod Existing ways of working Method Use Learning cycle Method Construction Method Evaluation & improvement Suggestions for improvements Experience, new knowledge AMF - Agile Methodology Framework

21. Analyse characteristics of the company and its development team

22. Measure socio-technical suitability of the method used

23. Point out deficiencies of the existing ways of working

24. Study and suggest changes/improvements…

25. Select appropriate method weight

26. Design method structure (metamodel)

27. Define basic rules…

28. Elicit and capture method fragments

30. Rules in base method 17 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

31. Process flow rules 18 Process flow rules define conditional transitions among activities – theconditions that have to be met to perform the transition. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France R1: IF A2 is finished AND C THEN start A3

32. Structure rules 19 Structure rules similar to process flow rules; can constrain any link between method elements (not just links between activities). IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France R2: IF A2 AND C THEN OA1

33. Completeness rules 20 Completeness rules: help to check whether a project-specific method includes all required components, apply to a metamodel and not to a base method, tell the cardinality of the relationships between metaelements. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France Activity Technique 1 n n 1 Tool

34. Consistency rules 21 Consistency rules similar to completeness rules; aimed to assure consistency of the elements comprising a project-specific method; deal with interdependency between any two elements; help to avoid conflicting situations. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France The deliverable Business Model depends on the activity Business modelling

35. Facts 22 Facts Assertions that define the characteristicsof project for which a project-specific method is being created; Can be classified into base facts or derived facts; Base facts define project variables directly; Derived facts are derived from base facts using inferences or calculations. Examples: The project domain is well known. If the project field is telecommunications or healthcare then the project domain is well known. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

37. The project timeframe is 12 months,

38. …PCA algorithm

39. PCA as SME and its characteristics 24 PCA – Process Configuration Approach Based on ME principles Single point of departure (base method); Works on low layer of granularity (maximal flexibility); Combines the meta-modelling and extension/reduction based SME approaches. PCA advantages: Uses org.-specific method to create project-specific methods; Does not need or allow fragments to be changed during the creation process; Is easy, yet powerful; Focuses on software process improvement. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France Bajec M., Vavpotič D., Krisper M. Practice-driven approach for creating project-specific software development methods. Information and Software Technology. 2007, vol. 49, no. 4, pp. 345-365.

40. Performing PCA 25 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France Process process flow rules Process structure rules Process completeness rules Process consistency rules

41. Method evaluation & improvement 26 Aim: Continuous method evaluation and improvementto retain social and technical suitability of the base method. Approach: IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France Evaluate method elements Discuss possible improvements Identify strategies Implement changes

42. Evaluation method 27 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

43. SPI in terms of CMM 28 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France Method Configuration Method Configuration Optimized L5 Managed After a few learning Cycles L4 Method Use Method Construction Learning cycle Method Use Method Construction Defined L3 Repeatable Method Construction L2 Method Evaluation & improvement Method Evaluation & improvement Initial L1 Maturity levels of the CMM

44. AMT – Agile Methodology Toolset 29 AMT high-level architecture IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

45. AMT – MethElicit, MethModel 30 MethElicit: method elicitation Definition of a metamodel. Metamodel population (instances of metamodel elements, instances of relations). MethModel: process modelling IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

46. AMT – MethAdapt 31 MethAdapt: method adaptation Facilitates method tailoring. Based on rules and project specifics, MethAdapt creates an instance of the method that is tailored to the project characteristics. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

47. AMT – MethEval 32 MethEval: method evaluation Facilitates continuous evaluation of the method. Comprises two sub-modules: S1 allows method engineer to configure survey questionnaires and analyse the survey results. S2 generates survey questionnaires and distributes them among method users according to the context of their role. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

48. AMT – MethGen, MethUse 33 MethGen: method generation MethGen generates a method reference book in PDF format suitable for printing. Can be used as a learning material, given to customers, etc. MethUse: method use Enables dynamic access to the method contentthrough web portal. The main purpose of the module is to make access to the method content as easy and quick as possible. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

49. Longitudinal Study(3) - 34 - IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France 2008-2010 T i m e l i n e 2010 2011 2006 2003 2004 2005 2007 2008 2009 3 Survey Unsupervised usage MasterProc Future work Check Point

50. Check Point(1) 35 From May, 2007 till September 2010, unsupervised usage. Findings at check point: In four of the five companies, AMF learning cycle (Evaluate, Construct/Improve, Configure, Use) discontinued. In one company, SDM managed, in all other defined (documented) but not up to date. The maturity level raised in one of the companies in others kept at the same level. Companies reported they are managing project better that they used to and are still interested in capturing the knowledge gained at IT development projects (e.g. in a form of a base method) but did not want to spend too much time for that… IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

51. Check Point(2) 36 Reported obstacles: Software development process rather chaotic– i.e. difficult to describe and capture. Method engineer role not common – project managers like to manage projects by their own ways. The AMF learning cycle time consuming. There is no immediate advantage if decisions are discussed and documented. The AMT good as a prototype but not for professional use. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

52. Longitudinal Study(4) - 37 - IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France 2011- T i m e l i n e 2010 2011 2006 2003 2004 2005 2007 2008 2009 4 Survey Unsupervised usage MasterProc Future work Check Point

53. Prospects for the future 38 Making AMF learning cycle moreuser transparent and less time consuming would increase its use in practice! Most prominent future directions: Auto capturing and monitoring of In-action Methods (IM) based on projects’ Revision Control System (RCS) Repositories; Automaticadjustments of the Base Method based on projects’ IMs; Adequate tool-support for project monitoring, management, analysis and execution. IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011

54. Auto Capturing & Monitoring of IM 39 (Large) software projects typically use a RCS (e.g., Subversion, Visual Source Safe, Rational Team Concert, CVS, etc.) H1: Based on the RCS repository content, project’s IM can be inferred. H2: IMs, inferred from the RCS repository content, can be used to alter and generalize company’s Base Method. IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011

55. Silent Learning cycle 40 IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011 IM MonitoringSystem R C S R e p o s i t o r y Guidance Alterations Existing ways of working Artefacts Method Use Method Construction B a s e M e t h o d

56. RCS Repository 41 RCS repository consists of files and directories. Various repository layouts available. trunkrojectsidgetource trunkrojectsidgetoc trunkrojectsidgetmages branches IM artefacts written into repository structure. Majority of RCS support Custom properties Automatic property setting Pre/post-event hooks IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011

57. IM Auto Capturing 42 After an artefact is inserted into repository: Custom properties can be used to identify the position in the base method. If the position can not be not found then base method needs alterations. Examples: Insert (d6, A5, R3)  connect (A5, d6), getRule(A5, d6). Insert (d6, A6, R3)  new branch (A3, A6), getRule(A3, A6), connect (A6, d6), connect (A6, R3) Pre/post-event hookscan be used to capture additional information. IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011 d1 A1 d2 R1 R2 R3 A2 A3 d3 Rule1 Rule2 Rule3 A4 A5 A6 d5 d4 d6 d6

58. IM Guidance 43 At any time user may invoke monitoring system and get all available information based on: his role, metamodel of the base method and current position in the base method. E.g.: Next activities, Deliverables of an activity, Examples of deliverables, Deliverables‘ templates… IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011 d1 A1 d2 R1 R2 R3 A2 A3 d3 Rule1 Rule2 A4 A5 d5 d4

59. IM Control 44 Monitoring system can be used to control IM. E.g.: After d1 and d2, d4 is inserted. Monitoring system detects missing artefact d3 and alerts responsible user. User may introduce a new rule under which d3 is not required. Pre/post-event hooks can be used to invoke dialog box for user to add required info. IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011 d1 A1 d2 R1 R2 R3 A2 A3 d3 Rule1 Rule2 A4 A5 d5 d4

60. Auto generation of a Base Method 45 Base method can be automatically created step-by-step from IMs captured to RCS repository. Repository content represented with labelled multigraphs. Subgraph isomorphism algorithm used to identify an IM or its part in the base method. Constraints (Structure, Process, Consistency, Completeness) and Facts identified using Inductive Logic Programming. IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011

61. Important remarks 46 Adopted meta-model must be consistent among projects and with the company’s base method; Project parameters, properties and facts need to be captured at the project kick off. When checking RCS repository, only last revisions are considered. IFIP 8.1 Working conference on ME Paris, France, April 20-22, 2011

62. Conclusions 47 SDMs underused in practice, which effects negatively the development process and its products. AMF learning cycle useful but too time consuming for everyday practice. Automation of the learning cycle (Silent Learning): IM auto Capturing, Monitoring and Control Base method auto adjustment With an adequate tool support the silent learning cycle would possibly bring positive results. IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France

63. Questions 48 IFIP 8.1 Working Conference on ME April 20 - 22, 2011 - Paris, France University of Ljubljana Faculty of Computer & Information Science Tržaška 25, 1000 Ljubljana Marko Bajec Laboratory for DatabaseTechnologies Contact: Tel: +386 (1) 4768816 e-mail: marko.bajec@fri.uni-lj.si