[QUATIC 2012] PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation
•
1 j'aime•1,840 vues
The document presents a tool called PSP PAIR that automatically analyzes performance data from the Personal Software Process (PSP) to identify problems and recommend improvements. PSP generates large amounts of data but analyzing it manually is time-consuming. PSP PAIR addresses this by developing a performance model and using it to analyze time estimation accuracy and other metrics from PSP data. It identifies potential problems and suggests actions like stabilizing productivity. An evaluation found PSP PAIR could help engineers using PSP by speeding up analysis and proposing targeted improvements. Future work includes validating the model with more data and expanding PSP PAIR to support the Team Software Process.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à [QUATIC 2012] PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation
Similaire à [QUATIC 2012] PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation (20)
Plus de Strongstep - Innovation in software quality
Plus de Strongstep - Innovation in software quality (20)
Dernier
Dernier (20)
[QUATIC 2012] PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation
- 1. PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation Lisbon, 3 of September of 2012 César Duarte (Strongstep), João Pascoal Faria (INESC TEC/FEUP) and Mushtaq Raza (Abdul Wali Khan University Mardan/FEUP) SEI, TSP, PSP, Team Software Process and Personal Software Process are service marks of Carnegie Mellon University
- 2. About Strongstep • Strongstep is a company specialized in software engineering that contributes to the improvement of software quality in Portugal and in the world Universities • We want to induce a positive change in organizations. This will represent a step with a strong, sustainable and innovative way - a strong step Strongstep • Projects portfolio: Reference Enterprises Institutions • Process improvement with CMMI DEV L2, L3, L5, CMMI SRV, TSP/PSP, combining agile/CMMI, Six Sigma, NP4457, Kanban, Scrum, ITIL, PMBOK, ISTQB, RUP Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 1
- 3. About the authors • Consultant in Software Engineering at Strongstep • Software Engineer, holds a Master's Degree in Informatics and Computing Engineering. He graduated from the Faculty of Engineering, University of Porto (FEUP) in Portugal • Master thesis entitled Automated Software Processes Performance Analysis and Improvement Recommendation. • Worked as a Research Assistant at ESB - Universidade Católica Portuguesa – Porto • Specialties and research interests are related to Software César Duarte Engineering and Software Process Improvement. Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 2
- 4. About the authors • PhD in Electrical and Computer Engineering from the Faculty of Engineering of the University of Porto (FEUP). • Assistant Professor at the Department of Informatics Engineering of FEUP and researcher at INESC Porto. • Certified Personal Software Process (PSP) Developer, Authorized PSP Instructor, and trained Team Software Process (TSP) Coach by the Software Engineering Institute (SEI) of the Carnegie Mellon University. João Pascoal Faria • Currently involved in research projects, supervisions and consulting activities in the areas of model-based testing, model- driven development and software process improvement. • PhD student in MAP-i Doctoral Program at University of Porto, Portugal. • Assistant Professor at Abdul Wali Khan University Mardan, Pakistan (Currently on study leave). • Research interest includes Personal Software Process (PSP), Global Software Development (GSD) and Usability Engineering. Mushtaq Raza Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 3
- 5. “Measurements are key. If you cannot measure it, you cannot control it. If you cannot control it, you cannot manage it. If you cannot manage it, you cannot improve it.” Harrington 1991 Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 4
- 6. Expected Outcomes • Understand the context of the research topic and the problem address • Understand the approach taken to tackle this problem • Develop an understanding of the implementation and some concepts of the developed tool (PSP PAIR) • Sharing ideas on what could be improved and discuss them Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 5
- 7. Context • Medium/High maturity software development processes (e.g. PSP/TSP) • Lots of base data generated • This data can be periodically analyzed • Identify performance problems, root causes and remedial actions Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 6
- 8. TSP, PSP and CMM(i) image from www.sei.cmu.edu/tsp/ Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 7
- 9. Problem • Too much data • Too much time wasted analyzing the data manually • Expert knowledge needed to do the analysis Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 8
- 10. Objectives • Develop a model and respective tool for: • Automating the analysis of performance data • Recommending improvement actions • Tool results evaluation: • Comparison between results using the tool and manual analysis results on real data from PSP users Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 9
- 11. Related Work • There is a large amount of tools for PSP/TSP • Most of them automate the data collection • None of them really automates the data analysis part • They only transform the data into graphs and charts to an easier analysis of the data. • Relationships between indicators and recommended values Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 10
- 12. Approach Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 11
- 13. Time Estimation Performance Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 12
- 14. Performance Model Time Estimation Error Size Productivity Estimation Error Estimation Error Missing Expurious Productivity Parts Parts Stability Part Estimation Plan and Postmortem Error Productivity Stability Defect Density Process DLD & DLD Review in Unit Tests Stability Productivity Stability Defects Injected Def. Removed Before Unit Test (Tot.Def.Density) Tests (Review Yield) Productivity Stability Review to Review Code & Code Review Development Ratio Rate Productivity Stability Legend: affects according to formula may affect, according to literature (validation required) Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 13
- 15. Performance Model Time Estimation Error Size Productivity Estimation Error Estimation Error Missing Expurious Productivity Parts Parts Stability Part Estimation Plan and Postmortem Error Productivity Stability Defect Density Process DLD & DLD Review in Unit Tests Stability Productivity Stability Defects Injected Def. Removed Before Unit Test (Tot.Def.Density) Tests (Review Yield) Productivity Stability Review to Review Code & Code Review Development Ratio Rate Productivity Stability Legend: affects according to formula may affect, according to literature (validation required) Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 14
- 16. Performance Model Time Estimation Error Size Productivity Estimation Error Estimation Error Missing Expurious Productivity Parts Parts Stability Part Estimation Plan and Postmortem Error Productivity Stability Defect Density Process DLD & DLD Review in Unit Tests Stability Productivity Stability Defects Injected Def. Removed Before Unit Test (Tot.Def.Density) Tests (Review Yield) Productivity Stability Review to Review Code & Code Review Development Ratio Rate Productivity Stability Legend: affects according to formula may affect, according to literature (validation required) Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 15
- 17. Performance Model Time Estimation Error Size Productivity Estimation Error Estimation Error Missing Expurious Productivity Parts Parts Stability Part Estimation Plan and Postmortem Error Productivity Stability Defect Density Process DLD & DLD Review in Unit Tests Stability Productivity Stability Defects Injected Def. Removed Before Unit Test (Tot.Def.Density) Tests (Review Yield) Productivity Stability Review to Review Code & Code Review Development Ratio Rate Productivity Stability Legend: affects according to formula may affect, according to literature (validation required) Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 16
- 18. Implemention: PSP PAIR tool • Personal Software Process Performance Analysis and Improvement Recommendation • Simple usable GUI • Interprets and analyzes the data from PSP student workbook • Suggests improvement recommendations Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 17
- 19. PSP PAIR: Results Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 18
- 20. PSP PAIR: Recommended Actions Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 19
- 21. Results Assessment Manual Analysis PSP PAIR (Performance Analysis Report) Problems Problems Time estimation accuracy Value of Productivity Stability in DLD + (same as time estimation DLDR suggests a problem. error). Problem with the changes in the way of Productivity, namely at DLD working. phase. Value of Total Defect Density suggests a Defect density in UT. potential problem. Based on the average of all the projects, problems identified at: Time estimation error Total productivity Stability Productivity stability at DLD and DLD Review Defect Density in UT Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 20
- 22. Results Assessment 2 Manual Analysis PSP PAIR (Performance Analysis Report) Process Improvement Proposals Recommended Actions P1: Stabilize and make more efficient the Try to keep a stable productivity and check DLD process to improve and stabilize that has been modified lately. productivity: If the productivity has increased try to avoid expensive DLD representations keep it stable at that new value in (e.g., equations in Word) future projects. avoid long DLD documents (compared Improve the process quality. to code size) Do an analysis on the more frequent do Logic Specifications that can be and more expensive errors in order to used as code comments and can be not repeat them, define preventive easily written actions. P2: Widen the size range to have more basis for estimations. Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 21
- 23. Comparision With Other Approaches (a) guidelines Concepts, methodologies and tools CAR – Cause Analysis Process Dashboard SEI PSP Workbook SEI TSP Workbook Decision Support SPC - Statistical Process Control and Resolution PSP PAIR Hackystat PSP/TSP Systems Steps Necessary definitions Base metrics collection along the x x x x x projects Metrics x x x x x x Performance Indicators calculation Performance indicators Identification of performance x (a) x x problems Control Limits, etc. Identification of possible causes Cause-effect relationships x (a) x Identification of potential Potential improvement actions for x (a) (a) improvement actions each cause Evaluation of cost-benefit of actions x (a) (a) x Action selection/recommendation x (a) x Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 22
- 24. Conclusions • Results suggest that the performance model is valid • Results support the idea that PSP PAIR can provide a good support to engineers using the PSP by: • Speeding up the analysis of collected data with automation • Suggesting improvement actions based on the analysis • The architecture of PSP PAIR gives room for further development Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 23
- 25. Future Work • Validate further and refine the performance model with larger datasets • Ranking the improvement recommendations • Import data from other sources • Using user feedback to improve results • Extend PSP PAIR to Team Software Process Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 24
- 26. Presentation Takeaways • Understand the context of the research topic and the problem address • PSP generates a lot of data that can be periodically analyzed to Identify performance problems, root causes and remedial actions • Overwhelming amount of data and a lot of time wasted doing the analysis manually • Understand the approach taken to tackle this problem • Performance model to analyze time estimation performance using related indicators • Develop an understanding of the implementation and some concepts of the developed tool • Sharing ideas on what could be improved and discuss them Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 25
- 27. “Measurements are key. If you cannot measure it, you cannot control it. If you cannot control it, you cannot manage it. If you cannot manage it, you cannot improve it. It is as simple as that” Harrington 1991 Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 26
- 28. Contacts Email: cesar.duarte@strongstep.pt Mobile: + 351 91 898 70 25 Strongstep - Innovation in software quality Email: geral@strongstep.pt Web: www.strongstep.pt Telephone: + 351 22 030 15 85 Porto: UPTEC - Parque de Ciência e Tecnologia da U. Porto Rua Alfredo Allen, 455/461 4200-135 Porto, Portugal Lisbon: Av. Infante Santo, Nº 63-3ºEsq. 1350-177 Lisbon, Portugal Strongstep 2012 | PSP PAIR: Personal Software Process Performance Analysis and Improvement Recommendation 27
Notes de l'éditeur
- Good Morning and welcome to my presentation regarding PSP PAIR, Personal Software Process Performance Analysis and Improvement Recommendation. The authors of this paper are César Duarte, João Pascoal Faria and MushtaqRaza.João Pascoal Faria, PhD in Electrical and Computer Engineering from the Faculty of Engineering of the University of Porto (FEUP) is a Assistant Professor at the Department of Informatics of Engineering of FEUP and a researcher at INESC Porto. He’s a PSP Instructor and TSP Coach. He also was the supervisor of my Master’s thesis.MushtaqRaza is a PhD Student in MAP-I Doctoral Program at University of Porto. He also is an Assistant Professor at Abdul Wali Khan University in Pakistan, currently on study leave. His research interests include the Personal Software Process, Global Software Development and Usability Engineering.Just a little introduction of Strongstep:we are a company specialized in software engineering that contributes to the improvement of software quality in Portugal and in the world.In order to do that, Strongstep has a close relationship with reference institutions, such as SEI (Software Engineering Institute)and also with Universities such has FEUP.We have a great portfolio of success projects including CMMI for Development ML 5 in a Portuguese company. ML5 is the higher level of maturity that a company can achieve in the CMMI.
- Just a little introduction of Strongstep:we are a company specialized in software engineering that contributes to the improvement of software quality in the world.In order to do that, Strongstep has a close relationship with reference institutions, such as SEI (Software Engineering Institute of the Carnegie Mellon University) and also with Universities such has FEUP (Faculty of Engineering, University of Porto).We have a great portfolio of success projects including CMMI for Development ML 5 in a Portuguese company. ML5 is the higher level of maturity that a company can achieve in the CMMI.
- I concluded my Master’s Degree at FEUP in Informatics and Computing Engineering and, currently, I ama Consultant in Software Engineering at Strongstep. The research topic of my thesis was the Personal Software Process and I’ve recently been to Madrid at one of the biggest conferences in software engineering, called SEPG, presenting my research.ALT: As Karol was so kind to introduced me I will skip this slide about me.
- Theothertwoauthorsofthispaper are João Pascoal Faria andMustaqRaza.João Pascoal Faria, PhD in Electrical and Computer Engineering from the Faculty of Engineering of the University of Porto (FEUP) is a Assistant Professor at the Department of Informatics of Engineering of FEUP and a researcher at INESC Porto. He’s a PSP Instructor and TSP Coach. He also was the supervisor of my Master’s thesis.MushtaqRaza is a PhD Student in MAP-I Doctoral Program at University of Porto. He also is an Assistant Professor at Abdul Wali Khan University in Pakistan, currently on study leave. His research interests include the Personal Software Process, Global Software Development and Usability Engineering.
- Nowadays there is a great need for quality in software because it has become fundamental in our daily lives and is very important, if not essential, that software quality is as high as possible to ensure that no failures occur that could result in economic damage or even loss of human lives.This leads companies and organizations to strive to improve their software development processes since it is already accepted by all that the quality of the process directly affects the quality of the product.- Ler a frase.- This was written by James Harrington in his book "Business Process Improvement", and reflects the importance of metrics in the control, management and improvement of processes.[Dr. H. James Harringtonnow serves as theInternationalQualityAdvisor for Ernst & Youngand Chairman oftheBoardofEmergenceTechnologyLtd.Trabalhou na IBM durante 40 anos.Book:Business ProcessImprovement“...Itis as simple as that.”]
- What are the expected outcomes (or takeaways) of this presentation ?I hope that by the end of it you have understood the context of the research topic and the problem addressI would like to shed some light on the approach taken to tackle this problemAnd that you develop an understanding of the implementation and some concepts of the developed tool.Lastly I would like to share and discuss some of ideias with you of what could be improved
- High-maturity software development processes, making intensive use of metrics and quantitative methods, such as the Personal Software Process (PSP) and the Team Software Process (TSP), can generate a significant amount of data that can be periodically analyzed to identify performance problems, determine their root causes and devise improvement actions.-Manual analysis of this performance data is problematic because of the large amount of data to analyze and the time and expertise required.
- Putting TSP, PSP and CMMi in perspective. Basically CMMi is for the whole organization, TSP is for teams, and PSP is for individuals.PSPIs a High-maturity process framework for individuals that focus on improving individual skills and discipline.PSP makes use of a set of practices that enables engineers to build their own defined process; helps them to control, analyze and track it; and then improve it (their personal process) systematically It is commonly referred as a CMMi level 5 process for individualsTSP, on the other hand, focuses on team building and team managementIt adds a Project Management layer to the PSP with high-maturity practices for teams of PSP-trained individualsIt addresses performing Software Development using high performance inter-disciplinary work teams
- The amount of data generated by the use of these high maturity software development processes can be overwhelming, even for small projects and for experienced people.The time lost to properly analyze this data is huge, and although there are some tools to help us and guide in the analysis, most of this analysis is done manually.Despite PSP having some guidelines, you need a person with enough knowledge and experience to make a proper analysis of the data and obtain the respective improvement actions.
- The initial objectives were to:Develop a tool and respective model in order to automate the data analysis and recommend improvement actions.The model focuses on the analysis of time estimation performance in PSP but the tool is more generic and flexible, as I’ll show later on.The tool takes as input data gathered from projects using the PSP, interprets it by using the developed model and gives as output the corresponding improvement actions- The validation of the results obtained from the use of this tool was made using data from a document called Peformance Analysis Report (PAR). That is, we compared the results of the developed tool with the improvement actions derived manually by the engineers in this document.
- In the literature review we found a large number of tools for the PSP and TSP and plenty of them had automated data and metrics collection.The problem is that none really does an automatic analysis of the collected data, most of these tools only transforms the data into graphs for easier interpretation.In a briefmanner, thesetools are notfocusedontheintrepertationof data in order to giveimprovementactions.There are studies that show relationships between indicators and indicate recommended values. These studies were taken into account in our work.
- The performance model comprises: a set of performance indicators (PI) and cause-effect relationships, for identifying root causes of performance problems (in this case, time estimation problems); recommended limits for the PIs, for identifying performance problems; and a list of improvement actions for each leaf PI (that is, the bottom leaves of the performance model tree that will be shown in a couple of slides). The performance model was conceived based on PSP specifications (of base and derived measures, estimation methods, etc.), literature review, published analysis reports of PSP data, and opinion of PSP experts. We focused our attention on the time estimation performance because: delivering products on time is one of the most important goals in every project; one of the biggest strengths that is pointed out on the PSP/TSP is the delivery of products on time (or with comparatively little deviation) and virtually defect free (as low as 0.06 defects/KLOC) [1]; the time estimation performance is affected by other performance aspects (namely, quality and productivity), that have also to be taken into account. We intend in the future to build similar models for analyzing other PSP performance aspects, namely quality and productivity performance.
- In the PSP, the time estimation accuracy, or time estimation error (TimeEE), is defined as Actual Time minus Estimated time divided by the estimated time. In order to identify the factors that influence the behavior of this PI, one has to look at the PROBE estimation method used in the PSP. In this method, a time (effort) estimate is obtained based on a size estimate of the deliverable (in lines of code, function points, etc.), and a productivity estimate (in LOC/hour, FP/hour, etc.). So, the quality of the time estimate will depend on the quality of the size and productivity estimates. From the formulas of Size Estimation Error and Productivity Estimation and the definition of productivity as a size/time ratio, we got the formula on the blue box. Hence, we concluded that TimeEE is affected by SizeEE and PEE. I won’t get into details here but if you want to see all the calculations you may want to read my master thesis that is available online.
- So we already know that TimeEE is affected by SizeEE and PEE. Now bear with me as I explain our model and the relationships between the PI.In order to identify the factors that, in turn, influence the SizeEE, one has to understand further the PROBE method. In this method, in order to arrive at a size estimate of a deliverable, one has to: identify a list of parts that are needed to build the deliverable (usually depicted in a so-called conceptual design); define the type of each part (I/O, calculation, etc.); estimate the relative T-shirt size of each part (small, medium, large, etc.); use a so-called relative size table, based on historical data, to convert relative sizes to numerical sizes; sum up the numerical sizes to obtain a size estimate of the deliverable. Hence, possible causes for a poor SizeEE are: missing parts (parts that werenot planned but actually used); expurious parts (part that were planned but not actually used); and errors in the size estimates of the parts whose need was correctly identified. The dashed arrows indicate relationships between performance indicators that require further validation. The others, in continuous trait, were concluded by calculation method.
- In the PROBE method, productivity estimates are based on historical productivity, so their quality depends on the stability of the productivity, as indicated in the Figure. We calculate the productivity stability up to the project under analysis, based on the actual productivity of that project and past projects of the same developer.In the PSP, time is recorded per process phase (Plan, DLD-Detailed Design, DLDR-Detailed Design Review, Code, CR-Code Review, Compile, UT-Unit Test, and PM-Postmortem), so, productivity variations can be located in specific phases, reason why the productivity stability per phase is also considered in our model (Figure).It should be noted that the scope of the PSP is the development of individual programs or components, reason why Requirements, High Level Design and System Testing phases are not included, but can be found in the more complete TSP. We also ignore the Compile phase, since we intend to analyze projects developed with programming languages without a separate Compile phase (like Java).
- On the other hand, process changes can cause productivity disturbances: usually, after a process change, productivity decreases and later on returns to the original productivity or a new one.This is an important factor when analyzing PSP training data, because the process is changed along the training (from PSP0 to PSP1 and PSP2), and existing data from PSP courses show that productivity variations follow. Hence, we indicated Process Stability as potentially affecting Productivity Stability in the Performance tree. In the book PSP: A Self-Improvement Process for Software Engineers by Watts Humphrey, it is claimed that the effort for finding and fixing defects through reviews is much more predictable than through testing, because in the former case the review effort is proportional to the size of the product under review and defects are immediately located, whilst in the latter case the time needed to locate defects is highly variable. High defect density in tests is considered a common cause of predictability problems. Hence,we indicate in the performance model tree that the defect density in unit tests may affect productivity stability.
- In turn, the defect density in unit tests is affected by the total number of defects injected (and found) and the percentage of defects removed in reviews prior to testing, i.e., the review yield (a lagging indicator of the quality of detailed design and code reviews). In the same book and based on existing PSP data, Watts Humphrey claims that the time spent in reviewing a work product, measured both in relation with its size (review rate) or the time spent in developing it (review to development ratio), is a leading indicator of the review yield. A recent study on “The Impact of Design and Code Reviews on Software Quality” confirms that the PSP review rate is a significant factor affecting review yield; the recommended review rate of 200 LOC/hour or less was also found to be an effective rate for individual reviews, identifying nearly two-thirds of the defects in design reviews and more than half of the defects in code reviews. Another recent study in an industrial setting [15] shows an increase in inspection effectiveness when the review rate is reduced to a value closer to the recommended value. In “The Software Quality Profile” by Watts Humphrey, based on extensive data from the PSP, it is presented a profile of software processes that generally produce high quality products; among other aspects, design review time and code review time should be greater than 50% of design time and code time, respectively. Hence, we indicate in the performance model that the review yield may be affected by the review to development ratio and the review rate.
- As previouslystated, to developthistooltherewastheneed to automatethe performance analysis. Thiswasdoneusingour performance model.Using PSP, this analysis is manually performed and registered in the form of a document called Performance Analysis Report where an analysis of the data is carried out using graphs and tables created by the tool provided by SEI, arriving to a set of conclusions. These findings or conclusions give way to proposals of improvement actionsThe tool created to implement the developed performance model was named PSP PAIR, (it has same name as the paper)which comes from PersonalSoftware ProcessPerformance AnalysisandImprovementRecommendation.It was developed with the purpose of being simple and with an user friendly interface to save time in the data analysis, as we see here in this example. I'll show some more screenshots later on.The tool interprets and analyzes the data found in a database of the PSP student workbook (or one that follows the same format), and based on the developed performance model, it suggests improvement actions.
- Here we see an example of the results report, where it shows a tree of the performance model and control limits for each indicator, as well as its unit of measure. In this case two projects were chosen for analysis (project 411 and 412), as you can see in the table.For comparisonandanalysispurposes, the calculation of the results of all projects selected is done.In thecolumnAllselected, you can see the average and the percentage of values in that are red from the chosen set of projects (ie, that are outside the recommended control limits).In order to recognize performance problems, we defined control limits for classifying values of each PI into three categories: green - no performance problem; yellow – a possible performance problem; red – a clear performance problem. These limits are based on recommended values for PSP projects, present in the literature or orally stated by PSP instructors. Nevertheless, they can be easily configured by the users of our PSP PAIR tool. The toolallows to load other performance models, using XML to represent the tree and control limits. This feature was introduced so that potential improvements to the performance model could be quickly and effortlessly introduced into the tool.
- - Here you can see an example of some of the suggested improvements actions based on thevalues of the performance indicators and in theperformance model- This is more or less what you see when you open a project in the table of the previous slide for more detail. In this exemple weonly show some oftherecommendedactionssoitisreadable. -> Here we see the performance tree in more detail and it is noteworthy that the main indicator (Time Error Estimation) is with a potential performance problem since it is inside the yellow control limit.We can see that the PSP PAIR suggested improvement actions according to the problems identified in some indicators. These improvement actions are a work in progross but they were based on the judgment and advice of PSP experts and instructors and by taking into consideration the methods involved.
- For resultsassessmentpurposesa comparison was made between the problems detected in the PAR (Performance Analysis Report) and the problems detected by the tool in all the projects present in the PSP student workbook.We can see that the problems identified manually, on the left column of the table, are identical to those detected by PAIR PSP, on the bottom of the the right column of the table.
- Still in resultsassessment, a comparativeanalysiswasmadebetweenthe PIPs (Process Improvement Proposals) of the PAR document and the improvement actions recommended by the tool.One must have in mind that the PIPs obtained in the PAR analysis takes into consideration factors that the tool can not take into account, such as artifacts of the project and the experience of the person doing the analysis. The analysis of PSP PAIR is carried out solely on the basis of data collected by the engineer using the PSP methodology with the tool provided by SEI for this purpose.
- This is a comparative table of concepts, methodologies and tools (that support these the steps) and the Steps of our approach. We can see in orange the steps that the developedtool supports. We can also see that there is no tool, methodology or concept that supports all the required steps.The first two steps are related to the gathering and calculation of metrics and for that there already exists a large number of tools, such as the SEI Workbooks for PSP and TSP and the Process Dashboard. So, the challenge was to automate the remaining steps, which is done by PSP PAIR.
- Results suggest that the developed performance model is valid.The results also support the idea that PSP PAIR can be a good support tool for engineers who are using the PSP methodology in the development of their projects. Since this accelerates data analysis through automation and suggestions of improvement actions based on that analysis.Is worth noting that the architecture of the PSP PAIR gives space to the continuation of its development as it is well structered.
- As future work:- We can begin by further validating the performance model with a dataset larger than the one used. The reason for this is that SEI failed to provide the data on time, so a small datased of one PSP training was used. We're still waiting for this data from hundreds of students as wehope to use it to furthervalidateandmaybe improve the performance model.The improvement actions can have some sort of rating or ranking, in order to take into account several indicators with different weights which influence in different ways the final recommendation to the user.We want to make possible for the tool to import data from other sources, such as SQL or Excel. At this time it only supports Microsoft Access, which is the file type of the database that is used by SEI’s tool.The tool could be more effective in its recommendations if it included some kind of user feedback. This feedback would be used to improve future results and recommendations.Lastbutnotleast, we think that would be very useful for businesses and organizations if the PSP PAIR was extended to the Team Software Process (TSP).
- As presentationtakeways, I hopethatyouunderstoodtheproblemaddressedandthe research topic. Andalsotheapproachwetook to tacklethisproblem. In summary, wedeveloped a performance model to analyze time estimation performance using the large amount of data that is generated by engineers using PSP. Then we developed a tool using this model to help to agilize this analysis and suggest improvement actions.
- So, going back to the same quote by James Harrington from the initial slides, (LERFRASE).
- Thank you very much for your bearing with me. I would like to leave you with my contacts if you would like to contact me later on or if you are interested in Strongstep services! Thank you very much for your attention.