Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Developing and Evolving a DSL-Based Approach for Runtime Monitoring of Systems of Systems

444 vues

Publié le

Complex software-intensive systems are often described as systems of systems (SoS) due to their heterogeneous architectural elements. As SoS behavior is often only understandable during operation, runtime monitoring is needed to detect deviations from requirements. Today, while diverse monitoring approaches exist, most do not provide what is needed to monitor SoS, e.g., support for dynamically defining and deploying diverse checks across multiple systems. In this talk, I will describe our experiences of developing, applying, and evolving an approach for monitoring an SoS in the domain of industrial automation software, that is based on a domain-specific language (DSL). I will first describe our initial approach to dynamically define and check constraints in SoS at runtime, including a demo of our monitoring tool REMINDS, and then motivate and describe its evolution based on requirements elicited in an industry collaboration project. I will furthermore describe solutions we have developed to support the evolution of our approach, i.e., a code generation approach and a framework to automate testing the DSL after changes. We evaluated the expressiveness and scalability of our new DSL-based approach using an industrial SoS. At the end of the talk, I will also present general lessons we learned and give an overview of other projects in the area of software monitoring as well as other areas such as software product lines, that I am currently involved in.

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Developing and Evolving a DSL-Based Approach for Runtime Monitoring of Systems of Systems

  1. 1. Developing and Evolving a DSL-Based Approach for Runtime Monitoring of Systems of Systems Priv.-Doz. Dr. Rick Rabiser Feb 7, 2019 – AAU Klagenfurt Christian Doppler Laboratory MEVSS Institute for Software Systems Engineering Johannes Kepler University Linz, Austria rick.rabiser@jku.at http://mevss.jku.at/rabiser https://twitter.com/RickRabiser
  2. 2. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser About me • 2016 (Jun) Habilitation in Practical Computer Science • 2013-today: Senior Researcher/Module Head, CDL MEVSS (http://mevss.jku.at) • 2009-2013: Post-Doc, CDL ASE (http://ase.jku.at) • 2006-today: Lecturer at JKU • PAT („Präsentations- und Arbeitstechnik“) • Software Engineering • Product Line Engineering • 2009 (May): PhD: A User-Centered Approach to Product Configuration in Software Product Line Engineering 2
  3. 3. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Disclaimer • This is work by many people, not only by me, thanks particularly go to the MEVSS M1 Team (current and former members): Paul Grünbacher, Thomas Krismayer, Lisa Kritzinger, Dario Romano, Jürgen Thanhofer-Pilisch, and Michael Vierhauser • Our industry partner Primetals Technologies • And collaborators 3
  4. 4. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser What you can expect • Motivation (Systems of Systems Monitoring) • The REMINDS Approach (Model, Framework, and Tool Support) • Developing and Evolving a DSL-based Approach for Runtime Monitoring of Systems of Systems • Lessons Learned • Other/Ongoing Work 4
  5. 5. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Monitoring Systems of Systems (SoS) 5 • SoS are big, heterogeneous, and complex and comprise operationally independent systems  Difficult to understand and analyze • Full behavior emerges only at runtime  Runtime monitoring is required • Continuous evolution and evolutionary independence  Checking compliance of behavior to requirements is essential [Maier 1998] [Nielsen et al. 2015] [Robinson 2006] [Maiden 2013] [Vierhauser et al. 2016] [Rabiser et al. 2017]
  6. 6. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Level 2 (process optimization) Level 1 (process control) Level 3 (production planning systems) HMI Database Customer and Third- party Systems PLC Iron Steel CastingI-HMI S-HMI C-HMI PLC I-DB S-DB C-DB Monitoring Systems of Systems (SoS) 6 • SoS are big, heterogeneous, and complex and comprise operationally independent systems  Difficult to understand and analyze • Full behavior emerges only at runtime  Runtime monitoring is required • Continuous evolution and evolutionary independence  Checking compliance of behavior to requirements is essential [Maier 1998] [Nielsen et al. 2015] R R R RR R R [Robinson 2006] [Maiden 2013] [Vierhauser et al. 2016] [Rabiser et al. 2017]
  7. 7. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Requirements Monitoring Model 7 Sc1: PAS (SoS) Sc3: Iron (S) Sc5: Caster (S) Sc6: Discharge (C) Sc9: HMI (C) Sc2: Lab System (S) Sc10: C- Archiving (C) L2I (SI) H2A (CI) Sc4: Steel (S) Sc8: Laddle Tracking (C) Sc7: S- Archiving (C) T2A (CI) Casting HMI.request Optimization HMI Optimizer Mat. Tracking Cooling HMI.setData HMI.request Length HMI.request Speed Cooling.initN ozzles HMI.restrict Metal Cooling.setM ode Optimizer.fini shed HMI.request Optimization HMI.request Optimization SoS Events SoS Monitoring Scopes SoS Requirements SoS Syst em ... ... ... ... ......Component Requireme nt ... Constraint ... Requireme nt ... Requireme nt ... Constraint ... Probe Event Probe Event Probe Event Probe Event Comp. Interacti on M. Vierhauser, R. Rabiser, P. Grünbacher, and B. Aumayr, A Requirements Monitoring Model for Systems of Systems, RE 2015.
  8. 8. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Framework 8 Aggregation & Distribution PersistorEvent Broker Probe Templates Persistence Distribution Unified Event Model Aggregation Processing & Analysis Variability Manage- ment System of Systems Probe Probe Probe Probe Probe Probing & Instrumentation Views Event AnalysisConstraintsProcessors View View View View View Core Layers Extension Layers M. Vierhauser, R. Rabiser, P. Grünbacher, K. Seyerlehner, S. Wallner, and H. Zeisel, ReMinds: A Flexible Runtime Monitoring Framework for Systems of Systems, JSS 2016. REquirements Monitoring Infrastructure for Diagnosing Systems of Systems
  9. 9. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Framework 9 Aggregation & Distribution PersistorEvent Broker Probe Templates Persistence Distribution Unified Event Model Aggregation Processing & Analysis Variability Manage- ment System of Systems Probe Probe Probe Probe Probe Probing & Instrumentation Views Event AnalysisConstraintsProcessors View View View View View Core Layers Extension Layers Casting HMI.request Optimization HMI Optimizer Mat. Tracking Cooling HMI.setData HMI.request Length HMI.request Speed Cooling.initN ozzles HMI.restrict Metal Cooling.setM ode Optimizer.fini shed HMI.request Optimization HMI.request Optimization HMI C# Probe Optimizer Java Probe Iron BF C++ Probe DB Trigger Probe L1 Interf. Probe M. Vierhauser, R. Rabiser, P. Grünbacher, K. Seyerlehner, S. Wallner, and H. Zeisel, ReMinds: A Flexible Runtime Monitoring Framework for Systems of Systems, JSS 2016. Focus of this talk REquirements Monitoring Infrastructure for Diagnosing Systems of Systems
  10. 10. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Tool Support 10 Key capabilities • Show system status • Visualize monitored events and data • Highlight violations • Visualize trends and statistics • Support investigating violations Tool Demo https://youtu.be/iv7kWzeNkdk L. M. Kritzinger, T. Krismayer, M. Vierhauser, R. Rabiser, and P. Grünbacher: Visualization Support for Requirements Monitoring in Systems of Systems, ASE 2017.
  11. 11. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Defining and Checking Constraints Rick Rabiser, Jürgen Thanhofer-Pilisch, Michael Vierhauser, Paul Grünbacher, and Alexander Egyed: Developing and Evolving a DSL-Based Approach for Runtime Monitoring of Systems of Systems, Automated Software Engineering, pp. 1-41, 2018. 11 DOI: 10.1007/s10515-018-0241-x
  12. 12. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Event-based Monitoring 12 Optimizer_start Optimizer_finished Cutting_completed cutlength=4.75 t
  13. 13. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Types of Constraints 13 trigger event past events future events Data Conditions The cutlength must not be > maxtransportable
  14. 14. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Types of Constraints 14 trigger event past events future events Data Conditions Past Occurrence When the tundish car moves to the casting position the ladle must have been mounted within the last 30 seconds
  15. 15. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Types of Constraints 15 trigger event past events future events Data Conditions Past Occurrence Future Occurrence
  16. 16. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Architecture 16 Event Model Probes (instrumenting running system) Constraint Engine Constraint Instance Store CI CI CI CI Constraint Definition Compiled Constraint Definition EM Facade Runtime Error Manager Error Handler Constraint Manager DSL Editor Eval. Delay Manager [Vierhauser, ASE 2010] & [Egyed, ICSE 2007] M. Vierhauser, R. Rabiser, P. Grünbacher, and A. Egyed: Developing a DSL-Based Approach for Event- Based Monitoring of Systems of Systems: Experiences and Lessons Learned, ASE 2015 and SE 2016.
  17. 17. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Constraint Engine Eval. Delay Manager Event Model Probes (instrumenting running system) Constraint Instance Store CI CI CI CI Constraint Definition Compiled Constraint Definition EM Facade Runtime Error Manager Error Handler Constraint Manager DSL Editor Architecture 17 End-user definition of constraints Constraint diversity Incremental definition Runtime management
  18. 18. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser “Evolution is what happens while you are busy making other plans” • When monitoring other systems, new requirements emerged • Composite constraints • Multiple data checks in one constraint • Cross-event data access • Multiple data checks for one event • Event-based evaluation criteria • Multiple evaluation criteria • Flexible event sequences • Optional events and negations • Additional tool support (violation explanations) • Also, usability study (Rabiser et al. ICSE SEIP 2016) with industrial end users revealed usability flaws 18 composite constraint = constraint "A" or constraint "B" or constraint "C" trigger = if event "Tap_Start" occurs condition = events !"Slag_Start", ?"Tap_Analysis", "Tap_End" occur in any order within 3 minutes
  19. 19. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Evolving our DSL-based Approach • To address issues and integrate new features, we adapted: • DSL Grammar • Editor (generated) • Compiler • Engine • Error Handler • Tool Views • To increase maintainability and extensibility • we reengineered the code generation component to better support future changes • To facilitate better testability and reliability • we introduced an automated testing framework 19 Constraint DSLGenerator Matcher Generator Event Generator Data Generator Util Generator Code Generation Snippets (Xtext / Xtend) Output (Java Code) Constraint Checker Sequence Checker Evaluation CritQueue Evaluation Criteria CDSL Model CDSLEvent Model EventModel Facade Data Checker DataDef Model Error Generator ErrorReporting Model Dynamic Constraint Template Template Generated Code (managable method accessing an extensible model to execute the check) Input (Constraint)
  20. 20. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Two “extreme” examples of the new DSL features trigger = if event "Optimization_Start" as "trigger" occurs condition = events "Fetch_Data" where data("meta−data", "run−id") == "trigger".data("meta−data", "run−id") AND data("meta−data", "cut−point") == "trigger".data("meta−data", "cut−point"), ? "Optimize_Run" where data("meta−data", "run−id") == "trigger".data("meta−data", "run−id") AND data("meta−data", "cut−point") == "trigger".data("meta−data", "cut−point"), "Optimization_End" where data("meta−data", "run−id") == "trigger".data("meta−data", "run−id") AND data("meta−data", "cut−point") == "trigger".data("meta−data", "cut−point"), occur ignore others until "Optimization_Start" occurs 20 trigger = if event "Tap_End" occurs condition = events "Tap_Analysis" where data("analysis" , "type") == "Metal" AND data("analysis" ,"temperature") > 1690, "Tap_Analysis" where data("analysis" , "type") == "Slag" AND within( data("analysis", "thickness"), 10%, 2.5) occur in any order within 5 minutes checking multiple data items of an Iron tap analysis checking a sequence of events in a cyclic optimization process
  21. 21. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Testing Framework: Writing DSL Tests 21 @Test public void MF_within3sec_SeqOrderViol() throws Exception { // create constraint String constrID = "MF_within3sec_SeqOrderViol"; IConstraintGenerator cGen = getConstraintGenerator(constrID); ITestConstraint constraint = cGen.generateConstraint(ITestConstraint.CONSTRAINT_TYPE.FUTURE) .setTriggerEvent(cGen.generateConstraintEvent("Iron.TapStart")) .setEventsToCheck(cGen.generateConstraintEventSequence("Iron.SlagStart", "Iron.TapEnd")) .setWithinTime("3 seconds"); // define events for simulation run List<Event> fireEventSequence; synchronized (ParallelViolationTestRunner.SEQUENCE_CREATION_LOCK) { fireEventSequence = cGen.getEventGenerator().createSequence(IEventSequenceGenerator.Seq_IDs.IRON_TS_TE_SS_2SEC); } // define expected errors Set<IExpectedError> expected = new HashSet<>(); expected.add(ExpectedViolationFactory.createViolation(fireEventSequence.get(0), fireEventSequence.get(1), ViolationErrorType.WRONG_TYPE)); addExpectedViolations(constrID, expected); // execute the test executeTestCase(constrID,constraint.getConstraintCode(),fireEventSequence); }
  22. 22. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Evaluation v.1 22 • Workshops & interviews with Primetals architects and engineers • Analyzed technical specification documents PAS RMM Element # Scopes 21 Reqs/Const. 40 Event-Types 109 Probes 22 Approach ScalabilityDSL Expressiveness Evaluation Runs Element 6h Run 168h Run Active Scopes 5 5 Checks 10,572 262,979 Events Captured 12,484 363,491 Active Probes 14 14
  23. 23. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Evaluation v.2 (13 v.1 constraints + 8 v.2) 23 Every CDSL version 2 constraint that could be compared to its version 1 pendant, except CST-05, became (at least slightly) faster. CST-05 performed worse than in CDSL version 1 because we now collect more violation information, which particularly affects this constraint.
  24. 24. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Lessons Learned • Systems of systems require an iterative language design • Keep the YAGNI (“you aren’t gonna need it”) principle in mind when developing a DSL • Simplify and automate extending the DSL • Keep the mapping of constraint DSL to constraint engine flexible • Runtime monitoring requires dynamic constraint management • Design the DSL to keep the effort for adaptations minimal • Consider backward compatibility • Automated testing is crucial 24
  25. 25. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Summary 25 Key Take-Aways wrt DSLs and their Evolution • YAGNI-Principle + Plan for future extensions • Automate, automate, automate (generation of editor, testing, etc.) • Get early and frequent feedback Key Take-Aways wrt DSLs and their Evolution • YAGNI-Principle + Plan for future extensions • Automate, automate, automate (generation of editor, testing, etc.) • Get early and frequent feedback
  26. 26. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Other Work (Completed, Ongoing, Future) Usability Assessments of SE Tools (ICSE SEIP’16) 26
  27. 27. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Other Work (Completed, Ongoing, Future) Usability Assessments of SE Tools (ICSE SEIP’16) Monitoring (CDL MEVSS) • Monitoring Cyber-physical Systems (UAVs, Jane Cleland-Huang/Michael Vierhauser; SEAA’18) 27
  28. 28. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Other Work (Completed, Ongoing, Future) Usability Assessments of SE Tools (ICSE SEIP’16) Monitoring (CDL MEVSS) • Monitoring Cyber-physical Systems (UAVs, Jane Cleland-Huang/Michael Vierhauser; SEAA’18) • Constraint Mining (Dissertation Thomas Krismayer; ASE’17/18, SAC’19, REFSQ’19) 28
  29. 29. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Other Work (Completed, Ongoing, Future) Usability Assessments of SE Tools (ICSE SEIP’16) Monitoring (CDL MEVSS) • Monitoring Cyber-physical Systems (UAVs, Jane Cleland-Huang/Michael Vierhauser; SEAA’18) • Constraint Mining (Dissertation Thomas Krismayer; ASE’17/18, SAC’19, REFSQ’19) • Monitoring RoboCode (Dario Romano, https://youtu.be/Yqcr28OvLaw) 29
  30. 30. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Other Work (Completed, Ongoing, Future) Usability Assessments of SE Tools (ICSE SEIP’16) Monitoring (CDL MEVSS) • Monitoring Cyber-physical Systems (UAVs, Jane Cleland-Huang/Michael Vierhauser; SEAA’18) • Constraint Mining (Dissertation Thomas Krismayer; ASE’17/18, SAC’19, REFSQ’19) • Monitoring RoboCode (Dario Romano, https://youtu.be/Yqcr28OvLaw) • Comparison Framework for Monitoring Approaches (with Luciano Baresi, Sam Guinea, and Michael Vierhauser; JSS 2017) • Domain Model and Reference Architecture for Software Monitoring (so far: Requirements & Resource Monitoring; with Klaus Schmid, Michael Vierhauser and Sam Guinea; under revision in IST) 30
  31. 31. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Other Work (Completed, Ongoing, Future) Usability Assessments of SE Tools (ICSE SEIP’16) Monitoring (CDL MEVSS) • Monitoring Cyber-physical Systems (UAVs, Jane Cleland-Huang/Michael Vierhauser; SEAA’18) • Constraint Mining (Dissertation Thomas Krismayer; ASE’17/18, SAC’19, REFSQ’19) • Monitoring RoboCode (Dario Romano, https://youtu.be/Yqcr28OvLaw) • Comparison Framework for Monitoring Approaches (with Luciano Baresi, Sam Guinea, and Michael Vierhauser; JSS 2017) • Domain Model and Reference Architecture for Software Monitoring (so far: Requirements & Resource Monitoring; with Klaus Schmid, Michael Vierhauser and Sam Guinea; under revision in IST) Variability Management/Software Product Lines • Teaching about SPLs (with Mathieu Acher and Roberto Lopez-Herrejon; ACM TOCE 2017) 31
  32. 32. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser Other Work (Completed, Ongoing, Future) Usability Assessments of SE Tools (ICSE SEIP’16) Monitoring (CDL MEVSS) • Monitoring Cyber-physical Systems (UAVs, Jane Cleland-Huang/Michael Vierhauser; SEAA’18) • Constraint Mining (Dissertation Thomas Krismayer; ASE’17/18, SAC’19, REFSQ’19) • Monitoring RoboCode (Dario Romano, https://youtu.be/Yqcr28OvLaw) • Comparison Framework for Monitoring Approaches (with Luciano Baresi, Sam Guinea, and Michael Vierhauser; JSS 2017) • Domain Model and Reference Architecture for Software Monitoring (so far: Requirements & Resource Monitoring; with Klaus Schmid, Michael Vierhauser and Sam Guinea; under revision in IST) Variability Management/Software Product Lines • Teaching about SPLs (with Mathieu Acher and Roberto Lopez-Herrejon; ACM TOCE 2017) • Academic vs. Industrial SPL Research (with Klaus Schmid, Martin Becker, Goetz Botterweck, Matthias Galster, Iris Groher, and Danny Weyns; SPLC’18) 32
  33. 33. Feb 2019 | CD-Lab MEVSS | http://mevss.jku.at/rabiser | https://twitter.com/RickRabiser References • R. Rabiser, S. Guinea, M. Vierhauser, L. Baresi, P. Grünbacher: A Comparison Framework for Runtime Monitoring Approaches, Journal of Systems and Software, vol. 125, pp. 309-321, 2017. • R. Rabiser, K. Schmid, M. Becker, G. Botterweck, M. Galster, I. Groher, D. Weyns: A Study and Comparison of Industrial vs. Academic Software Product Line Research Published at SPLC, SPLC 2018. • R. Rabiser, J. Thanhofer-Pilisch, M. Vierhauser, P. Grünbacher, A. Egyed: Developing and Evolving a DSL-Based Approach for Runtime Monitoring of Systems of Systems, Automated Software Engineering, vol. 25(4), pp. 875-915, 2018. • R. Rabiser, M. Vierhauser, P. Grünbacher: Assessing the Usefulness of a Requirements Monitoring Tool: A Study Involving Industrial Software Engineers, ICSE (SEIP) 2016. • M. Vierhauser, J. Cleland-Huang, S. Bayley, T. Krismayer, R. Rabiser, P. Grünbacher: Monitoring CPS at Runtime – A Case Study in the UAV Domain, SEAA 2018. • M. Vierhauser, R. Rabiser, P. Grünbacher: Monitoring Requirements in Systems of Systems, IEEE Software, vol. 33(5), pp. 22-24, 2016. • M. Vierhauser, R. Rabiser, P. Grünbacher, B. Aumayr, "A Requirements Monitoring Model for Systems of Systems, RE 2015. • M. Vierhauser, R. Rabiser, P. Grünbacher, K. Seyerlehner, S. Wallner, H. Zeisel: ReMinds: A Flexible Runtime Monitoring Framework for Systems of Systems, Journal of Systems and Software, vol. 112, pp. 123-136, 2016. • T. Krismayer, R. Rabiser, P. Grünbacher: Mining Constraints for Event-based Monitoring in Systems of Systems, ASE 2017. • T. Krismayer, R. Rabiser, P. Grünbacher: Mining Constraints for Monitoring Systems of Systems, SAC 2019. • T. Krismayer, P. Kronberger, R. Rabiser, P. Grünbacher: Supporting the Selection of Constraints for Requirements Monitoring from Automatically Mined Constraint Candidates, REFSQ 2019. • L. M. Kritzinger, T. Krismayer, M. Vierhauser, R. Rabiser, P. Grünbacher: Visualization Support for Requirements Monitoring in Systems of Systems, ASE 2017. • M. Acher, R. Lopez-Herrejon, R. Rabiser: Teaching Software Product Lines: A Snapshot of Current Practices and Challenges, ACM Transactions on Computing Education, vol. 18(1), pp. 2:1-2:31, 2017. 33

×