Slides for our presentation at The 29th IEEE Conference on Software Engineering Education and Training (CSEE&T) 2016 conference, Dallas, TX USA

1. PavelHeroutandPremekBrada
CSEE&T 2016, Dallas
UML-test Application for
Automated Validation of
Students’ UML Class Diagrams
Department of Computer Science and Engineering
University of West Bohemia
Pilsen, Czech Republic

2. 1st year undergraduates
~90 novice students, 1 experienced teacher
parallel Intro to programming (Java)
Prepare for routine programming
fluency in core O-O programming notions
hands-on knowledge of design patterns, collections
Get acquainted with associated practices
UML as ex-post code documentation
passive experience with automated testing
All needed downstream – algorithms, systems,
technologies
Context and teaching goals
UML-test for Automated Validation of Student's Diagrams 2

3. Factors
student numbers (90:1) => need for efficiency
limited initial skills => need for feedback and learning
Key parts of the approach
2hr lecture, detailed materials (books, slides, examples)
simple tools – BlueJ, UMLet
homework: 11 tasks, evolving complexity
combination of automated and manual assessment
small group learning and 1:1 only for those that need help
Automate the assessment more than usual
Teaching approach
UML-test for Automated Validation of Student's Diagrams 3

4. Part of a larger scheme. Goals:
filter out noise and common mistakes
still provide enough learning feedback
make assessment more objective and repeatable
UML diagram checking
class diagram only (basic code documentation)
presence/absence of important diagram elements
relation types, cardinality
NOT diagram layout and aesthetic
Process:
Automated UML Checking in KIV/OOP
UML-test for Automated Validation of Student's Diagrams 4
Create master diagram
Generate tests,
build tool
Check student
diagrams

5. Master diagram
UMLet
Testing tool
Eclipse + Maven project
in-house framework for
UXF -> JUnit code generation
build => uml-test-NN.jar + batch variant
Available tweaks
levels of testing (configuration), hide tests from students
stop-on-first / check all
UML checking: Preparatory phase
UML-test for Automated Validation of Student's Diagrams 5

6. Students hand in
University portal
Validator runs testing .jar
(set up by teacher first)
on each hand-in
displays feedback incl. errors
Results linked with
detailed error description list
Teacher grades using .jar or batch.jar (no hidden tests)
UML checking: Testing tool use
UML-test for Automated Validation of Student's Diagrams 6

7. First year of full deployment
Teacher observations
system lets us analyse if/how students improve
unique error codes => statistics
scales to small dozens of elements
Data hint at student learning curve
ratio errors:tests
number of attempts before success
Students: „Even though a programming novice, I do not
regret signing up for the course in my first semester. Do not
give up at the beginning.“
Experience
UML-test for Automated Validation of Student's Diagrams 7

8. Inspiration and motivation
efficiency, ICPC, geeky nature ;-)
Benefits
efficiency while maintaining baseline quality and feedback
data to analyse (potentially time series)
Costs and weak aspects
effort to develop and maintain supporting tools
not for everyone (adoption levels across the department)
student attitude „program to pass unit tests“
tool feedback does not (most cases) tell how to improve
Discussion
UML-test for Automated Validation of Student's Diagrams 8

9. Automated UML class diagram verification is possible
students learn correct code documentation
teacher(s) assess efficiently
Scope needs to be managed
introductory year, large student populations, few teachers
=> assignment count and complexity
simple tools (UMLet + JUnit + test generation framework)
Positive experiences
Work needed on tool & data analysis improvements
Conclusions
UML-test for Automated Validation of Student's Diagrams 9