This document describes a system called ORVViS that was developed at the University of Zagreb to validate student source code submissions and detect plagiarism. ORVViS integrates various validation tools like HTML Tidy, CSSutils, and Sherlock to check submissions for several courses. Initial results found some plagiarized assignments which decreased after informing students. Over time, submissions showed fewer errors as students' skills improved. ORVViS provided detailed validation reports and helped reduce forum questions. Future work includes fully integrating it with the learning management system and adding semantic validation plugins.

1. Source code validation and plagiarism detection:
technology-rich course experiences
Ivana Bosnić
Branko Mihaljević
Marin Orlić
Mario Žagar
University of Zagreb, Croatia

2. Who we are?
University of Zagreb (UniZG)
Faculty of Electrical Engineering
and Computing (FER)
Computer Systems
and Processes Group (RASIP)

3. What do we do?
Open systems Web technologies
Software engineering Enhancing education

4.  Open
 Systems
 Technologies
 Standards
 Culture
 Licences

5. Motivation

6. Open Computing
 50 - 100 students
 3rd year
 Undergraduate study of Computing
 Topics:
 Open systems, client-side technologies
 Server-side (open) technologies
 Integration

8. On a practical side...
6 course assignments
 Quickhands-on experience
 Loosely defined tasks
• Final result: web application with search capabilites
 Versions:
• DVD store, library, phone book...
 Various open technologies
• Integration!

9. On a practical side...
 Open technologies used
1. HTML, CSS
2. XML, XSL, DTD/XSD
3. PHP, DOM
4. Java, XML
5. Java Servlets
6. JavaScript, AJAX
 Upload & discuss...

10. The challenges
 Students  Staff
 Lack of student  Answering beginners’
experience in OOP questions
 Fast-changing  Preventing invalid
technologies submission files
• information dispersion  Ensuring solution
 Detailed configuration structure and standard
instructions missing  Reducing plagiarism
 Unexperienced:
not enough time

11. ORVViS

12. ORVViS - implementation
 Assignment validation system
 Web application in PHP
 Validation core & plugins
 Plagiarism detection – Sherlock, BOSS
 Integration with Moodle LMS

13. Validators used
 HTML:  PHP:
 HTML Tidy  internal
 CSS:  JavaScript:
 Cssutils  JavaScript Lint
 XML, DTD, XSL:  Java:
 DOM, Matra  PMD

14. ORVViS use case
Setup Validation Plagiarism

16. Results & experiences

17. Plagiarism detection
 2006/07
 Pilot period for system testing
 2007/08
 113 students enrolled
 Informed students about our intentions 

18. Plagiarism detection
 1st assignment
1 pair of similar solutions – almost the same
 3rd assignment
1 pair of similar solutions – 80% similarity
0% 80%

19. Plagiarism detection
 4th assignment – Java
6 cases altogether – 17 students 
 Steps taken 
 Later years:
 No significant similarities

20. Plagiarism detection - experiences
 Inform students in advance
 it is somehow “fair”
 Show the technology and results
 to show you’re serious
 Perservere in analysis and decisions
 as some iterations are needed

21. Offtopic: Structure validation
 2007/08
 52% - incorrectly named archives
 10% - packaged as RAR instead of ZIP
 25% - incorrect file structure
 4% - incorrectly named files
 Yes, the structure IS important

22. Assignment validation
 2009/10
 70 students
 2010/11
 53 students
 Comparison of
 number of submission instances
 validation results

23. Total/final submission instances
 submitting the solution multiple times allowed
 similarity between years

24. Submission instances
 First 2 assignments
 high number of submissions:
• ~5 per student
 3rd assignment (PHP)
 straightforward, one technology
• ratio drops: ~2,4 per student
 4th assignment (Java)
 steeplearning curve
 submsissions increase again

25. Validation
results
 2009/10
 2010/11

26. Validation results
 Relates to the Submissions
 first 3 assignments:
• fully correct submissions increase
• total number of submission decreases
 4th and 5th assisgnments (Java, servlets)
• fully correct submissions – big decrease
 6th assingment
• back to “normal”

27. ORVViS experiences
 Easier to analyze the assignments
 validation chart + error logs
 Detailed report on each submission
 before the discussions
 Help to ensure the real code was submitted
 no “pancake” recipes 
 Number of simple forum questions reduced
 better focus on more serious stuff

28. ORVViS experiences
 Integrated validators in LMS
 easier to use, due to a number of technologies
 Indirect help of validators
 to configure the environmnent (XML)
 Level of validator sensitivity
 good hints with warnings, when using
the different servers and environments

29. Future - ORVViS
 Full LMS integration
 no need for external administration
 2.x APIs
 Plugins for verifying semantic properties
 Increase the students’ awareness on testing
 Reflection-in-action

30. www.fer.hr/predmet/or
ivana . bosnic @ fer . hr