COLMENA: Assistance in computer programming learning using educational data mining and learning analytics ITICSE 2013. 1st-3th July. Canterbury

1. COLMENA: Assistance in computer
programming learning using educational data
mining and learning analytics
ITICSE 2013
Carlos Fernández Medina
Juan Ramón Pérez-Pérez
Mª Puerto Paule-Ruiz
Víctor Manuel Álvarez-García
Pulso research team · University of Oviedo · www.pulso.uniovi.es
miércoles 10 de julio de 13

2. Lahtinen, E. et al. (2005)
MOTIVATION
How much do students know about their
own learning process?
How much do teachers know about the
learning process of their students?
Programming is DIFFICULT for students
miércoles 10 de julio de 13

3. ASSISTANCE
• Programming tasks require some
ASSISTANCE.
• Traditionally: unaid environments (notepad, pen
and paper).
• Integrated Development Environments (IDEs),
do they give us assistance?
• Some implementations are oriented to
learning (BlueJ)
• Most are not (Eclipse,Visual Studio, NetBeans)
• IDE + assistance = improvement of Teaching-
Learning process
Zeller (2007)
Jadud (2006)
miércoles 10 de julio de 13

4. GOAL
To enhance knowledge about the
programming learning process for
teachers and students
Generate feedback
Mory, E.H. (2004)
miércoles 10 de julio de 13

5. LEARNING
ANALYTICS
DATA
MINING
SUPPORT &
GUIDANCE
teachers &
PROGRAMMING
LEARNING
ACTIVITIES
students
COLMENA
miércoles 10 de julio de 13

6. Fernández-Medina, C. et al. (2011)
Glahn, C. et al. (2008)
PROPOSED MODEL
miércoles 10 de julio de 13

7. •Collect compiler errors and
warnings
•Each compilation/save event
starts data collection
EXTRACTION LAYER
miércoles 10 de julio de 13

8. EXTRACTION LAYER
miércoles 10 de julio de 13

9. • Redundances filter
• Discard errors present in last
collection
• (Unless error line has been edited)
• Persistence mechanisms
FTP, DB, files
PROCESSING LAYER
miércoles 10 de julio de 13

10. PROCESSING LAYER
miércoles 10 de julio de 13

11. compiler family
Compiler
Taxonomy
explanation
and hints
global group
of errors
The
output
error id concept
Ben-Ari, M. (2007)
conceptual family
global group
of errors
ANALYSIS LAYER
miércoles 10 de julio de 13

12. ANALYSIS LAYER
miércoles 10 de julio de 13

13. error
conceptfamilies
Teachers
Students
VISUALISATION LAYER
miércoles 10 de julio de 13

14. • 43 students
• Algorithms, CS2, 6 weeks
• dynamic programming, backtracking, branch
and bound, divide and conquer
• OOP basics
• 1 practical lesson = 1 session
• Eclipse + Java
APPLICATION CONTEXT
Study of processed errors and creation of concepts
miércoles 10 de julio de 13

15. Error/Warning
FamilyFamily
ConceptError/Warning
Compiler error Conceptual
Concept
Value of field/local variable X is not used Internal Type Warning
A variable or field was declared but never was used
(Variables Definition, Scope of Variables)
Syntax error, insert X to complete statement. Syntax Syntax Error
Incomplete code structure, is missing a final semicolon,
bracket, etc. (Code Blocks usage)
X cannot be resolved to a variable Field Syntax Error
A used variable is never declared before (Variables Definition,
Scope of Variables)
X cannot be resolved to a type Type Syntax Error
It is trying to use a variable of a type that does not exist
(Variables Definition, Import Usage)
Type mismatch: cannot convert from X to X Type Semantic Error
Possible loss of precision between variables (Types,
Conversion of Types)
X is a raw type. References to generic type X<T>
should be parameterized
Type Genericity Warning
X interface is used but it is never parameterized with the type
of the object that implements the interface (Genericity, Types,
Types Checking)
The import X is never used Import Import Warning A class not existent in the package or project was imported
References to generic type Y<T> should be
parameterized
Type Genericity Warning
Method X returns a generic type. X return type should be
parameterized
The method X(parameters) is undefined for the
type Y
Method Semantic Error Method X is called, but it is not declared in the Class Y
º
miércoles 10 de julio de 13

16. syntax errors decrease
errors depend of the task type
errors related to variables are most frequent.
0
10
20
30
40
50
Field Syntax Type Import Method Constructor Internal
1 2 3 4 5 6
Sessions
miércoles 10 de julio de 13

17. • Most IDEs provide insufficient information to
adequately support the learning of programming.
• We propose a flexible assistance model for
programming learning
• Our model collects and provides quantitative and
qualitative feedback on students’ programming
learning experiences
• Teachers can use their knowledge to facilitate the
learning process.
TO SUM UP
miércoles 10 de julio de 13

18. FUTURE WORK
Combine more variables and indicators of
the programming learning process
Analyse student behavioural patterns
Make it available to students
miércoles 10 de julio de 13

19. COLMENA: Assistance in computer
programming learning using educational data
mining and learning analytics
Carlos Fernández Medina
carlosfernandezmedina@gmail.com
http://www.pulso.uniovi.es
miércoles 10 de julio de 13