The document discusses using learning analytics to support just-in-time teaching. It identifies problems that emerge in highly interactive lab sessions through literature reviews and experiments. Requirements for an in-class live analytics system were determined based on input from teachers, students, and experts. The system was developed and tested over multiple experiments and provided real-time analytics and awareness tools to help teachers manage sessions and provide personalized feedback to students. Evaluation found it helped address initial problems and contributed to the field of learning analytics and orchestration in technology-enhanced learning environments.

1. Learning Analytics
support for
Just-in-time Teaching
Israel Gutiérrez
Jornadas GAST - 8 de julio de 2013

2. Just-in-time
Teaching

3. Awareness

4. Learning
Analytics

5. Lab sessions
photo: berkeleylab at flickr

6. Research
Questions

7. Research
Questions
“What are the problems that emerge in highly interactive
face-to-face sessions?”

8. Research
Questions
“Could a just-in-time teaching system be used to overcome
the previous problems, facilitating the teacher
management of the session driving it to the intended
learning objectives and improving the efficiency of the
students allowing them to focus on the assignment?”
“What are the problems that emerge in highly interactive
face-to-face sessions?”

9. Methodology
Design-based research

10. Methodology
Design-based research
• Based on theory but pragmatic

11. Methodology
Design-based research
• Based on theory but pragmatic
• Real-world setting

12. Methodology
Design-based research
• Based on theory but pragmatic
• Real-world setting
• Iterative process

13. Methodology
Design-based research
• Based on theory but pragmatic
• Real-world setting
• Iterative process
• Mixed methods

14. Methodology
Design-based research
• Based on theory but pragmatic
• Real-world setting
• Iterative process
• Mixed methods
• Research agenda updated with findings

15. Problems in
lab session
Literature Experiments
Experts

16. Literature
Lantern (Alavi & Dillenbourg, 2009)
PLITAZ (Dong & Hwang, 2012)
Problem Source Stakeholder
Demand an answer Alavi09 Students
Wait longer Alavi09 Students
Fails to notice requests Alavi09 Teachers
Heavy workload Dong12 Teachers
Isolation Dong12 Students
Limited awareness Dong12 Teachers
Simultaneous requests Dong12 Teachers

17. AMM11 (Gutiérrez Rojas, Crespo García, & Delgado Kloos, 11)
AS11 (Gutiérrez Rojas & Crespo García, 12)
AMM12 (Gutiérrez Rojas, Crespo García, & Delgado Kloos, 12)
Problem Source Stakeholder
Methodology increases workload AMM11 Teachers
Huge number of requests AS11 Teachers
Attention to personalized feedback AS11 Students
Ask without thinking AS11 Students
Shorten explanation if overwhelmed AS11 Teachers
Perceive not enough teachers AS11 Students
Attention order unfairness AS11 Teachers
Devote time monitoring the teacher AS11 Students
Not aware of time AS11 Teachers
Not like writing questions AMM12 Students
Not relevant not current info AMM12 Teachers
Interface not straight-forward AMM12 Teachers
Experiments

18. OUNL
UC3M
Experts
KU Leuven
RWTH Aachen
Problem Source Stakeholder
Not support for project-based learning EXP Teachers
Not support for exchange questions EXP Students
Not support for exchange progress EXP Students
Not collecting evidences for summative EXP Teachers

19. Requirements
Requirement Stakeholder
Provide time management mechanisms Teachers
Provide info about students progression Teachers
Provide info about students requests Teachers
Straightforward UI Teachers
Responsive UI Teachers
Store and process students records Teachers
Support different teaching methods Teachers
Provide info to students for self-awareness Students
As much personalized feedback as possible Students
Enable students to focus on the assignment Students
Share and rate questions and answers Students
Share progression Students

20. in-Class Live Analytics
for Assessment and
Orchestration
http://www.class-on.org

22. Student UI

23. Student UI

24. Teacher
UI

25. Teacher
UI

26. Analytics

27. Video

28. Authentic
experiments
AMM12 AS12 AMM13
Date
March-May
2012
November
2012
February-May
2013
Duration 18 sessions 6 sessions 10 sessions*
Teachers 4 3 1
Students 70 121 30
Group
settings
Dyads Dyads Dyads

29. Results
Teachers
interviews
Students
surveys
Collected
traces

30. Results
Teachers
interviews
Students
surveys
Collected
traces

31. Results
Teachers
interviews
Students
surveys
Collected
traces Requirements

32. Lessons
Learned

33. Lessons
Learned

34. Lessons
Learned

35. Lessons
Learned

36. Lessons
Learned

37. Contributions

38. Contributions

39. Contributions

40. Contributions

41. Related
Publications
Gutiérrez Rojas, I., Crespo García, R. M., & Delgado Kloos, C. (2012). Enhancing
orchestration of lab sessions by means of awareness mechanisms.
EC-TEL 2012: 21st century learning for 21st century skills: 7th European
Conference of Technology Enhanced Learning, 18-21 September 2012, Saarbrücken
(Germany) (Vol. 7563, pp. 113–125).
Gutiérrez Rojas, I., & Crespo García, R. M. (2012). Towards efficient provision
of feedback in lab sessions. International Conference on Advanced Learning
Technologies (ICALT) (Vol. 1). Rome.
Gutiérrez Rojas, I., Crespo García, R. M., & Delgado Kloos, C. (2011).
Orchestration and feedback in lab sessions: improvements in
quick feedback provision. Towards Ubiquitous Learning. 6th European Conference
ofTechnology Enhanced Learning, EC-TEL 2011 Palermo, Italy, September 20-23, 2011
Proceedings. LNCS, 6964. doi:10.1007/978-3-642-23985-4

42. Future Work

43. Future Work

44. http://www.moocrank.com

45. Thank you
very much!