1.
ICALT 2012 – Rome - 05 July 2012 The Impact of Social Performance Visualization on Students M. H. Falakmasir, I-Han Hsiao, L. Mazzola, N. Grant, P. BrusilovskyUniversity of Pittsburgh, USA, University of Lugano, CH, and Community College of Allegheny County, PA, USA 1

2.
AgendaThe ContextOur approachResults of an experimentAnalysis of data collectedConsideration/Conclusions 2

3.
ContextProblem: researches reported possbile isolation anddisengagement in online educational exp.Proposal: recovering some contextual and socialinformation in web-based Intelligent Tutoring Systems (ITS),or Learning Management Systems (LMS) Secondary usage of the already collected data (learnermodel), taking some hints from Information Visualization tolimit the informative/cognitive overload problem.To visualize student learning progress: targeting for the needs of instructors / academic advisors (*) present visualizations directly to the student (OSLM) 3

4.
Claimproviding students with explorable social performance visualization could improve their engagement in learning and positively impact their performance in the educational system. 4

5.
Our approachUsed data in the experiment from ADAPT2 ITS: average mark achieved on self-evaluation questionnaire total number of attempts done to solve itbasic representation of learners knowledgefacilitating their meta-cognitive activitiespromoting self-reflectionsocial aspects:to get a view of their performance in the context of the class progress, positive sense of competition using a representation of average class performance 5

6.
Infrastructure Mazzola L., Mazza R. (2011). Visualizing Learner Models through data aggregation: a test case. Red-conference, rethinking education in the knowledge society, Monte Verità, Switzerland, March 7-10, pp.372- 380, ISBN 978-88-6101-010-9 .Mazzola L., Mazza R. (2010). GVIS: A Facility for Adaptively Mashing Up and Representing Open Learner Models. "Sustaining TEL: From Innovation to Learning and Practice" Lecture Notes in Computer Science, Volume 6383/2010, pp. 554-559, Proceedings of EC-TEL 2010, DOI: 10.1007/978-3-642-16020-2_53 .Mazzola L., Eynard D., Mazza R. (2010). GVIS: a framework for graphical mashups of heterogeneous sources to support data interpretation. 3nd Conference on Human System Interactions, 2010. HSI 10. 6

7.
An initial prototype 7

8.
The “running” implementation – part 1 8

9.
The “running” implementation – part 2 9

10.
Some Results Weekly representation of usage of the system (red: WebEx + NavEx; blue: QuizGuide). On the top the usage with Visualization, on the bottom the one without.→ .. so a success? Mhhhh, lets go deeper in the data... 10

11.
Some Results Weekly representation of usage of the system (red: WebEx + NavEx; blue: QuizGuide). On the top the usage with Visualization, on the bottom the one without.Seems to suggest a positive impact, but needs more analysis... 11

12.
Some ResultsInitial idea suggested by the data: Learner uses more the system and are more interested in give feedback Average activity in the system higher but lower on quiz (less try&guess) The average success rate increases (but the std. dev. is higher)Needs more deep analysis from statistical point of view... 12

13.
Some Results: deeper analysisDifferent patterns before and after midterm (the green dashed line) 13

14.
More Analysis of the data...At a p-level of 0.05 not verified an effect of the social visualization tool(on whole semester, left), but seems to work in involving learneron the first stages (that means here before midterm, on the right) 14

15.
Consideration and ConclusionsWe designed and deployed a Social visualization tool for ITS, devoted to learners self usageTested against usage of the system (contents and self- evaluation quizzes), with logs.Seems to produce more engagements and to reduce the phenomenon of gaming for the evaluation systemThe indicated enhancements seems to be even statistically sustained, at least analyzing the data before the first midterm.But require more analysis... 15

16.
Falakmasir M., Hsiao I., Mazzola L., Grant N., Brusilovsky P. (2012). The Impact of SocialPerformance Visualization on Students. Proceedings of the 12th IEEE InternationalConference on Advanced Learning Technologies and Technology-enhanced Learning, Rome,Italy, July 4-6, 2012, pgg. 565-569 Thank you for the attention... ...any questions? http://www.slideshare.net/mazzolaluca/icalt2012-presentation-13548761 luca.mazzola@usi.ch 16