1. Information evening National e-Learning Laboratory

10. Usability Testing Timeline Planning: framing the question Planning: target group recruitment Session recording: n users by x sessions Analysis Report

13. Web-server Logging: navigation paths, performance, task completion. Server-side Logs

14. Each mouse-click is recorded: time stamp, location, application Each key-press is recorded, too.

15. Example of Screen Recording Screen behaviour: dynamic web-sites & applications, mouse clicks, keyboard use.

16. Screen behaviour: dynamic web-sites & applications, mouse clicks, keyboard use. Screen Behaviour Server-side Logs

17. Example of User Video from different Perspectives

18. The remote eyetracker is hidden under the screen. It uses infrared light to track the eyes.

19. The eyetracker records the gaze position and the duration of each fixation.

20. User behaviour: gesture and posture, facial expression, off-screen activities, gaze position. User Behaviour and Reaction Screen Behaviour Server-side Logs

21. User feedback: quantitative feedback, e.g., usability questionnaire

22. User feedback: qualitative feedback, e.g., open-ended questions or interview

23. User feedback: qualitative feedback, e.g., narrative over screen-recording User Feedback User Behaviour and Reaction Screen Behaviour Server-side Logs

24. Screen Behaviour User Behaviour and Reaction Server-side Logs User Feedback

27. Microphone

28. Dome Camera

30. User workstation 3 & 4 User workstation 1 & 2 Control desk Test and Observation Rooms

31. Observation Desk

32. Screen Recorder Machines

33. Switchboard

38. Eyetracking

39. We developed a Peer Finder for on-line learning. Design A is straightforward: A list of peers.

40. In design B users can indicate their preferences.

41. In design C users can search for suitable peers.

42. Which works best? A, B, or C? Let’s explore how learners are using the different designs. A B C

43. The remote eyetracker is hidden under the screen. It uses infrared light to track the eyes.

44. The eyetracker records the gaze position and the duration of each fixation.

45. Areas of Interest We can define Areas of Interest to track what learners look at, e.g., which information of a user is considered. For example, in this area, learners can see whether a peer is available or not.

46. In fact they spent most of the time reading the names rather than assessing their peers’ knowledge or availability.

47. Usability Questionnaire Results While everybody said all three designs are easy to use ... the eyetracker data revealed that ...

48. … many users ignored the instructions and the peers’ knowledge completely.