The human eye, with the assistance of an eye tracking apparatus, may serve as an input controller to a computer system. Much like point-select operations with a mouse, the eye can "look-select", and thereby activate items such as buttons, icons, links, or text. Evaluating the eye working in concert with an eye tracking system requires a methodology that uniquely addresses the characteristics of both the eye and the eye tracking apparatus. Among the interactions considered are eye typing and mouse emulation. Eye typing involves using the eye to interact with an on-screen keyboard to generate text messages. Mouse emulation involves using the eye for conventional point-select operations in a graphical user interface. In this case, the methodologies for evaluating pointing devices (e.g., Fitts' law and ISO 9241-9) are applicable but must be tailored to the unique characteristics of the eye, such as saccadic movement. This presentation surveys and reviews these and other issues in evaluating eye-tracking systems for computer input. Scott MacKenzie is associate professor of Computer Science and Engineering at York University, Toronto, Canada. His research is in human-computer interaction with an emphasis on human performance measurement and modeling, experimental methods and evaluation, interaction devices and techniques, alphanumeric entry, language modeling, and mobile computing. He has more than 100 peer-reviewed publications in the field of Human-Computer Interaction, including more than 30 from the ACM's annual SIGCHI conference. He has given numerous invited talks over the past 20 years.

1. Evaluating Eye Tracking Systems for Computer Data Entry I. Scott MacKenzie York University http://www.yorku.ca/mack/

2. Evaluating Eye Tracking Systems for Computer Data Entry I. Scott MacKenzie York University http://www.yorku.ca/mack/

5. Human-Computer Interface Using the eye as the input control

6. “ Double Duty”

7. An Eye Tracking Session at CHI Input control Visual search Visual search Visual search

10. The Eye

11. The Eye

12. The Eye

13. Where Is The User Looking? Pointing  = 0.17  1000 ft 3 ft

14. Where Is The User Looking? 30 in 0.25 in  = 0.48 

15. Where Is My Thumb?

16. Where Is My Thumb? 0.6 in 25 in  = 1.2 

17. Double Trouble! 1.2 in 25 in  = 2.3 

19. Where is the User Looking?

20. Demo

24. Dwell-time Progress Indicators 1 1 Hansen, J. P., Hansen, D. W., & Johansen, A. S. (2001). Bringing gaze-based interaction back to basics. Proceedings of HCI International 2001 , 325-328. Mahwah, NJ: Erlbaum.

25. Shrinking Letter Progress Indicator 1 1 Majaranta, P., MacKenzie, I. S., Aula, A., & Räiha, K.-J. (2003). Auditory and visual feedback during eye typing. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems - CHI 2003 , 766-767. New York: ACM.

26. Grab And Hold Algorithm 1 1 Miniotas, D., Špakov, O., & MacKenzie, I. S. (2004). Eye gaze interaction with expanding targets. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems - CHI 2004 , 1255-1258. New York: ACM.

27. Next-letter Prediction 1 1 MacKenzie, I. S., & Zhang, X. (2008). Eye typing using word and letter prediction and a fixation algorithm. Proceedings of the ACM Symposium on Eye Tracking Research and Applications -- ETRA 2008 , 55-58. New York: ACM.

28. Speech-assisted Selection 1 1 Zhang, Q., Imamiya, A., Go, K., & Mao, X. (2004). Resolving ambiguities of a gaze and speech interface, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2004 (pp. 85-92): New York: ACM.

29. Speech-assisted Selection 1 1 Miniotas, D., Spakov, O., & MacKenzie, I. S. (2006). Speech-augmented eye gaze interaction with small closely-spaced targets. Proceedings of the Symposium on Eye Tracking Research and Applications - ETRA 2006 , 67-72. New York: ACM.

30. Snap-on Selection 1 1 Tien, G., & Atkins, M. S. (2008). Improving hands-free menu selection using eye gaze glances and fixations. Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2008 , 47-50. New York: ACM.

31. Snap-clutch Selection 1 1 Istance, H., Bates, R., Hyrskykari, A., & Vickers, S. (2008). Snap clutch: A moded approach to solving the Midas touch problem. Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2008 , 221-228. New York: ACM.

32. Fisheye Lens Selection 1 1 Ashmore, M., Duchowski, A. T., & Shoemaker, G. (2005). Efficient eye pointing with a fisheye lens . Proceedings of Graphics Interface 2005 , 203-210. Toronto: Canadian Information Processing Society.

33. Cursor Warping 1 1 Zhai, S., Morimoto, C., & Ihde, S. (1999). Manual gaze input cascaded (MAGIC) pointing . Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI '99 , 248-253. New York: ACM.

34. Early-raw-smoothed-late Selection 1 1 Kumar, M., Klingner, J., Winograd, T., & Paepcke, A. (2008). Improving the accuracy of gaze input for interaction. Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2008 , 65-68. New York: ACM.

35. Gesture-based Selection 1 1 Mollenbach, E., Hansen, J. P., Lillholm, M., & Gale, A. G. (2009). Single stroke gaze gestures. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems - CHI 2009 , 4555-4560. New York: ACM.

36. Nearest Neighbor Selection 1 1 Sibert, L. E., & Jacob, R. J. K. (2000). Evaluation of eye gaze interaction. Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2000 , 281-288. New York: ACM.

37. Big Targets 1 1 Hansen, J. P., Johansen, A. S., Hansen, D. W., Itoh, K., & Mashino, S. (2003). Command without a click: Dwell time typing by mouse and gaze selection. Proceedings of INTERACT 2003 , 121-128. Berlin: Springer. Demo

41. Independent Variables (eye tracking)

42. Dependent Variables (eye tracking)

43. Read Text Events (RTE)

44. Eye Tracking Path Characteristics Are there observable, measurable behaviours here – that can be used as dependent variables?

46. Based on…

52. tolerantDrift Example

54. Model Results Note: Small buttons (diameter = 76 pixels)

55. Model Results (including word stem size) Weighting method 0 (each bar is the average of the error rates computed over the range of tolerantDrift) Weighting method 1 tolerantDrift = 70 pixels

57. Setup

58. Independent Variables (IV)

59. Button Size, Word Prediction

60. Letter Prediction

61. Button Feedback Normal Letter Predict Eye- over Select (+audible “click”)

66. Entry Speed by Button Size Difference NOT statistically significant ( F 1,15 = 1.162, p > .05 )

67. Entry Speed by Prediction Mode Difference (marginally) significant ( F 1,15 = 5.531 , p < .05 )

68. Entry Speed by Fixation Algorithm Difference NOT statistically significant ( F 1,15 = 0.878 , ns)

71. Based on…

73. BlinkWrite Application Demo

76. Setup

79. Results – Entry Speed Differences NOT statistically significant ( F 2,22 = 2.05, p > .05 )

80. Results – Error Rate Differences NOT statistically significant ( F 2,22 = 1.89, p > .05 )

81. Results – Deletes Per Phrase

82. Video Demo

83. References 1. Ashmore, M., Duchowski, A. T., and Shoemaker, G., Efficient eye pointing with a fisheye lens, Proceedings of Graphics Interface 2005, (Toronto: Canadian Information Processing Society, 2005), 203-210. 2. Hansen, J. P., Hansen, D. W., and Johansen, A. S., Bringing gaze-based interaction back to basics, Proceedings of HCI International 2001, (Mahwah, NJ: Erlbaum, 2001), 325-328. 3. Hansen, J. P., Tørning, K., Johansen, A. S., Itoh, K., and Aoki, H., Gaze typing compared with input by head and hand, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2004, (New York: ACM, 2004), 131-138. 4. Itoh, K., Aoki, H., and Hansen, J. P., A comparative usability study of two Japanese gaze typing systems, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2006, (New York:, 2006), 59-66. 5. Majaranta, P., MacKenzie, I. S., Aula, A., and Räiha, K.-J., Effects of feedback and dwell time on eye typing speed and accuracy, Universal Access in the Information Society (UAIS), 5, 2006, 199-208. 6. Pan, B., Hembrooke, H. A., Gay, G. K., Granka, L. A., Feusner, M. K., and Newman, J. K., The determinants of web page viewing behavior: An eye-tracking study, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2004, (New York: ACM, 2004), 147-154. 7. Sodhi, M., Reimer, B., Cohen, J. L., Vastenburg, E., Kaars, R., and Kirschenbaum, S., Off road driver eye movement tracking using head-mounted devices, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2002, (New York: ACM, 2002), 61-68. 8. Takahashi, K., Nakayama, M., and Shimizu, Y., The response of eye movement and pupil size to audio instruction while viewing a moving target, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2000, (New York: ACM, 2000), 131-138. 9. Zhai, S., Morimoto, C., and Ihde, S., Manual gaze input cascaded (MAGIC) pointing, Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI '99, (New York: ACM, 1999), 248-253. 10. Zhang, X. and MacKenzie, I. S., Evaluating eye tracking with ISO 9241 -- Part 9, Proceedings of HCI International 2007, (Heidelberg: Springer, 2007), 779-788. 11. Zhang, X., Ren, X., and Zha, H., Improving eye cursor's stability for eye pointing tasks,,, Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2008. Florence, Italy: New York:ACM, 2008, 525-534.

84. The End (almost) but first…

86. Thank You – Questions?