In certain applications such as radiology and imagery analysis, it is important to minimize errors. In this paper we evaluate a structured inspection method that uses eye tracking information as a feedback mechanism to the image inspector. Our two-phase method starts with a free viewing phase during which gaze data is collected. During the next phase, we either segment the image, mask previously seen areas of the image, or combine the two techniques, and repeat the search. We compare the different methods proposed for the second search phase by evaluating the inspection method using true positive and false negative rates, and subjective workload. Results show that gaze-blocked configurations reduced the subjective workload, and that gaze-blocking without segmentation showed the largest increase in true positive identifications and the largest decrease in false negative identifications of previously unseen objects.

1. Understanding the Benefits of Gaze Enhanced Visual Search Pernilla Qvarfordt, Jacob T. Biehl, Gene Golovchinsky and Tony DunniganFX Palo Alto LaboratoryPalo Alto, California, USA

2. Inspecting images is common: Radiologist inspect medical images Airport security inspects x-rays of luggage Satellite images are inspect for threats Quality control of products often include visual inspection

3. Visual search is error prone We miss looking everywhere Radiologist overall error rate ~20% (Goddard et al., 2001) Current solutions: Systematic inspection for all parts of the image Documentation of review process Second reviewer Pattern recognitions models (e.g CAD) (From Mello-Thoms et al. ETRA 2002)

4. Past research on improving visual inspection Training Prescribed scan paths Kollera, Drury and Schwaninger (2009), Nickles, Melloy and Gramapadhye (2003) Scan paths from expert to guide novices Sadasvian et al. (2005) Improving user interfaces Augementing display of images Haiman et al (2004) Segmentation of images Forlines and Balakrishnan (2009) Re-presentation of viewed but not selected regions Nodine and Kundel (1987)

5. Two phase inspection method Detect fixations Phase 1 Phase 2 Cluster fixations Gaze Data Determine clusters to exclude

6. Experimental design 2 x 2 within-subject design & 8 participants 24 images: 6 images per condition 1 training image per condition 260-300 shapes ~25 x 25 pixels 5-20 targets per image (random) 10-40 close distractors 67.5 sec per phase Each segment shown 7.5 sec Gaze block: 270 ms threshold to block cluster Tobii X120 Eye tracker & 18” CRT Monitor Gaze block No block Full image Segmentation Target Close distractors

7. Results: Performance Overall no difference in True Positive identifications after both phases Increase in True Positive rate in 2nd phase (Block + full image) Near sig. interaction Increase in FN not viewed in 1st phase transitioning to TP in 2nd phase (Block + full image) Sig. interaction Significant reduced mental workload (TLX) for Gaze Block

8. Results: Performance Overall no difference in True Positive identifications after both phases Increase in True Positive rate in 2nd phase (Block + full image) Near sig. interaction Increase in FN not viewed in 1st phase transitioning to TP in 2nd phase (Block + full image) Sig. interaction Significant reduced mental workload (TLX) for Gaze Block

9. Results: Gaze Behavior Longer durations on True Positives than on False Negatives Inline with previous research: (Nodine and Kundel, 1987; Manning, Ethell and Donovan, 2001) Adopt to fixation length Longer fixation in phase 2 Sig. shorter fixation on FN viewed in phase 1 with gaze block 550 ms 1032 ms

10. Future work How to use gaze patterns to guide inspectors to better performance? Optimize use of the two phases How to combine information from gaze and image processing to guide inspectors to important parts of the image?

11. Conclusion Two phase inspection method Reduces workload (with gaze block) Have positive effect on FN not viewed transitioning to TP during Possible to estimate targets benefiting fromsecond review

12. Now for your questions… Pernilla Qvarfordt, Jacob T. Biehl, Gene Golovchinsky and Tony DunniganFX Palo Alto LaboratoryPalo Alto, California, USA