Display Matters: A Test of Visual Display Options in a Web-Based Survey
•
0 j'aime•442 vues
This document summarizes a study on the effects of different visual display options in a web-based survey. The study tested four versions that varied the location of the Next and Previous buttons and the presentation of response options as single or double columns. Participants completed the survey while their eye movements were tracked. Results showed higher satisfaction ratings for versions that placed the Previous button on the left and presented response options in double columns. Eye tracking data provided insights into how participants viewed the different display elements.
Recommandé
Recommandé
Contenu connexe
Similaire à Display Matters: A Test of Visual Display Options in a Web-Based Survey
Similaire à Display Matters: A Test of Visual Display Options in a Web-Based Survey (20)
Plus de Jennifer Romano Bergstrom
Plus de Jennifer Romano Bergstrom (17)
Dernier
Dernier (20)
Display Matters: A Test of Visual Display Options in a Web-Based Survey
- 1. Display Ma*ers: A Test of Visual Display Op6ons in a Web-‐Based Survey Jennifer C. Romano Bergstrom1, Jennifer M. Chen1, Timothy R. Gilbert2 & Ma* Jans1 1 Center for Survey Measurement 2 Demographic Surveys Division U.S. Census Bureau AAPOR 66th Annual Conference May 13, 2011
- 2. Current Survey Environment • Increasing number of surveys online • Design considera6ons – Naviga6on methods – Presenta6on of response op6ons 2
- 3. Current Survey Environment • Increasing number of surveys online • Design considera6ons – Naviga6on methods – Presenta6on of response op6ons 3
- 4. Background on Next and Previous • Next should be on the leU – Reduces the amount of 6me to move cursor to primary naviga6on bu*on (Couper, 2008) – Frequency of use (Dillman et al., 2009; Faulkner, 1998; Koyani et al., 2004; Wroblewski, 2008) 4
- 5. Background on Next and Previous • Previous should be on the leU – Web applica6on order – Everyday devices – Logical reading order 5
- 6. Background on Next and Previous • Previous should be on the leU – Web applica6on order – Everyday devices – Logical reading order 6
- 7. Background on Next and Previous • Previous should be below Next – Bu*ons can be closer (Couper et al., 2011; Wroblewski, 2008) 7
- 8. Background on Long Lists • One column – Visually appear to belong to one group – When there are two columns, 2nd one may not be seen (Smyth et al., 1997) • Two columns: Double banked – No scrolling – See all op6ons at once – Appears shorter 8
- 9. Measuring “Best” Design • Typical: In the Field – Drop-‐off rates – Keystrokes – Survey comple6on 6mes • Our Study: In the Lab – User sa6sfac6on – Eye-‐tracking data – Usability metrics 9
- 10. Usability • The extent to which a product can be used by specified users to achieve specified goals with effec6veness, efficiency, and sa6sfac6on. ISO/ TR 16982:2002 • For web-‐based surveys, the design must – Meet respondents’ needs – Facilitate easy comple6on – Provide a sa6sfying experience – Reduce respondent burden – Produce high-‐quality data 10
- 11. Na6onal Survey of College Graduates (NSCG) • Collects educa6on and job informa6on • Respondents have Bachelor’s degree • Was available in PAPI and CATI • Usability study for a web-‐based self-‐ administered instrument 11
- 12. Method • Lab-‐based usability study • TA read introduc6on and leU le*er on desk • Separate rooms • R read le*er and logged in to survey • Think Aloud (Olmsted-‐Hawala et al., 2010) • Eye Tracking • Sa6sfac6on Ques6onnaire • Debriefing 12
- 13. Par6cipants Gender N Age N Educa.on N Male 14 < 30 8 Bachelor’s 21 Female 16 31-‐45 7 Master’s 6 46-‐60 10 Ph.D. 3 > 60 5 Mean: 46 13
- 15. Ques6ons Eye Tracking Can Answer • Do respondents look at Next and Previous? • What do they look at first? • Is it distrac6ng when Previous is located in a par6cular place on the screen? • How long does it take respondents to see the Next bu*on? • Does presenta6on of long lists affect what users look at on the list? 15
- 16. Previous and Next Bu*ons 16
- 17. One Column vs. Two Columns 17
- 18. 4 Versions N_P1 N_P2 Next bu*on on leU, Next bu*on on leU, 1-‐column job code 2-‐column job code PN1 PN2 Previous bu*on on leU, Previous bu*on on leU, 1-‐column job code 2-‐column job code 18
- 19. Results: Sa6sfac6on I * p < 0.0001 19
- 20. 8.5 Results: Sa6sfac6on II 8.5 Mean Sa.sfac.on Mean Sa.sfac.on 8 8 Ra.ng 7.5 Ra.ng 7.5 7 7 6.5 6.5 6 6 Mean N_P PN Mean N_P PN Overall reac6on to the survey: Informa6on displayed on the screens: terrible – wonderful. p < 0.05. inadequate – adequate. p = 0.07. 8.5 8.5 Mean Sa.sfac.on Mean Sa.sfac.on 8 8 Ra.ng 7.5 Ra.ng 7.5 7 7 6.5 6.5 6 6 Mean N_P PN Mean N_P PN Arrangement of informa6on on the screens: Forward naviga6on: illogical – logical. p = 0.19. impossible – easy. p = 0.13. 20
- 21. Eye Tracking: Next / Previous 21
- 22. Eye Tracking: Previous / Next 22
- 23. Eye Tracking: N_P vs. PN • Par6cipants looked at Previous and Next in PN condi6ons • Many par6cipants looked at Previous in the N_P condi6ons – Consistent with Couper et al. (2011): Previous gets used more when it is on the right 23
- 24. Eye Tracking: Time to First Fixa6on 8 7.5 7 6.5 Seconds 6 PN 5.5 N_P 5 4.5 4 Next Previous Mean 6me to first look at the naviga6on bu*on 24
- 25. N_P vs. PN: Respondent Debriefing • N_P version – Counterintui6ve – Don’t like the “bu*ons being flipped.” – Next on the leU is “really irrita6ng.” – Order is “opposite of what most people would design.” • PN version – “Pre*y standard, like what you typically see.” – The loca6on is “logical.” 25
- 26. 1 Column vs. 2 Column 26
- 27. Time to First Fixa6on 25 20 15 Seconds 1 col * p < 0.01 10 2 col 5 0 First half of list Second half of list 27
- 28. Total Number of Fixa6ons 40 35 30 Number of Fixa.ons 25 20 1 col 15 2 col 10 5 0 First half of list Second half of list 28
- 29. Time to Complete Item 120 100 80 Seconds 60 1 col 2 col 40 20 0 Mean Min Max 29
- 30. 1 Col. vs. 2 Col.: Debriefing • 25 had a preference – 6 preferred one column • They had received the one-‐column version – 19 preferred 2 columns • 7 had received the one-‐column version • Prefer not to scroll • Want to see and compare everything at once • It is easier to “look through,” to scan, to read • Re one column, “How long is this list going to be?” 30
- 31. Conclusions • Par6cipants were more sa6sfied when Previous was on the leU. • Par6cipants preferred the long lists in two columns. • Par6cipants looked at the first half of the list sooner than the second half when in one column. • Par6cipants looked at the second half of the list more when it was in two columns. 31
- 32. Bigger Picture: Recap on Next and Previous • Next should be on the leU – Reduces the amount of 6me to move cursor to primary naviga6on bu*on – Tab order – Frequency of use • Previous should be on the leU – Web applica6on order – Everyday devices – Logical reading order – People are more sa6sfied – It takes longer to first look at Previous when on the right 32
- 33. Bigger Picture: Recap on Long Lists • One column – Visually appear to belong to one group • Two columns: Double banked – No scrolling – See all op6ons at once – Appears shorter – Second column may not be seen – People look at the second half more – People look at the first half sooner when it is in one column – People prefer two columns 33
- 34. Future Direc6ons • This is just a small nugget. • N_P vs. P_N study in progress – Same layout – No skip pa*erns – Efficiency measure • Long list of items condi6on – Which items do people pick? – Alphabe6zed vs. random order 34
- 35. Thank you! For more informa6on, please contact Jennifer Romano Bergstrom Jennifer.C.Romano@gmail.com Jennifer.Romano@census.gov Twi*er: @romanocog 35