Developing User Interface Census Data Collection

Developing a User Interface for Large-
scale, Multi-mode, Survey Data Collection

Jennifer Beck & Elizabeth Sinclair
U.S. Census Bureau

User Focus 2012
October 19, 2012

Saturday, January 5, 2013

Economic Census
• A measure of the health of U.S. businesses and the American economy
every five years

• Collects detailed information on revenue and business performance

• Goes out to more than 4 million businesses across industries:
1. Multi-unit businesses = businesses with multiple locations
2. Single-unit businesses = businesses with a single location

• Businesses receive a combination of forms tailored to their specific business
activities
– more than 800 different survey forms

• The number of form combinations creates unique multi-mode data collection
challenges


Economic Census reporting options
• The 2012 Economic Census has three different
modes for data collection:

1. Paper forms
2. Surveyor – a software application
3. Internet


Paper form


Surveyor


Internet option


2012 Economic Census
Internet option
• Only available to single-unit businesses:
– A pilot for future Economic Censuses
– Alternative to Surveyor software

• Respondents log into a Web interface through
Census website

• Respondents fill out and submit all information
online


Multi-mode data collection
• Offering multiple modes of data collection has
benefits:
– Can save costs
– Good customer service

• Designing for multiple modes is challenging:
– Mode consistency
– Mode capabilities and constraints


Multi-mode challenges in the
Economic Census
• The paper forms, Surveyor (software program) and the Web
interface “share” a creation database

• The database contains “images” that will make up the form
pages for all three modes

• Because the systems “share” pages, the pages have to work
for all three modes:

– The “shared” design creates limitations and usability issues,
especially for Surveyor and the Web

– Changes made for the Web also make the same changes to
Surveyor


2012 Economic Census Internet
option: Usability testing goals
1. Evaluate the overall function and performance
of the online interface

2. Assess respondents’ reactions to the interface


Participants
• 17 respondents from small businesses

• Diverse job titles: accountants, office managers,
business owners

• From three different industries:
– Retail/Service companies
– Manufacturing companies
– Construction companies


Methodology
Where:
• Met at respondents’ place of business

What:
• Respondents used their own computers (or iPad)

How:
• Respondents filled out the census forms
• Completed tasks to test features of the interface
• Used standard “think-aloud” protocol


Usability findings
• The Web interface performed well during the
testing

• We uncovered some key findings about the
interface:
1. Questionnaire navigation
2. Scrolling
3. Response quality control


Questionnaire navigation
• Survey navigation:
– Forward and backward paging
– Questionnaire navigation – response path through
a survey

• Skip patterns:
– Questionnaire navigation path instructions for
respondents
– Help respondents locate and navigate to the
relevant form sections


Skip pattern: Paper questionnaire


Skip pattern: Web interface


Using skip patterns
• Skip patterns can have a mode-specific advantage:

– Paper:
• Require reading, understanding, and following skip instruction
navigation
• Respondents do not always follow skip patterns on paper forms
well
• Can create response errors

– On the Web:
• Skip patterns can be automated
• Directly navigate user down the correct questionnaire navigation
path
• Can reduce response error


Problems with skip patterns:
Usability findings
• The Web Interface (and Surveyor) had no automated skip patterns
– Questions still contained skip instructions
– Respondents had to page through irrelevant sections
– Due to timing and programming complexity

• Respondents were confused as to how to navigate to the appropriate
section of the questionnaire
– Expected forward navigation to take them to the appropriate screen
– Expected to be able to click “Remarks” to navigate to that section

• Lack of automation was especially problematic for the Web interface
– Web interface had no navigation tree
– Surveyor software has a navigation tree


Surveyor navigation tree


Fixing problems with skip patterns
The recommendations:
1. Program automated skip patterns to navigate the
respondent to the next questions (and remove skip instructions)
OR
2. Include a navigation tree

The resolutions:
• Skip instruction stayed with the question
– Shared image database made changing question layout and
presentation not possible

• Programming complexities and time constraints prevented
adding a navigation tree


Scrolling
• Economic Census has a number of “long”
questions
– Tend to be detailed business activities and cumulative
calculations
– Do not fit on a single “page”

• Long questions require mode-specific solutions:
– Paper:
• Pagination
– Web interface (and Surveyor):
• Paging
• Scrolling


Paper


Internet


Choosing between paging and scrolling

• Paging creates mode-consistency problems:
– Paper:
• Easy to “flip” back and forth between the question sections
– Web interface (and Surveyor):
• Requires forward and backward navigation to go back and
forth between the question sections
• Breaking up the item across pages can create response
errors

• Scrolling “preserves” mode consistency
– The 2012 Economic Census Web interface employed scrolling


Problems Scrolling: Usability findings

• Scrolling created mode-specific problems:
1. Long items were “intimidating” to respondents
2. Length attenuated the benefits of scrolling over
paging

• Scrolling was especially problematic for the
Web interface
– Timing out:
• Long items take longer to fill out
• Data save linked to navigating to the next page
• Respondents could lose data


Fixing problems with scrolling
The recommendations:
1. Break up the items over multiple pages
2. (Make changes to the question to maintain mode consistency)
3. Include a navigation tree

The resolutions:
• Questions maintained scrolling design:
– Shared image database made changing question layout and
presentation not possible
• Not enough time to re-program and add a navigation tree
• Time out increased to 50 minutes
• Data saved on time out


Response Quality Control
• Data quality is a big concern:
– Follow-up is expensive
– Post-collection editing is expensive and time consuming

• Respondents find data edits helpful
– Provides a “check” against errors
– Communicates the importance of the information

• Controlling for data quality is mode-specific advantage:
– Paper forms offer no “built-in” data quality control
– The Web interface allows for “real time data editing”
– Can link responding to satisfying quality requirements

• The web interface had edit triggers for individual items and for of the


Item-specific data edits: Errors
Errors
= items respondents
must fill out in order to
submit their data to
Census


Item-specific data edits: Warnings
Warnings
= items we would like
respondents to fill out *

* Respondents can still
submit their data to
Census with out filling
them out


Overall data edits: Form Review
Form Review
= indicated problem
sections with color
coding

• One-click navigation to
problem section


Response quality control:
Usability findings
• Respondents liked the quality control features:
– Item-specific data edits:
• The edit messages were clear and helpful
• Liked having the “extra check”
– Form Review:
• Really liked the layout of the form review
• Liked the ease of navigating to problem sections

• Capabilities were examples of successful design features

• Provide in quality-control measure not possible in a paper
mode


Challenges to designing the 2012
Economic Census Web interface
• The Economic Census is a large data-collection
undertaking

• Presents challenges for how to maintain consistency
between modes and optimize mode advantages

• More work needs to be done to refine web data
collection for future Economic Censuses


Contact Information
Jen Beck:
jennifer.l.beck@census.gov

Elizabeth Sinclair:
elizabeth.sinclair@census.gov

72


Developing User Interface Census Data Collection

Recommandé

Recommandé

Contenu connexe

Similaire à Developing User Interface Census Data Collection

Similaire à Developing User Interface Census Data Collection (20)

Plus de uxpa-dc

Plus de uxpa-dc (20)

Developing User Interface Census Data Collection