From 3x5 to LCD: Considerations and How-tos for Online Card Sort Studies
1. From 3x5 to LCD*
Considerations and how-tos for
conducting online card sort
studies
Emily Ford
Urban & Public Affairs Librarian
Portland State University
forder@pdx.edu
*This presentation is based on an article forthcoming in College &
Research Libraries News titled - Is Digital Better than Analog?
Considerations for Online Card Sort Studies
2. What is a card sort study?
http://www.flickr.com/photos/rosenfeldmedia/3343504093
3. Open Card Sort
http://www.flickr.com/photos/rosenfeldmedia/3344343842/
5. 3x5 Card Sort
Get IRB waiver or approval
Make cards
Recruit and schedule participants
Observe talking out loud
Ask questions/interview subjects
Transfer analog data to digital
Crunch data
Make sense of data
Plan and implement
design/organization/changes
6. 3x5 Card Sort
Pro
• Rich qualitative
and quantitative
data
• Inexpensive
• Complements data
from other
usability testing
methods
7. 3x5 Card Sort
Con
• Time intensive
• Margins for error
• Participant
comfort/nerves
8. The Mission
• Create a LibGuide for OMPH
students.
• Share it between PSU and OHSU.
9. The Partner
Laura Zeigen
User Experience Librarian
Oregon Health & Science
University
Oregon Master of Public
Health Student
16. LCD Card Sort
Get IRB waiver or approval
Make cards
Recruit and schedule participants
Observe talking out loud
Ask questions/interview subjects
Transfer analog data to digital
Crunch data
Make sense of data
Plan and implement
design/organization/changes
18. LCD Card Sort
Con
• No “think out loud”
data
• Potential interface
issues with online
product
• Potential online
product functionality
issues
• Tracking discarded
cards
19. 3x5 or LCD?
Considerations:
• How many cards are in our set?
• What other methods are being used
for this project?
• What kind of users are our
subjects?
• Does the online product do what
we need?
• Do we want perfection or will
good enough suffice?
In a card sort study, subjects arrange a list of items or concepts (usually written on index cards) into logical organizational categories. Researchers typically encourage participants to “talk aloud” during the study to hear their thought process. The end result is a rich set of data in terms of how users organize information, but also how they think about doing it. In libs: web sites, infrastructure, research guides, RDA
Good because you get ideas about how users will label stuff
Good because you can figure out organizational structures of existing things
The components of what it means to do a 3x5 card sort
Lots of good data!Cheap to doWorks in conjunction with other kinds of usability testing, such as task oriented, etc.
Data entryHow many cards x number of participants (Jakob Nielsen says 15 is ideal)Error margins, what if you drop the cards? What if you lose them?Strange environment with strangers looking on for participant
Oregon MPH studentsDeceptively simple:WifiDifferent course of studyDifferent resources
FamiliesJobsSchool at nightPart-timeDifferent school experiences/different needs
Email input as to needs to understand between two libraries and research, then put that into cards and card sort, the rest was magic
Integrated a short survey so we could collect demographic data from subjects; easy to use interface; 30 cards and ten users for free although Jakob Nielsen says we should have 15
The interface looked like this, we also had demographic data from participants; auto shut off after 10 completed the study
And it spit out a csv file for us, too!Dendograms, participant agreement, similarity matrix, and coded csv file
This is what we made. Yes, we have two instances and have to email back and forth about changes
10 participants x 30 cards = 300 rows of data
Comfort of their own homesDistance students
Masking of label name inputsLoss of discarded cardsWhat if they don’t work in browsers or users have tech problems at home?