3. GETTING STARTED
Read two books available on our shelves:
Web Site Design with the Patron in Mind,
by Susanna Davidsen & Everyl Yankee
Design talk : understanding the roles of usability
practitioners, web designers, and web developers in
user-centered web design, by Brenda Reeb
Reviewed mission statement of PSU Special Collections
(“…to provide primary source materials for researchers from
Penn State and around the world”)
4. GETTING STARTED, CONTINUED
• Brainstormed user groups
• Brainstormed user tasks
• Got input/approval from colleagues
• Met with Usability Team
• Became certified for IRB
• Secured funding
5. STUMBLING BLOCKS
IRB (Institutional Review Board) process
Funding/reimbursement process
Recruitment
OTHER NOTES
o Not testing finding aids or digital projects
o Specific eye towards administrative organization
o Having deadline was good to force IT cooperation
6. STUDY PROTOCOL
Introductions
Explanation of the study:
We are testing the usability of the website, NOT your skills as a searcher.
Please think aloud while performing the tasks,
about how and why you make choices,
and what confuses or frustrates you.
We can clarify the tasks, but cannot help you perform them.
The session will be recorded, and notes will be taken, but your involvement will be
anonymous. Recordings will only ever be shown within the department for internal
purposes. Your name will never be associated with the recordings or the notes.
After you have done the tasks we will ask you to complete a brief survey. The entire
session should last about 1 hour.
Participation in this study is voluntary, and you can choose to end this session any time.
Do you have any questions?
Consent forms
Set up computer and give participant list of tasks
Take notes
Questionnaire
Ask again if any questions
Thank them, provide compensation, get receipt
7. TASK LIST
Warm-up:
Use our website to answer the following questions:
What hours is the Special Collections Library open to the public?
Where is the Special Collections library located?
PSU students (undergrad and grad):
You have been assigned to write a paper using historical documents from the Penn State
Special Collections library. Your professor has told you that there are special procedures for
using Special Collections. Using our website, find two ways using a special collections library is
different from using a general academic library.
You have found sources within Special Collections and written your paper. Use our website to
find the correct format for citing those materials.
PSU staff:
Your filing cabinets are overflowing, and your supervisor has asked that you transfer old “stuff”
to the University Archives to make room for current “stuff”. Locate information on what records
we collect on our website.
General:
Can you locate the following on our website: (Each subject asked 2)
Digital images of Pennsylvania railroads?
Information about special collections materials related to the 8th air force?
Information about special collections materials related to Amish settlements in Pennsylvania?
Information about special collections materials related to African American history?
Can you use our website to:
Figure out who you would ask a question about the United Steelworkers archives?
Order a Fred Waring Christmas CD?
Find a yearbook photo of David Arthur Valentine, who graduated in 1961?
Find out if we have video of the October 1940 football game with Temple?
8. QUESTIONNAIRE
1) How much time do you spend using the Internet for homework or work, per week?
___ Never ___ 1-5 hours ___ 5-10 hours ___More than 10 hours
2) Do you consider yourself an expert computer/Internet user? Yes/No
3) Have you ever been oriented to special collections (generally or at Penn State)
through a class, tour, online tutorial, or other outreach activity? Yes/No
4) In the last year, how many times have you used special collections materials?
At Penn State: Never, Once or twice, Between 2 and 5 times, More than 5 times
Anywhere Else: Never, Once or twice, Between 2 and 5 times, More than 5 times
5) The Special Collections Library Web Site is easy to use:
__ Strongly Agree __Agree __Disagree __Strongly Disagree
6) If you selected Disagree or Strongly Disagree please tell us what causes difficulties:
7) What changes would you make to improve the Special Collections Library’s website?
9. OBSERVATIONS – TASK CONSTRUCTION
Terms matter
When phrase “historical materials” in
question, they go to “Historical Collections” unit
– same for “archival materials” and University
Archives
“Reference” confused for citation
Along these lines, subject knowledge matters –
“Amish” in question, “Pennsylvania German” in
collection title
Order of questions can influence approach
May need to change order from study to study
10. OBSERVATIONS – SEARCH (GENERAL)
About half didn’t even look at the side bar
navigation
Everyone likes to use the site search
But it is unclear that this searches all the
University Libraries.
They also want to search on format (and can’t)
11. OBSERVATIONS – SEARCHING SUBJECTS
None of those looking for Amish materials
found our largest Amish collection
Only one person looking for African-American
materials found our largest A/A collection
Most commented they wanted collections
listed by topic
12. OBSERVATIONS – THE USELESSNESS OF
LONG EXPLANATIONS ON WEBSITES
Half of the subjects commented there was
too much text on any given page
Several could be on the right page but not
see a link right in front of them, if there were
too many “ other things” to look at
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
Yadda yadda yadda yadda yadda
13. IMMEDIATE CHANGES
Add curator contact info to all collection pages
Add “(yearbook)” to LaVie link
Add citation info to tutorial & online reference pages
Add link to digital collections from search collections page
Hopefully, organize by subject rather than unit
Hopefully, further edit long textual descriptions on top-
level pages
15. NEXT STEPS
First round not perfect, but we’ve learned a
lot, not just about the web site but about the
user study process.
For the next round, we may use a more
ethnographic approach.
Google Analytics.
Site search just for Special Collections?
Better finding aids & FA search (then test).
I will be asking to join the Usability Team.
16. AREAS FOR FUTURE
RESEARCH/COLLABORATION/BRAINSTORMING
Apply Kulthau?
Standardized list of questions for
archival website user studies?
Explore ideas for “Research Help”:
Diagrams/visual dictionary of common terms
Possible to do mouseover definitions in CMS?
FAQ since looks simpler/quicker than tutorial?
"Search Tips“
Thinking about how to illustrate
concepts/collections with just one image
