Presentation at Erasmus University Library 11-12-2012. For the most part a combination of slides from previous presentations, mostly from http://www.slideshare.net/MaxKemman/mapping-the-use-of-digital-sources-amongst-humanities-scholars-in-the-netherlands

1. User research for the
development of search systems
Max Kemman MSc
Martijn Kleppe MA
ESHCC
www.axes-project.eu

2. www.axes-project.eu

3. www.axes-project.eu

4. How do we search for images?
 By using words!
(your own & someone else’s)
New possibilities:
Speech- & Image recognition
AXES:
Access to Audiovisual Archives
www.axes-project.eu

5. Developing tools providing
new engaging ways to interact
with audiovisual libraries…
Not just search:
Browse Explore Experience
www.axes-project.eu

6. ... for a multitude of end users...
Broadcast professionals
Academics
Home users
www.axes-project.eu

7. www.axes-project.eu

8. www.axes-project.eu

9. PoliMedia research question
What choices do different media make in the coverage
of people and topics while reporting on debates in the
Dutch parliament since the first televised evening news
in 1956 until 1995?
www.axes-project.eu

10. Issues with current approach
Media analysis
+ = Too much
work
Limited
material
+ = and
different
systems
www.axes-project.eu

11. PoliMedia approach
PoliMedia
Newspapers
Portal
KB
Staten
- Browse:
Generaal Television
debate and
Digitaal Sound and Vision
date
KB
- Search:
debate and Radio
person KB
www.axes-project.eu

12. Presentation UB - 11-12-2012
www.axes-project.eu

13. Our research agenda
• Possibilities of eHumanities for research and
education
• How and what do people search in our digital
heritage collections?
– Requirements
– Testing
Presentation UB - 11-12-2012
www.axes-project.eu

14. User research
• Understanding the user
– Acceptance
– Performance
– Capabilities
– Weaknesses
• Goal
– Creating systems that are intuitive and helpful to
the users
www.axes-project.eu

15. Who are the users?
• We can distinguish groups, using
classifications of
– Roles (e.g., by occupation)
– Demographic characteristics (e.g., age, gender)
• In AXES, we distinguish by role
– Broadcast Professional
– Academics (teachers, researchers, students)
– Home users
www.axes-project.eu

16. Getting to know the users
• Prior research
– StatCounter, other literature
• Asking potential users
– Interviews
– Focus groups
– Surveys
• Experiments
www.axes-project.eu

17. The more, the merrier?
PhD student, History, 24
Presentation UB - 11-12-2012
www.axes-project.eu

18. The more, the merrier?
“If there is an easier way, I will
do it another way. So I won’t
go on the Internet to search.
There’s so much, there is so
much information. So you can
better call someone who
knows than search it for
yourself.”
Presentation UB - 11-12-2012
www.axes-project.eu

19. Academic user research
• Interviews
• Survey
– Netherlands and Belgium
– Online
– N=294
– 15-20 minutes
– Lots of data (250 variables in SPSS)
Presentation UB - 11-12-2012
www.axes-project.eu

20. Who did the survey?
• Position
– Largest group: PhD student
– But good distribution over
other positions
• Age
– Largest group: 25-34
– But good distribution over
other age groups
• Discipline
– Largest group: History
– Other disciplines: Social Studies, Mass Communications,
Linguistics, Literature, Philosophy
Presentation UB - 11-12-2012
www.axes-project.eu

21. Research questions
1. To what extent are online databases used?
2. Which subdisciplines use digital sources
more and which less?
3. Which search techniques are applied?
Presentation UB - 11-12-2012
www.axes-project.eu

22. Which of the following digital data do
you use professionally?
Very often
Often
Text (books, news, etc.)
Scholarly publications
Regularly
Statistical data
Sometimes
Numerical
Digitized
objects
Images
Never
Audio
Video
data
Presentation UB - 11-12-2012
www.axes-project.eu

23. Differences between disciplines
‘Traditional’ digital data Modern digital data
Images: Literature, History Video: Mass Communications
Digitized objects: Literature, History, Audio: Linguistics
Philosophy
Numerical data: Social Studies, Linguistics
Statistical data: Social Studies, Mass
Communications, Linguistics
Overall: Literature, History, Philosophy Overall: Mass Communications,
Linguistics, Social Studies
Presentation UB - 11-12-2012
www.axes-project.eu

24. Don’t
Sometimes
Very often
Often
Regularly
Never
know it
Google
Google Images
Google Scholar
YouTube
JSTOR
Uitzending Gemist
KB
Flickr
EBSCO
Presentation UB - 11-12-2012
Nationaal Archief
Web of Knowledge
Yahoo!
Bing
Academia.nl
or databases do you use?
Europeana
Scopus
Microsoft Academic Search
Which of the following search engines
EUscreen
www.axes-project.eu
Arkyves

25. What about sEURch?
• 31 respondents from EUR
Presentation UB - 11-12-2012
www.axes-project.eu

26. When do you trust a search engine or
database?
250
Number of participants
200
150
100
50
0
Experienced Expertise High quality Understand Heard about Broad range Read about
it behind it selection its inner it of results it online
workings
Presentation UB - 11-12-2012
www.axes-project.eu

27. Research questions
1. To what extent are online databases used?
– Mostly text-based data usage, followed by images
– Google dominant in every way
– Trust is based mostly on experience
2. Which subdisciplines use digital sources more and
which less?
– ‘Traditional’ digital data (Images, Digitized objects):
• Literature, History and Philosophy
– ‘Modern’ digital data (Video, Audio, Statistical and
Numerical data)
• Social studies, Mass Communications, Linguistics
3. Which search techniques are applied?
Presentation UB - 11-12-2012
www.axes-project.eu

28. Search techniques
1. Keywords 4,75
2. Advanced search 3,36
3. Related terms 2,52
4. Boolean 2,42
5. Browsing subject
categories 2,29
6. Filters 2,19
7. Thesaurus 1,87
8. Visualization 1,22
Presentation UB - 11-12-2012
www.axes-project.eu

29. (Audiovisual) search behaviour
• Academics are “surfers” with a non-specific
goal in mind
• Academics are positively confident in their
ability to use search tools
– Respondents below 45 years are more confident
than those above 45 years
Presentation UB - 11-12-2012
www.axes-project.eu

30. Research questions
1. To what extent are online databases used?
2. Which subdisciplines use digital sources
more and which less?
3. Which search techniques are applied?
– Most important are ease and speed
– Search behaviour: younger academics (below 45
years) are more confident in their use of
(audiovisual) search tools
Presentation UB - 11-12-2012
www.axes-project.eu

31. Survey conclusion
• We see that
– Text is the dominant medium
– Google is the dominant search system
– Google dominates search techniques
– Trust in a search engine or database is based
primarily on experience
Presentation UB - 11-12-2012
www.axes-project.eu

32. State of affairs?
Presentation UB - 11-12-2012
www.axes-project.eu

33. Consequences
• How do we use this information in our search
system development?
• How to make new search systems desirable?
Presentation UB - 11-12-2012
www.axes-project.eu

34. Desirable?
PhD student, History, 34
Presentation UB - 11-12-2012
www.axes-project.eu

35. Influence of usability
“I think it’s very
important to have a good
and workable interface
for these objects. That’s
why I used a paper
catalogue, because the
interface and the way to
search these objects is
not ideal”
Presentation UB - 11-12-2012
www.axes-project.eu

36. Consequences
• Making new search systems desirable by
incorporating user-centred design
• Basing system design on findings from user
studies
Presentation UB - 11-12-2012
www.axes-project.eu

37. AXES search wireframe
• Clear and
immediate
keyword-search
• Separate
advanced-search
• Support for
Booleans
Presentation UB - 11-12-2012
www.axes-project.eu

38. PoliMedia search wireframe
• Clear and
immediate
keyword-search
• Separate
advanced-search
• Support for
Booleans and
(some) Google-
search operators
Presentation UB - 11-12-2012
www.axes-project.eu

39. AXES browsing wireframe
• Keyword-search
remains
prominent
• Engaging user by:
 Suggesting query
terms
 Suggesting
related videos
Presentation UB - 11-12-2012
www.axes-project.eu

40. PoliMedia browsing wireframe
• Keywords-search
remains
prominent
• Easy access to
related material
Presentation UB - 11-12-2012
www.axes-project.eu

41. Consequences
• Making new search systems desirable by
incorporating user-centred design
• Basing system design on findings from user
studies
• Relevancy for UB?
Presentation UB - 11-12-2012
www.axes-project.eu

42. Thank you for you attention!
Martijn Kleppe
kleppe@eshcc.eur.nl
L2-47
Max Kemman
kemman@eshcc.eur.nl
L2-47
www.axes-project.eu
www.polimedia.nl
Presentation UB - 11-12-2012
www.axes-project.eu