Europeana User Survey 2014 Report

Alan Blackwood– User Experience Consultant
alan@uservision.co.uk
Europeana User Survey 2014 –
Report
March2014

2 Europeana User Survey2014 – Report | 19/03/2014
Outline
• Executive Summary
• Methodology
• Summary of 2014 Survey content
• Results of 2014 Survey
• Comparisons with 2011 Survey

Executive Summary 1 of 2
 Reported overall experience of the site is 'Good'
 There was a high proportion of first-time users in the overall respondent
group
 The majority of respondents were in the 25-54 age group with very few
aged under 18
 A high proportion (over 60%) of respondents had University degrees with
IT and Education the most common employment sectors
 Most respondents arrived at the site following a link from another site or
having read about it in a paper or journal
 Art, Photography and Maps were the most popular genres/categories
 The most common reason for the last visit to the site was 'exploration
within a topic' suggesting the need for prominent explicit links to support
this behaviour
Continued…

Executive Summary 2 of 2
 Self-reported success rates for task completion were high across all
languages (79 - 87%)
 Those that could not complete tasks most commonly attributed this to
either 'No search results' or 'Couldn't find the page [they were] looking for‘
 Many respondents claimed not to download items from the site - if this is a
priority for Europeana there need to be clearer calls to action to support
this activity
 Trust in Europeana as a content source is much higher than other
equivalent sites
 Navigation and Search were seen as being only on a par or slightly better
than competitors
 'Browse by subject' was the most commonly requested additional feature
 User-generated content or collections were the least valued potential
additions

Outline
• Methodology

Methodology
 Surveys were provided in 6 languages (total respondent numbers in
brackets):
– English (1652)
– French (178)
– German (302)
– Italian (74)
– Polish (56)
– Spanish (134)
 A link to the survey was then distributed via the Europeana website
 A Google Nexus 10 prize draw entry was offered as incentive for
participating
 The surveys were open during January and February 2014.

Outline
• Methodology

Summary of 2014 Survey content 1 of 2
 Frequency of visits
 Demographics
– Gender
– Age
– Education
– Employment
– Language
 Usage of Europeana
– Route to Europeana
– Most popular genres/categories
– Purpose of last visit
– Self-reported success rates
– Reasons fortask failure
– Downloads
 Other sites used

Summary of 2014 Survey content 2 of 2
 Additions/future developments
 Overall experience

Outline
• Methodology

Results: Notes
 Due to the relative volume of responses and thereforehigher likely statistical
significance,patternsof responsehave beenidentifiedin the English versionof
the survey first.
 Any relevantvariationsfrom these patternsin the survey responses in other
languages havethen beenreportedand underlying reasons suggestedwhere
discernable.
 This report focuseson the closedresponses from the survey. Open response data
is available to Europeana and further analysis may shed further light on the
underlying reasons for some of the trends in the data.

Post-test interview
Frequency of visits

Frequency of visits
 High proportionof 'first-time’andinfrequent visitors:
– ‘First-time’ is the single biggest groupinthe English and
German surveys,whichhad the largest overall number
of respondents
– Over 72% of respondents visit the site 'a fewtimes a
month‘ orless
– Those visiting 'severaltimes a week' ormore make up
only 13% of the total number of respondents.
 Recommendations:
– These results imply a need forclearroutes to helpnew
users quickly establishthe purpose of the site and to
draw these users into content relevant to their interests.
– These couldinclude:
• An 'About the site' link in the primary navigation
area.
• A dedicatedroute forfirst-time orirregularusers
such as a site tour orbulleted list of features &
benefits. This couldbe included in an‘About the
site’ section.
• Greateruse of ‘deep linking’ fromthe homepage
via subject searches relatedto featureditems.
Visit frequency
Englishlanguage

Post-test interview
Demographics

Demographics: Gender
 Generally more male thanfemale users across most
languages:
– Roughly 60:40 male:female split inthe English version
(908 of 1497 respondents)
– Some variationinnon-Englishsurveys:
• Highest: 74.6% male (German)– (206 of 276)
• Lowest: 41.0% male (Polish)– (23 of 56).
– This may indicate some opportunity to raise the site's
profile withinthe less well representedgender in each
language.
– However,the relatively small number of respondents in
the non-Englishlanguage surveys limits their likely
statistical significance.
Gender
Englishlanguage

Demographics: Age
 Over60% of respondents are in the 25-54 age range across all
languages.
 Between 1 and 6% are under18 across all languages.
– 25-54 age group:
• Accordingto demographic trends identified
elsewhere people in this age range are likely to be
heavy users of smartphone andtablet devices.
• Again, this will be verifiable onthe Europeana
portal by examiningsite analytics data.
• Optimising the site’s user experience across mobile
platforms is already a priority for Europeana and
should continue.
– Under 18s:
• There may be anopportunity to extendthe site’s
reachinto schools by:
– Creatingcollections or themes that are
closely tiedwith regional curriculums
– Promotingthe social sharingoptions within
these groups.
Age
Englishlanguage

Demographics: Education
 Direct comparisonis not possible here betweendifferent
nationalities due to variations inthe precise qualifications.
 64% of respondents to the English versionof the survey have
a University degree ofsome kindwithmany also havingsome
formof postgraduate orequivalent degree.
 This is broadly consistent across all languages andsuggests a
highly-educated user base.
Education level
Englishlanguage

Demographics: Employment
 The most prominent employment groups across all languages
were:
– Student (fullorpart-time): 7-8% acrosslanguages
– Unemployed/retired: between8% (Italian)and 22%
(French)
– Creative arts and culture: between 10% (German)and
24% (Spanish)
– IT and information services:between8% (Spanish)and
13.6% (English)
– Public sector: between7% (English) and 11% (German)
– Teaching andeducation:between6% (German)and
18% (Spanish)
 There was some large deviationhere betweenthe Polish
results and the other languages:
– Forexample,23.2% of these respondents were inthe
Student (fullorpart-time) category.
– Again, because of the small sample size, it is less likely
that this will be representative of the full user
population.
Employment (partial)
Englishlanguage

Demographics: Language
 The most widely spokensecondlanguages are:
– English for non-native speakers:
• Lowest proportion: 30% of Germannative speakers
• Highest proportion: 90% of Polish native speakers.
– German& Frenchfornative English speakers;
• 30% comfortable speaking German
• 26% comfortable speaking French.
Comfort with languages (partial)
German language

Post-test interview
Usage of Europeana

Usage: Routes to Europeana
 The most commonresponsesacross all languages are 'Read
about it in paper/journal' and'Link from anotherwebsite'.
 Wordof mouth(‘Someone told me aboutit’)also features
heavily across most languages suggestinga strongviral effect
in promotionof the site.
 The exceptionis the German language respondents where
42% had'Heard about it on radio/TV' whichindicates that
recent broadcast media exposure is directingtraffic tothe
site.
 This is also consistent withthe high proportionof first-time
visitors (38% of respondents)inthe Germanversion.
– There is inconclusive evidence of a direct relationship
here betweenbroadcast media profile andsite traffic.
– This shouldbe triangulatedwith other sources suchas
market researchandcampaigntrackingdata to better
understandthe detail of this relationship.
Routes to Europeana (partial)
Englishlanguage

Usage: Most popular genres/categories
 There was little variationacross the various languages here.
 The most commonly selectedoptions were:
– Art
– Photography
– Maps
– Informationabout a historical period
– Academic andscientific articles/journals.
 Recommendation:
– Content focussedonthese areas is likely to holdmost
interest and have most resonance withthe user
population.
– Items fromthese categories are thereforemore likely to
engage users anddraw them into the site if featuredon
the site homepage.
Most popular genres (partial)
Englishlanguage

Usage: Purpose of last visit
 Inmost languages, ‘Exploration within a topic’comes out just
ahead of ‘findingout about Europeana’:
– ‘Finding a specific item’is a significantlylowerpriority
across all languages,featuringprominently only in the
Spanish language version.
– This suggests that most users’progressionthroughthe
site is fromthe generic to the specific. More are
motivatedby interest in a topic rather thana needto
locate a specificitemonthe site.
– The ‘Exhibitions’ link is currently placedwithinthe footer
area oneachpage. Usage statistics will helpto confirm
this but it is likely that this will be foundless oftenby
users than if it was further up the page.
– Its current inclusionas a carousel itemonthe homepage
couldpartly address this needbut there is justification
fromthis evidence for a more prominent andpermanent
location.
– Given the preference expressedhere for browsingrather
than searching,increasedprominence forlinks to
subject matter searches couldalso be justified.
Purpose of last visit
Englishlanguage

Usage: Self-reported success rates
 Self-reportedsuccessrates are highacross all languages
(between79% and87%).
 This indicates thatfewbarriers to task completionwere
noticedby respondents while using the site.
 However,there are some caveats here:
– People will generally report a higher success rate for
themselves than that whichmight be more objectively
observed.
– Respondents’goals couldvary widely here from
spending 90 seconds readinga blogarticle or browsing
the homepage to anextensive investigationofa series of
topics.
– For these reasons,the genuine level of success for
commonor core tasksis difficult to accurately determine
fromthis evidence.
– While this figure is encouraging,it cannot be reliedupon
in isolation.
– Usability testingon the site will give greater confidence
in the validity of the findings here.
Achieved goals during last visit?
Englishlanguage

Usage: Reasons for task failure
 Of the minority of respondents (between 13% and 21%)that
did not successfully complete theirtask,the most common
reasons givenacross all languages were:
– ‘No search results’: between11% (Italian) and27%
(English, German and Spanish)
– ‘Couldn’t find the pageI was looking for’: between15%
(German) and42% (Spanish).
– If the proportions affectedare representative,these
issues appear to be affectingonly a minority of users.
– Again, no insight is available here into potential
underlying causes for these reportedissues.
– Undertakingcorrespondingusability testing onthe site
will give greater confidence inthe findings here.Reasons for task failure
Englishlanguage

Usage: Downloads
 A high proportionof respondentsacross all languages do not
download material fromthe Europeana site:
– German- 84%
– Polish- 64%
– English - 61%
– French- 53%.
Do you download?
Englishlanguage

Usage: Downloads – Why not?
 The most commonreasons fornot downloading across all
languages were:
– ‘I didn’t knowI could’– the most commonresponse for
most languages
– ‘Not interested’– this was chosenless commonly but
still selectedby a significant proportionof respondents.
– For many items there is no clear ‘Download’call to
action.
– Clickingon a thumbnail image takes the user to the
source website inmostcases.
– If encouragingdownloads is a priority activity for
Europeana,andconsistent withthe licensing terms fora
particularitem, then adding a clearbutton/iconwill help
to achieve this.
– Examples usedelsewhere include:
Why don’t you download?
Englishlanguage

Usage: Downloads – Why?
 The most commonreasons fordownloadingacross all
languages were:
– ‘Fun/inspiration’– this was the most commonresponse
by far in many languages:
• English (55%),
• German (61%),
• French (44%),
• Spanish (46%),
• Italian (50%).
– 'Academicresearch' and'Republishing/sharing' are the
2 next most commonresponses acrossalllanguages.
– This further supports the needfor clearly highlighted
‘browsing’routes through the site, servicing the
exploratory impulse of a large proportionofvisitors.
Why do you download?
Englishlanguage

Post-test interview
Other sites used

Other sites used
 Wikipedia is the most popular site fromthoselisted. This
couldbe a simple product of familiarity withthe site from
more general,non-'cultural heritage item'use.
 The popularity of a less centrally-controlledsite suchas
Wikipedia in response to this questionmaybe a contributing
factor to 60-70%of respondents across all languages
'Agreeing' or 'Stronglyagreeing' that Europeana has more
‘trustworthy content’ thantheir 'other mainsite':
– Wikipedia has anextremely broadscope but lacks depth
and may be comparatively less reliable as a sourcedue
to its distributed curationsystem.
 Project GutenbergandGoogle Bookswere consistently the
2nd and3rd most popular options.
– Trust in Europeana as a source is very high in comparison
to the other options presentedhere.
– Emphasising this as a distinguishing factor in
communications about the site couldreinforce this
distinctionfromother less actively-curatedonline
collections.
Other sitesused (partial)
Englishlanguage

Other sites used vs Europeana
 Inresponse to the statement ‘I preferEuropeana.eu to the
othermain site I use (from previous question) in terms of:’,
there were very few negative ratings for any ofthecategories.
 However,the 'Neutral' response was the mostcommonly
selectedresponse inthe followingcategories:
– General look andfeel (34.2%)
– Ease of access to content (36.7%)
– Navigationaroundthe site (39.7%)
– Searchfunctions (42.7%)
 The 'look & feel' result was marginal withthe next most
popular option'Agree slightly' accountingfor 32.4%.
 However,the patternof responses inthe other categories
againpoints to some potentialconfusion oroverall
dissatisfaction with the available routesto,and through, the
content onthe site.
 Recommendation:
– As indicatedby responses to other survey questions,it
couldbe beneficial to investigate potential issues
relatingto content discovery more closely usinga
combinationof True Intent/Voice ofthe customer
studies, webanalytics data andusability testing.I prefer Europeana (excerpts)
Englishlanguage

Post-test interview
Additions/future developments

 'Browse by subject' was the most requestedfeature fromthe
list supplied:
– 90% of respondents inthe English language survey
consideredthe ability to browse by various dimensions
using facetednavigationto be either'Useful'or 'Very
useful‘.
– This couldalso be linked to the underlying issue in the
responses to Q.11 (What stoppedyouachieving what
you wanted?)where 'I couldn't find the page I was
looking for' was the most commonresponse.
– Providing more prominent links to relatedsubject
searches may helphere. For example,providing2 or 3
relatedsubject links beneatheachfeatureditemonthe
homepage.
– Also,rearranging the content onthe homepage could
have a positive effect.For example:
• Removingor re-formattingthe carousel element
• Adding more static feature items inplace of,or
above,highlighted Blogentries.
– A/B/Multivariate testingcouldhelpdetermine which
approachis most effective.
Additions/future developments(excerpt)
Englishlanguage

 User-generatedcomments/tags/gallerieswere consistently
the least-valuedadditions:
– Over30% of respondents in the English language survey
statedthat they wouldfind the ability to 'Create and
publish my own galleries of favourite items' 'rarely
useful' or'not usefulat all'.
– Over23% rated‘Improve informationby addingmy
own tags orcomments’inthe same way.
– The relativelylow popularity of these featuresmaypoint
to perceived duplication/overlap offunctionality with
existing sites used for content collecting& sharing such
as Pinterest or Google+.
– Identify andpromote any unique benefits to users of this
type of functionality ontheEuropeana site.
– Continue to offer a wide range of external sharing
options if this is the preferredmethodfor many users.
Additions/future developments(excerpt)
Englishlanguage

Post-test interview
Overall experience

Overall experience
 There was a 3.8 average response score for this questionover all languages,just below ‘Good’:
– This indicates that the overall experience waspositive,if not overwhelmingly so.
 Only between5-10% of respondents categorisedtheir overall experience onthe site as 'Belowaverage' or
'Very bad' across all languages.
 Fromother responses withinthis survey, it seems that the most obviousnegative contributingfactors here
are:
– Difficultyfindingspecific items
– Not being able easilyto browse the site by subject area.
Very bad Below Average Average Good Excellent
1 2 3 4 5
3.8
How would you rateyour overallexperience?

Outline
• Methodology

Post-test interview
2014 vs 2011: Competitors

2014 vs 2011: Competitors
0%
20%
40%
60%
80%
100%
2011 2014
Favourite non-Europeana site  Wikipedia has movedfrom2.7% of respondents' favourite
in 2011 to almost 50% in2014.
 Google Books remains reasonably consistent with12.4%
choosingthis in 2014 and 10.6% in 2011.
 However,there are some potentially distortingdifferences in
the options providedin bothyears:
– Google search was not offeredas anoptiononthe
2014 list. This was the most popular response in2011
with 45% of respondents selectingit.
– No 'null' option ('Do not have a favourite/main site')
was providedfor this questionin the 2014 survey. This
was present and was selectedby almost28% of
respondents in the 2011 survey.

 Trustworthiness of content:
– 2011 - 54.4% Better
– 2014 - 55.9% Better
 Usefulnessof content:
– 2011 - 34.3% Better
– 2014 - 60.8% Better
 General look and feel:
– 2011 - 32.1% Better
– 2014 - 54.2% Better
 Presentation of results:
– 2011 - 26% Worse
– 2014 - 52.5% Better
 Ease of access:
– 2011 - 26% Worse
– 2014 - 48.0% Better
 Navigation:
– 2011 - 26% Worse
– 2014 - 44.8% Better
 Search:
– 2011 - 29.9% Worse
– 2014 - 45.0% Better
2014 vs 2011: Europeana vs Competitor
-80%
-60%
-40%
-20%
0%
20%
40%
60%
80%
2011 2014
Europeana - Better or Worse?
BetterWorse%ofresponses
Notes
Main competitors
2011 - Google Search
2014 – Wikipedia
Options available
2011 - Better/Similar/Worse
2014 – Strongly
disagree/Disagree/Neutral/
Agree slightly/Strongly agree
Results included
To allow comparison withthe
2011 report, neutral ratingsare
not shown, onlythe most
prominentpositive ornegative
rating ineach category.
Summary:
• The 2014 resultsare more positive in
every category.
• Negative ratingsinthe 2011 survey
shouldbe seenin the contextof
havinga dedicatedand highlyfamiliar
site such as Google as comparator in
most cases.

Post-test interview
2014 vs 2011: Age

2014 vs 2011: Age
0%
20%
40%
60%
80%
100%
2011 2014
% in each Age Group  There is very little difference inthe proportionof respondents
in eachof the categories from'Under 15' to '19-24‘between
the surveys.
 Some variationinthe proportionof respondents fromsome
age groups above that (as shownin the chart to the left):
– 23.3% in the 25-34 age group in 2014, 17.8% in 2011
– 12.0% in the 65+ age groupin 2014, 9.5% in 2011
 Overall the majority of respondents are inthe same 25-54 age
group:
– 2011: 62.5%
– 2014: 63.8%
 This suggests a reasonably stable user age profile over the 3
years betweenthe surveys with a subtle shift towards the
lower (25-34 years)endof the mainage groups between2011
and 2014 and a minor increase inthe proportionof over-65s.

Post-test interview
2014 vs 2011: Profession

2014 vs 2011: Profession
 Education:
– The proportionof respondents fromthe Education
sector is definedas follows for eachsurvey:
• Lecturer/teacher/trainer(2011): 14.8%
• Teachingand education(2014): 13.2%
– Using this as the basis for comparisonshows a marginal
decline between 2011 & 2014.
– However,it is possible that some 2014 respondents who
are trainers may not have selected‘Teachingand
Education’inresponse to this question.
– If professions hadbeengrouped in 2014 as they were in
2011 this may have meant evenless variationbetween
the surveys.
0%
20%
40%
60%
80%
100%
2011 2014
Employed in Education
Notes
Direct comparisonis not possible due to the much
higher level of granularity in the 2014 survey thanin
2011 whenprofession/employment sectordivisions
were definedmuchmore broadly.
Comparisonis providedhere for the only broadly
similar category inboth surveys.

Post-test interview
2014 vs 2011: Finding Europeana

 The followingoptions were selectedbysimilar proportionsof
respondents in 2011 and 2014.
– ‘Someonetold me aboutit’:
• 2011 - 9.7%
• 2014 - 10.9%
– ‘Search engine result’:
• 2011 - 6.5%
• 2014 - 5.2%
– ‘Link on a blog’
• 2011 - 1.4%
• 2014 - 1.6%
 A slightly higherproportionselected'involvedin a Europeana
project‘in2014 than in 2011:
• 2011 - 2.2%
• 2014 - 4.9%
 No option was givenin2011 forradio/TV promotion:
• In2014 it was 7.4%
Similar proportions:
2014 vs 2011

 A much lower proportionof respondents readabout the site
in a paper or journal in 2014 than in 2011:
– 2011 - 32.0%
– 2014 - 15.3%
 This may be affectedby factors suchas a changing
promotional strategy forEuropeana.
 However,it is also consistent withthe broader trendof
decline in the general influence of print media.
0%
20%
40%
60%
80%
100%
2011 2014
Read about it in paper/journal

 Conversely,a muchhigherproportionfolloweda 'Link from
anotherwebsite‘in2014 than in2011:
– 2011 - 13.9%
– 2014 - 25.7%
 Again, this may be affectedby factors suchas a changing
promotional strategy but is consistent with relative
directionaltrendsfor these media.
0%
20%
40%
60%
80%
100%
2011 2014
Link from another website

 There was also a relativelylarge increase inthe proportionof
respondents following links from socialmedia:
– 2011 - 1.7%
– 2014 - 7.7%
 However,this still plays a relatively smallrole inreportedly
directing traffic tothe site.
 It is possible that there was some cannibalisationof responses
here by the ‘Link fromanother website’optiondepending on
respondents’interpretationof the options or the care they
took whenselectingfromthe list.
– Comparisonwithreportedtraffic sources fromsite
analytics may give a better idea of the reliability of this
figure.
0%
20%
40%
60%
80%
100%
2011 2014
Links from social media

Thank you
For further information, please contact:
Alan Blackwood
UX Consultant
User Vision
55 North Castle Street
Edinburgh
EH2 3QA
Tel: 01312401494
Email: alan@uservision.co.uk
Web: www.uservision.co.uk

Europeana User Survey 2014 Report

Recommended

Recommended

More Related Content

Similar to Europeana User Survey 2014 Report

Similar to Europeana User Survey 2014 Report (20)

More from Europeana

More from Europeana (20)

Recently uploaded

Recently uploaded (20)

Europeana User Survey 2014 Report