Bonus Lecture - UX from 30,000ft (Lecture 3 Extra - BBC Presentation) #comp33512
1. BBC R&D | Feb 2012
Penny Allen
Research Scientist
BBC Future Media : R&D
2. BBC R&D | Feb 2012
Contents
01 What is a Research Scientist?
02 Real World UX Research
03 Learning Vs Doing
3. BBC R&D | Feb 2012
What is a
Research Scientist?
4. 01 What is a Research Scientist?
BBC R&D | Feb 2012
Background
BSc Psychology
MSc Interactive Systems: User Experience
UX
Usability Methods
Interaction Design
Prototyping for Interactive Media
5. 01 What is a Research Scientist?
BBC R&D | Feb 2012
Research & Development
User Experience Researcher
Literature Reviews
Sampling of Participants
Experimental Design
Field & Lab Studies
Analysing Data
Reports & Recommendations
Prototypes
6. 01 What is a Research Scientist?
BBC R&D | Feb 2012
Research & Development
User Research Lab
7. 02 Real World UX Research
BBC R&D | Feb 2012
Understanding people
& Technology
8. 02 Real World UX Research
BBC R&D | Feb 2012
Requirements
Gathering
Voice User Interface
for TV
9. 02 Real World UX Research
BBC R&D | Feb 2012
Requirements Gathering
Voice User Interface for TV
Video
10. 02 Real World UX Research
BBC R&D | Feb 2012
“Interviewer: So when you are browsing through the TV do you use the guide?
No because I can’t always read it…no I tend to actually if I don’t know what’s on I tend to look it up on the
internet.
Interviewer: What do you go to to look for the programme guide?
Either BBC TV guide or ITV TV guide…swap between the 2.
Interviewer: Is that to plan to watch something later or be to watch something now?
Both…if there’s nobody in to go through the TV guide with me then that’s what I’ll do.
Interviewer: So if there are other people with you…
I get them to do it…I just take pot luck that they’re telling me the truth…that they’re telling me
everything…skip over ones that they think I don’t want to watch [so you’re relying on their opinion?] I
always have done but because it’s something that I’ve always done until someone shows me something
different then…”
11. 02 Real World UX Research
BBC R&D | Feb 2012
“Interviewer: So when you are browsing through the TV do you use the guide?
No because I can’t always read it…no I tend to actually if I don’t know what’s on I tend to look it up on the
internet.
Interviewer: What do you go to to look for the programme guide?
Either BBC TV guide or ITV TV guide…swap between the 2.
Interviewer: Is that to plan to watch something later or be to watch something now?
Both…if there’s nobody in to go through the TV guide with me then that’s what I’ll do.
Interviewer: So if there are other people with you…
I get them to do it…I just take pot luck that they’re telling me the truth…that they’re telling me
everything…skip over ones that they think I don’t want to watch [so you’re relying on their opinion?] I
always have done but because it’s something that I’ve always done until someone shows me something
different then…”
12. 02 Real World UX Research
BBC R&D | Feb 2012
Design for Accessibility
Typography for TV
13. 02 Real World UX Research
BBC R&D | Feb 2012
Design for Accessibility
Typography for TV
Video
14. 02 Real World UX Research
BBC R&D | Feb 2012
Screens
17. 02 Real World UX Research
BBC R&D | Feb 2012
Quantitative Analysis
Excel
6
Mean smallest size
5
acceptable (mm)
4
3
Over 60
2
Under 60
1
0
18. 02 Real World UX Research
BBC R&D | Feb 2012
Font Size Calculator
Video
19. 02 Real World UX Research
BBC R&D | Feb 2012
Large Scale Data
Gathering
Musical Moods
20. 02 Real World UX Research
BBC R&D | Feb 2012
Large Scale Data Gathering
Musical Moods
21. 02 Real World UX Research
BBC R&D | Feb 2012
Research Environments
Production Labs
22. 02 Real World UX Research
BBC R&D | Feb 2012
Research Environments
Production Labs
23. 02 Real World UX Research
BBC R&D | Feb 2012
Classic Usability Testing
Interactive TV Templates
24. 02 Real World UX Research
BBC R&D | Feb 2012
Classic Usability Testing
Interactive TV Templates
25. 02 Real World UX Research
BBC R&D | Feb 2012
User Journeys
1. Find Video and PG
2. 360 Carousel
3. Bookends
4. Depth of Navigation
5. Help
26. 02 Real World UX Research
BBC R&D | Feb 2012
Participant Age Gender Red Button iPlayer Template 2nd Template
Familiar but don't
1 44 F Never used use often Quarter Accordian
Experimental Design 2 40 M News, sport Yes Quarter Overlay
3 23 F News, music, other Yes Overlay Quarter
4 54 M News, music, other Yes Overlay Accordian
Familiar but don't
5 59 F Never used use often Overlay Quarter
6 23 M News, sport Yes Overlay Accordian
Familiar - sport,
7 25 M music Yes Quarter Accordian
8 29 F Familiar - music Yes Overlay Quarter
Familiar - sport,
9 63 F other Yes Quarter Overlay
10 28 F News, music, other Yes Quarter Accordian
11 35 M News, sport, music Yes Accordian Overlay
12 55 M Never used Yes Accordian Quarter
News, sport, music,
13 43 M other Yes Quarter Overlay
14 25 F Music Yes Accordian Overlay
Familiar but don't
15 46 F Never used use often Accordian Quarter
16 35 F Never used Yes Overlay Accordian
Familiar but don't
17 35 M Never used use often Accordian Overlay
Familiar but don't
18 50 F News, sport use often Accordian Quarter
19 37 M News, sport Yes Accordian Overlay
20 21 M Never used Yes Overlay Quarter
21 19 M Never used Yes Quarter Accordian
28. 02 Real World UX Research
BBC R&D | Feb 2012
Analysis
“Bottom Up”
29. 03 Learning Vs Doing
BBC R&D | Feb 2012
Comparing theory to
practical day-to-day
UX
30. 03 Learning Vs Doing
BBC R&D | Feb 2012
Pay for participants
Time constraints
“Quick and dirty”
Stage of development
Communication
Notes de l'éditeur
Intro
Intro
Didn’t specialise in user experience until my mastersBut still use a lot of my expertise in psychology in my every day job
In real terms Research Scientist = UX researcherAs work in R&D, scientist sounds much more importantWork with many engineers (called technologists)Do a variety of things across all stages of developmentEach project is very different (devices, users, theories) have to become an expert in something very quicklyDesign experiments – field (in people’s homes/work) and in our lab (show in a min)Making sense of what I see or what people tell me and translate into requirements for technologyMy work read by different people – make sure reports are useful to developers but also project managersDesign – prototypes (lo and hi fi) work closely with designers/engineers
Lab – control room and lab space (bachelor pad) Participants come in and sit on the sofas – might using something on TV or just being interviewed, also PC at the back of the room for web Control room is complete darkness so you can see though the one way mirrorAlso cameras on walls which can watch, mics to hear everythingNote takers in here recording everything that is done/said
At the BBC
Typical research project where we don’t know much about the system we want to buildVoice interface for TV which is localised in a mobile device which speaks to your TV6 participants come in and talk about their visual impairmentWant to know about user and current behavioursTV viewing habitsThings that they struggle withWhat they think about a voice interfaceExpectations related to interfaceSampling Participants – all severely visually impaired, familiar with voice technology (want them to be able to talk about it)In session:Informed Consent – very important part of research with participants, ethics, visually impaired – had to be read to themQuestionnaire Design – structure of the interview (loose, let participant tell you thingsAudio & Video recorded
Relaxed, free flowing but with some structure
With note takers, rare to have to fully transcribe the interviewAudio/Video used for referenceThis project very important – learnt at Uni and comes back to haunt you from time to timeTranscription – word by word written out10 mins of interview takes an hour to transcribeMakes you very familiar with the dataRead and re read when analysing
Don’t have any sort of framework for what we’re looking forPull out interesting points, look for themesUnderstanding what they currently do can provide opportunities or the system may need to fit into current behaviourNeeds, expectations, natural behaviours, where the system can provide things they can’t currently do.From thisDocument Requirements – developer can start to prototype a system that fulfils high level needsFrom thisCan’t use the programme guide, wants to browse TV when bored but has to go to the internet to do soParticularly important when alone (wants independence)Relies on others for new information – people in family (sighted) need to be able to use it to encourage usage
BackgroundGuidelines designing for TV, research is flawedText is no longer limited to subtitles and teletext. Red button – text with images, videoInitial research looked at different aspects of typographyAccess needs; sensory & cognitiveWhen I joined, wanted to isolate the size variable of font to inform guidelines for designing for TV
Bit of understanding as to what I did
Part of my role is design basedWork with developers to create screen prototype
One important aspect was the experimental designParticipants want to be consistent in their choicesPrototype included separate screen – I drove so they could only see output of choiceBigger or smaller – preferred and smallest would acceptContinuously developing the methodologyVariety in our audienceParticipantswith access needsHalf Older participants
Controlled scientific experimentExperimental designOther than designing a questionnaire and test scriptHad to understand the TV environmentDistance and size of the screen impact size of text seenAll analysis and subsequent calculations had to account for these 2 measurements
Quantitative analysis mainly – calculating the most appropriate size for TVExcel – could use SPSS but sufficient for my needs2 tailed T test – tests if 2 groups are statistically different from each otherOne of my findings hereComparing older users size preferences with younger groupStatistically differentLook back at means, older users smallest acceptable size larger than youngerImpacts guidelines as means the minimum recommendation needs to be higher than general average
What this translates toNeed to consider who will use the researchThis case, designersSo developed a tool to calculate appropriate text sizesNo one single answer so something flexibleUsedby designers to ensure they are choosing appropriate sizes when designing for TV
Designing an online survey to assess the mood of theme tunesDesigning an experiment to gather a lot of data
Informed ConsentExperimental DesignCounterbalacingThings to think aboutHow many tunesHow many questionsWordingWireframesUsabilityView experimentI wasn’t involved in analysis of data
Context in which you test out a new technologyProduction labs is a psuedo production enviroment to put new techTesting on real sets is costly and time consuming
Ingex logger – PA logging system straight into place where shots are stored so info attachedFieldresearch – observations, much less intrusive, sitting in the backgroundExperimental Design – one used paper then comp and vice versaQuestionnaire – more formal reflective questionsRecommendationsInterface Design
Classic usability studyBBC Red Button Service – glastonbury
3 different templates for presenting red button information – GlastonburyWant to know that each is usable, participants understand how to get around and find contentIdentify issues
Designing session Looking for user journeys to explore different aspects we know may have problemsUse of navigation map to understand how the application/system worksWork closely with the designers/developers to flag things we want to explore and then look for a way through the prototype that will investigate these
CounterbalacingHalf used red button, half not, of half that had used it half of those to have used it for musicCounterbalance participants according to age group, gender, red button use
Sat Ps in front of TV in lab and use remoteQuestionnaire; Play around with it to get used to it and reduce novelty, then task based – can you find ‘beyonce’ in Saturday highlights’Observations and think out loud
Qualitative analysis using bottom up processing of dataAll qual data – use a methodology of post it notes, pull out all relevant findings and then look for trends and consistencies, then input to excel and quantify issues, then when reporting use extensive verbatim quotations to back up resultsWrite report and make design recommendations
Pay for Ps – allows exact type of people to be recruited – not just studentsOften expected to roll things out very quickly – iterative development, changes from research need to be implemented and retested quicklySometimes you do what you can with the resources you have – BBC kidsAll believe in a utopian UX throughout the process, this isn’t often the caseSometimes lumped on the end just to check it works ok – usabilityThings are changing slowlyLearn to communicate your expertise to those that don’t understand itUX is often inconvenience and requires more work – have to be convincing Docs need to be done with UX – usable language, know your audience