This talk describes C-RITE , a method that maximizes impact through cross-disciplinary collaboration on research observation, analysis, and design exploration within an agile, user-centered development framework. We showcase real-world tested techniques that Google’s Android and TV teams have developed and share tactics for reducing logistical overhead in a lean, iterative user-centered design process.
C(ollab) RITE: How to run impactful iterative studies in a fast paced environment
1. C(ollab) RITE
How to run impactful iterative studies in a fast paced environment
Helena Roeber
Jhilmil Jain
Google
#uxpa2013
#crite
@jhilmiljain
@helenaroeber
#androidux
2. C-RITE Impact
• User research insights are adopted quickly
• Product team has clear sense of progress
• Researchers and designers earn
understanding and respect for their work
• Advocates for the user centered design
process are born
#uxpa2013 #crite #androidux
3. C-RITE
• Iterative UX research method suitable for agile
environments
• From research to new design solution in 1 day
• 7 research sessions in 2 days
• Collaborative design workshop each day
• Prototype changes between day 1 and 2
• Can be repeated weekly
• Stakeholders collaborate in all UX phases
#uxpa2013 #crite #androidux
4. User Research Stages
Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
#uxpa2013 #crite #androidux
22. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Research Timing
10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm
Day 1 Day 2
#uxpa2013 #crite #androidux
23. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Collaborative Analysis
10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm
Day 1 Day 2
#uxpa2013 #crite #androidux
25. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
User Goals Example
Observation Example
Goal example
P1P1 P2P2
Bob Lin Ali Nur Bob Lin Ali Nur
User knows what to say 2 1 2 2
The system understood user’s intent 5 3 1 1
The system doesn’t slow down user
w unnecessary confirmations 4 4 2 2
User can easily tell where speech
will work
2 2 2 2
User comfortable using their voice 4 4 4 4
Rating on a scale from 1- 5 1 = poor 5 = good
26. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Collaborative Observation
#uxpa2013 #crite #androidux
• Re-call to count observations, analysis step
reduced
• Disagreements about observations are resolved
immediately
• Research protocol can be adjusted based on
stakeholder feedback
• Researcher can explain what observations
mean
• Small sample size discussion is obsolete
27. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Analysis
10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm
Day 1 Day 2
2:00 pm 3:00 pm
#uxpa2013 #crite #androidux
41. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Reporting
Day 1
Send action items to team
Day 2
Send action items to team Short report
Day 3
#uxpa2013 #crite #androidux
44. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Recruiting
• Create a panel of users that meet
general recruitment criteria
• Panel members commit to availability at
short notice on research days
• Each week, schedule panel members
• Recruit for specific criteria when needed
#uxpa2013 #crite #androidux
45. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Room Design
• Comfortable seating for all
to observe sessions
• Whiteboard to record
observations, and drawing
during design workshop
• Sunlight and windows is
good for creativity
#uxpa2013 #crite #androidux
46. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Preparations
• Set expectations with all stakeholders
beforehand about time commitment,
collaboration and workshop format
• Decision makers need to participate in
C-RITE studies and workshops
#uxpa2013 #crite #androidux
47. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Summary
#uxpa2013 #crite #androidux
• Iterative UX research method suitable for agile
environments
• From research to new design solution in 1 day
• 7 research sessions in 2 days
• Collaborative design workshop each day
• Prototype changes between day 1 and 2
• Can be applied weekly
• Stakeholders collaborate in all UX phases
50. Agile Methods: RITE
• Changes are made as soon as a problem is
identified, sometimes after 1 session
• 1-2 sessions a day. Breaks between sessions long
enough to make changes
• 1-2 weeks of sessions
• Requires stakeholder attendance and participation
Medlock, Wixon, Fulton, Terrano and Romero, UPA, 2002
#uxpa2013 #crite #androidux
51. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
Agile Methods: KRUG + RITE
• 4 participants over 2 days, at least 1 hour
between sessions to make changes
• Researcher sends out observations and
recommendations 30 min after last session
• Team debrief of observations and action items on
day 2, 1 hour after last session
• No collaborative design workshop
McGinn, Chen, Journal of Usability Studies, 2013
52. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
Agile Methods: PULSE
• 3 participants, 1 day every week
• Analysis and write-up of results by researcher
within 2 days
• No collaborative analysis and design workshop
Konno, Agile 2012
53. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Variations We’ve Tried
• C-RITE Remote
• If time zones allow, stakeholders participate through ‘Hangouts’
• Sessions are recorded and watched with time delay
• Next day debriefs and design workshops for incompatible time
zones
• C-RITE Diary Studies
• In-lab diary kick-offs and debriefs are run in C-RITE format
• Researcher analyzes and summarizes diary data for debrief and
design workshop