Slides from the TryMyUI #ourUXflow webinar on July 23, 2015, about how to reach team synchronicity in an iterative UX design process. Ritvij Gautam, CEO of TryMyUI, and Jennifer Romano Bergstrom, UX Researcher at Facebook, shared insights on iterative UX and how to create an efficient team workflow. Every product team member needs (and sees) different things from the UX data. Together, UX Researchers, Stakeholders, UI/UX Designers, and Developers can collaborate in the research cycle to better inform the design process and expedite the analysis and implementation of findings. Watch the full video recording of Rit and Jen's conversation on YouTube at: https://www.youtube.com/watch?v=syIhAYFIO8M Read the follow-up Q&A session from the presentation on our blog at: http://blog.trymyui.com/2015/12/data-is-useless-webinar-qa/ "Everyone has seen a usability report before that's like, 'Half of the users liked this version, half of the users liked that version.' And that's where it stops. What we're really trying to tell you today is to think beyond that. What does it mean? What's the impact on the user? What's the impact on the business? How can we work around this pain point?"

1. Team Synchronicity in an
Iterative Design Process
July 23, 2015 | 11:00 am PDT
TryMyUI webinar
#ourUXflow
@romanocog @trymyUI
Jen Romano-Bergstrom
UX Researcher | Facebook
Marketing Director | UXPA
jenrb@fb.com
Ritvij Gautam
CEO & Co-Founder | TryMyUI
ritvij@trymyui.com

2. Iterative Usability Testing
2
Iterative Process Throughout Development
#ourUXflow
@romanocog @trymyUI

3. Iterative Usability Testing
3
Iterative Process Throughout Development
Benchmark
#ourUXflow
@romanocog @trymyUI

4. Who is involved
4
SATQ
Items
Other
materials
Debriefing
Reports
.
. .
.
.
.
.
“Make it a spectator sport” @skrug
Project
Managers
UX
Designers
Executives
Engineers
Researchers
#ourUXflow
@romanocog @trymyUI

5. Why people do not conduct
iterative testing
5
• “There is no time.”
• “We don’t have a budget.”
• “That’s way too many people to coordinate.”
• “I can’t recruit representative users quickly.”
• “We don’t have a lab.”
#ourUXflow
@romanocog @trymyUI

6. Benefits of
Unmoderated Remote Testing
6
• No moderator bias
• No coaching1,2
• Participants in natural
environment
• Hard to reach participants
• Lots of data quickly
• Data from many cities,
states, countries
• No travel costs
1. Olmsted-Hawala, E. L. & Romano Bergstrom, J. C. (2012). Think-aloud
protocols. Does age make a difference? Proceedings from the Society
for Technical Communication Summit, May 2012, Chicago, IL.
2. Olmsted-Hawala, E. L., Murphy, E. D., Hawala, S., & Ashenfelter, K. T.
(2010). Think-aloud protocols: A comparison of three think-aloud
protocols for use in testing data-dissemination web sites for usability.
Proceedings from CHI, April 2010, Atlanta, GA.
Higher accuracy and satisfaction
when moderators “coach.”
#ourUXflow
@romanocog @trymyUI

7. 7
Qualitative Quantitative
Self-Report
Satisfaction and knowledge
questionnaires
YES YES
Verbal think aloud YES NO
Moderator follow up NO NO
Observational
Time on page/task NO YES
Selection/click behavior YES YES
Success/fail rate NO YES
Conversion rate YES YES
Implicit
Verbalization analysis YES YES
Eye tracking YES YES
#ourUXflow
@romanocog @trymyUIUnmoderated Remote UX Data

8. Iterative Process – example timeline
Product Overhaul
8
Task
development
Iteration 1
testing:
6 days
N=7
Iteration 1
findings and
recommendations
presented to design
team:
13 days later
9.8.08 10.23.08
11.10.08
Meeting
with design
team:
13 days later
Iteration 2
testing:
1 month
N=14
12.12.08 6.23.09
Iteration 2
findings and
recommendations
presented to design
team:
8 days later
7.31.09
Meeting
with design
team:
2 weeks
later
8.12.09
Iteration 3:
Novice
users
testing:
8 days
N=9
9.09.09
Iteration 3: Novice
users
findings and
recommendations
presented to design
team:
8 days later
Meeting
with design
team:
2 weeks
later
10.08.09
9.25.09
Iteration 3:
Expert
users
testing:
3 days
N=4
10.14.09
#ourUXflow
@romanocog @trymyUI
Romano Bergstrom, J. C., Olmsted-Hawala, E. L., Chen, J. M., & Murphy,
E. D. (2011). Conducting iterative usability testing on a Web site:
Challenges and benefits. Journal of Usability Studies, 7, 9-30.

9. 9
Task
development
Iteration 1
testing:
6 days
N=7
Iteration 1
findings and
recommendations
presented to design
team:
13 days later
9.8.08 10.23.08
11.10.08
Meeting
with design
team:
13 days later
Iteration 2
testing:
1 month
N=14
12.12.08 6.23.09
Iteration 2
findings and
recommendations
presented to design
team:
8 days later
7.31.09
Meeting
with design
team:
2 weeks
later
8.12.09
Iteration 3:
Novice
users
testing:
8 days
N=9
9.09.09
Iteration 3: Novice
users
findings and
recommendations
presented to design
team:
8 days later
Meeting
with design
team:
2 weeks
later
10.08.09
9.25.09
Iteration 3:
Expert
users
testing:
3 days
N=4
10.14.09
#ourUXflow
@romanocog @trymyUI
Romano Bergstrom, J. C., Olmsted-Hawala, E. L., Chen, J. M., & Murphy,
E. D. (2011). Conducting iterative usability testing on a Web site:
Challenges and benefits. Journal of Usability Studies, 7, 9-30.
Iterative Process – example timeline
Product Overhaul

10. 10
• One morning a month with 3 users (Krug)
• 12 people in 2 days (Lebson & Romano Bergstrom)
• 16 people in 3 days (Patel & Romano Bergstrom)
#ourUXflow
@romanocog @trymyUI
Iterative Process – example timeline
Product Overhaul

11. Iterative Process – example timeline
Unmoderated Remote Testing
11
• Thousands of people in one day
• From different cities
• Test different designs
• Team is free to work on different things and assess data
as it comes in
#ourUXflow
@romanocog @trymyUI

12. A Typical UX Research Process
12
Data Collection:
● Create structured user flows for your website and have users navigate through them
● Get video/audio recordings and written responses to questions (PRIMARY RAW DATA)
● Additional data like SUS & SEQ, click streams, & heat-maps (AUXILIARY RAW DATA)
Data Analytics
● User researchers watch the videos and create notes of pain points.
● Share notes with the team -- they too watch the videos and add notes of their own.
● Collate the data, present a list of actionables to the Stakeholder/Client, explain why the
changes are necessary.
● Relay the changes and desired impact to the Dev team with relevant data. ITERATE...
#ourUXflow
@romanocog @trymyUI

13. Challenges
13
● Data != Insights
● Covering the different takeaways
from the results for the different
team members.
● Getting people “in on the loop” &
“on the same page”, so that they can
“circle back around” to “re-interpret
data”
● Getting all of this done in time for
the next build!
#ourUXflow
@romanocog @trymyUI

14. Data Analytics
14
● In doing UX research, we often get carried away with
the data collection process. In fact there is a terrible
pre-conceived notion that usability testing is purely a
data collection activity.
● We can collect primary & auxiliary raw usability data,
but if we do not establish an analytical methodology
that drives the way we extract insights from them, we
are simply wasting time.
● The auxiliary data should help you identify patterns
and trends in the primary data, but what you need is
a tool that will help you analyze and interpret it all.
#ourUXflow
@romanocog @trymyUI
“Big data is not about the data.”
-Gary King, Harvard University

15. Example Auxiliary Data
15
#ourUXflow
@romanocog @trymyUI

16. Time Sensitivity
16
● The more tools/vendors/platforms used to manage
the data, the more unwieldy the analytical process.
● The more isolated the team members’ respective
analysis, the more disjointed the analytical process.
● Analysis that could take a week often takes twice or
thrice the amount of time because of gaps in
communication and the time wasted in between
meetings and follow ups.
#ourUXflow
@romanocog @trymyUI
“The ultimate inspiration is the
deadline.”
- Nolan Bushnell, Founder of Atari

17. Team Synchronicity
17
● Usability testing data is relevant to your team members in different ways.
● They all need something from it, they view it through different lenses, and they draw
different insights from it.
● Aggregation of these insights will lead us to the treasure we seek!
#ourUXflow
@romanocog @trymyUI

18. Team Synchronicity: UX
Researcher
18
#ourUXflow
@romanocog @trymyUI
It falls to the UX Researcher to watch all of the results and
identify key findings, judge what is important and what isn’t,
and then persuasively communicate these to decision makers.
To achieve his or her goals, therefore, what the UX Researcher
really needs is a way to easily pinpoint and reference critical,
demonstrative moments in the results to back up their
arguments and justify making changes to the design.
UX Researcher
Decision Maker
UI/UX Designer
Developer

19. Team Synchronicity: Decision Maker
19
#ourUXflow
@romanocog @trymyUI
The Decision Maker’s objective, on the other hand, is to make
an informed decision about the direction of the product
roadmap and which issues will be prioritized over others, based
on the topline analysis of what the pain points are.
What the Decision Maker needs to achieve their goals is an
efficient means of knowing and understanding the issues, and
seeing the evidence needed to green-light changes.
UX Researcher
Decision Maker
UI/UX Designer
Developer

20. 20
#ourUXflow
@romanocog @trymyUI
The goal of the Designer, for their part, is to see why particular
design elements work or do not work, and then use that
information to create new solutions based on real user
behavior.
For the Designer to achieve his or her goals, they need to be able
to directly access the results and witness users interacting with
the product at key junctures and hear their thoughts and
reactions when they run into walls.
Team Synchronicity: UI/UX Designer
UX Researcher
Decision Maker
UI/UX Designer
Developer

21. Team Synchronicity: Developer
21
#ourUXflow
@romanocog @trymyUI
The Developer’s role is to determine how & where in the
usability test (and the production code) a functionality issue
arises. They need to see the actions leading up to it so that they
can replicate it, identify it and solve it.
For the Developer to achieve his or her goals, they need to be
able to see the user’s actions leading up to an error and also see
which aspects of recurring user behaviour will dictate the way
interactive elements are coded.
UX Researcher
Decision Maker
UI/UX Designer
Developer

22. Why do we need to collaborate?
22
It is all about bettering the UX of Usability Testing:
● Collaboration is inevitable. The data/data-derivatives will have to reach all members of the team
at some point. So why not optimize our workflow for it?
● Valuable insights with less organziational inefficiency.
● Increase:
o Relevance of data
o How many insights each unit of data yields
● Decrease:
o Time taken to deliver & implement the suggestions for change
o Entropy and miscommunication within the team
#ourUXflow
@romanocog @trymyUI

23. Why do we need to collaborate?
23
It is all about bettering the UX of Usability Testing:
● Collaboration is inevitable. The data will have to reach all members of the team at some point. So
why not optimize our workflow for it?
● Valuable insights with less back and forth
● Increase:
o Relevance of data
o How many insights each unit of data yields
● Decrease:
o Time taken to deliver & implement the suggestions for change
o Entropy and miscommunication within the team
#ourUXflow
@romanocog @trymyUI

24. Why do we need to collaborate?
24
#ourUXflow
@romanocog @trymyUI

25. Why do we need to collaborate?
25
#ourUXflow
@romanocog @trymyUI

26. Why do we need to collaborate?
26
#ourUXflow
@romanocog @trymyUI

27. Why do we need to collaborate?
27
#ourUXflow
@romanocog @trymyUI

28. Why do we need to collaborate?
28
#ourUXflow
@romanocog @trymyUI

29. Team Synchronicity in an
Iterative Design Process
July 23, 2015 | 11:00 am PDT
TryMyUI webinar
#ourUXflow
@romanocog @trymyUI
Jen Romano-Bergstrom
UX Researcher | Facebook
Marketing Director | UXPA
jenrb@fb.com
romanocog.com
Ritvij Gautam
CEO & Co-Founder | TryMyUI
ritvij@trymyui.com
Blog.trymyui.com
Trymyui.com/webinar