(Workshop for Business Analysts) When we say "Usability Testing", many immediately picture a prototype being presented to a project working group. On a good day, the working group would have one or two users of the system. More often than not, our working groups comprise of department managers and project resources. None of whom will be left using the system. The act of doing the same thing over again and expecting different results is Einstein's definition of insanity. Usability Testing takes a different approach with regards to understanding a user's behaviour and interactions with the system. Diana Adorno and Valeria Spirovski

1. 1
USER TESTING
Diana Adorno & Valeria Spirovski
December 3, 2014
#bamperth
@dianaadorno @charmedquark
Business Analyst Meetup
Photo credit: Diana Adorno

2. FORMAT
• Questions before we start
• Intro & frame
• Usability test demo
• Practice facilitating a test
• Questions & Discussion
2

3. HOUSEKEEPING
• Emergency procedure
• Bathrooms
• Considerations
• Questions
3

4. WHAT IS USER
TESTING? 4

5. WHY DON’T WE JUST ASK PEOPLE WHAT THEY THINK?
5

6. Usability testing
is the killing field of
cherished notions.
DAVID ORR
43

7. If it costs $1 to make a
change on paper, then it
costs $10 in code and
$100 when the site is up.
THERESA CUNNINGTON
COMPUTERWORLD 2006
37

8. DEMO: USER TEST (OPEN)
8
Photo credit: Diana Adorno

9. DEMO: USER TEST (CLOSED)
9
www.margaretriver.com

10. BASIC FORMAT FOR A USER TEST SESSION
welcome
a few background questions
think aloud
the test
the gift
10

11. TRY THIS
11

12. TRY THIS…
WORK IN PAIRS
One user, one test facilitator
■ 1 scenario * 2 tasks * Speaking aloud * Take a few notes
SCENARIO
Planning a holidays to Broome with a few
friends
12

13. THE TASKS
! Task 1: Find 4 places to visit around
Broome
! Task 2. Find 4 places to visit around
Broome using the WA Tourism site
(http://www.westernaustralia.com/au/Pages/
Welcome_to_Western_Australia.aspx)
13

14. SHAREBACK
Success or failure
14

15. KEY QUESTIONS
ABOUT USER
TESTING
15

16. APPEARANCE
Visual and UI Design
INTERACTION
Navigation and interactive elements
INFORMATION
Content & Information Architecture
STRUCTURE
The relationship between the product
components and the organisation
PROPOSITION
How product provides value to meet the
users needs, while being financially viable
& technically feasible
CONCEPT
The idea of how the value is delivered
to the user/customer
WHAT ARE YOU TESTING FOR

17. TYPES OF TESTING
Exploratory (formative)
! Use competitor sites
! Paper ideas
! Your existing site
Validation (summative)
! Test the actual system
! Test as you go
17

18. What are you looking for…
Effectiveness
• Do they reach their goal?
Efficiency
• How easy was it to reach the goal?
• time to complete, mistakes
• What made it difficult, what was confusing, backwards
and forwards-ing, needing help to complete
Satisfaction
• Emotional response
18

19. 3-clicks rule is
dead
19

20. WHO TO TEST WITH
20

21. HOW MANY PEOPLE TO TEST WITH?
Benefit
Cost
5
http://www.nngroup.com/articles/how-many-test-users/

22. BUDGET FOR USABILITY STUDIES
20 users
5 5 5 5

23. WHAT SYSTEM TO
TEST WITH
23

24. PAPER
PROTOTYPES 24

25. ASK PARTICIPANTS TO INTERACT
http://www.nngroup.com/reports/prototyping/video_stills.html
25

26. 26
MOCKUPS

27. INTERACTIVE PROTOTYPES
27

28. 28

29. 29
IMAGES WITH
LINKS

30. 30
EXISTING
SYSTEM OR
COMPETITOR

31. 31
INVISIBLE
SYSTEM

32. WHAT SHOULD YOU
DO?
32

33. 33
WHERE TO TEST
In the project room

34. 34
IN DESIGN WORKSHOPS

35. 35
IN THE FIELD

36. GUERRILLA USER TESTING

37. 37
RECORD
WHAT YOU
SEE
o credit: Diana Adorno

38. RECORD WITH A VIDEO
CAMERA
38
Photo credit: Diana Adorno

39. OR JUST USE THE CAMERA IN YOUR
LAPTOP
39

40. SILVERBACK & OTHER RECORDING
TOOLS
40

41. 41
Take notes on
a laptop

42. OR POST-IT NOTES
42

43. 43
On sticky notes
(observer out
of shot)

44. WHAT TO CAPTURE IN NOTES
• Only what you see (don’t summarise)
• Direct quotes
• Directly observed actions or problems
44
NOTE: Summarise later
when you distill with
your team

45. DISTILLING RESULTS
45

46. POST-IT NOTES GROUPED INTO THEMES/INSIGHTS
46

47. STRAIGHT ONTO THE SCREENS
47

48. Share back with your team as
soon as possible
48

49. SHARING THE
FINDINGS
49

50. HOW SHARE WHAT YOU DISCOVER
• Take video of the session
• Photos
• Share back the relevant parts
• Include different members of your team
or stakeholders as observer
• Share direct quotes
“It is not about the time it takes, it is about annoyances. If it is
annoying, I will spend a minute being annoyed”
50

51. WHAT CAN GO
WRONG
51

52. AGGRESSIVE PARTICIPANT
Thank them for their input
Give them their gift
Stand up
Move to the door
Open the door
52

53. NO TASKS
Ask a few questions so that they describe
actual tasks
Ask them about the last time, they actually
did x and use that
Work with the business stakeholders to
come up with realistic scenarios and likely
tasks
53

54. FLYING SOLO
Record – audio or video
Run through the test and then do a
walkthrough with the participant and mark
up the designs together
54

55. FORGET WHAT TO DO
Write a script
Helps if there are several things to test
55

56. EXAMPLES &
READING & TOOLS
56

57. READ & WATCH
Rocket Surgery Made Easy, Steve Krug
http://www.sensible.com/
Video of a User Test (Steve Krug)
www.peachpit.com/promotions/promotion.aspx?promo=137602
57

58. TOOLS
RECORDING
Silverback http://silverbackapp.com/ (Mac)
REMOTE TESTING
www.usertesting.com
http://www.loop11.com/usability-testing/
58

59. THANK YOU
diana.adorno@thoughtworks.com
valeria.spirovski@woodside.com.au