Using SUS to measure perceived usability in user interface design.

1. Benchmarking Using
SUS
Kick Ass UX Hour
AUGUST 2, 2016

2. SUS / System Usability Scale

3. About SUS

4. A 10 question
survey rating
perceived
system
usability.
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this
system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10.I needed to learn a lot of things before I could get going with this system.
Items vary from positive to negative ratings. Users rank each question
from 1 (strongly disagree) to 5 (strongly agree) Likert scale.

5. Launched in 1986
by John Brook at DEC

6. Free!

7. • Correlated with hundreds of
systems and thousands of users
of known usability
• Better reliability than
commercial surveys.
SUS is
Reliable
— measuringu.com

8. measuringu.com has a database
of 500+ studies (for a fee). Can
drill down on software type, do
custom analysis
SUS is
Benchmarked
• Average score is 68
• For consumer software average is 72

9. SUS Score >
with
Familiarity
— measuringu.com

10. SUS can be reliably used with low
numbers of users (10 - 15)

11. SUS is not diagnostic and doesn’t
correlate with actual task
performance

12. Let’s Try It:
https://docs.google.com/a/
cakeandarrow.com/
spreadsheets/d/1aMZ4uhX1Bz-
U05oZcxOX2m_JXtxHT_72tAFGn
8XBDgc/edit?usp=sharing

13. Scoring SUS

14. Scoring
normalizes
even and odd
questions to a
4 point scale
1. I think that I would like to use this system frequently. [4] - 1 = 3
2. I found the system unnecessarily complex. 5 - [2] = 3
3. I thought the system was easy to use. [3] - 1 = 2
4. I think that I would need the support of a technical person to be able to use this
system. 5 - [2] = 3
5. I found the various functions in this system were well integrated. [1] - 1 = 0
6. I thought there was too much inconsistency in this system. 5 - [5] = 0
7. I would imagine that most people would learn to use this system very quickly.
[3] - 1 = 2
8. I found the system very cumbersome to use. 5 - [4] = 1
9. I felt very confident using the system. [1] - 1 = 0
10.I needed to learn a lot of things before I could get going with this system. 5 - [1] = 4
18 * 2.5 = 45 SUS Score
Odd Questions: [Score] - 1
Even Questions: 5 - [Score]
[Total] * 2.5 = SUS Score

15. Not a percentage!
80.3 is an ‘A’ - also correlates to
NPS Promoters
Average score is 68
51 is an F (bottom 15% of results)
Scores are 0
to 100
— measuringu.com

16. • Helps provide a frame of
reference to stakeholders.
• Scale can also be phrased this
way to survey takers, though
it’s usually better to stick with
the original scale.
Adjectives
to map
scores
— measuringu.com

17. Why Perceived Usability?

18. Perceived Usability != Actual Usability

19. But usability includes satisfaction. Not
just success. Attitude and action.

20. PERCEIVED USABILITY IS IMPORTANT

21. High perceived usability impacts first impressions.
User impressions are impacted to varying degrees by
subsequent usage and actual usability.
PERCEIVED USABILITY IS IMPORTANT

22. High perceived usability increases trust and loyalty. Trust
and loyalty are correlated to perceptions of usability.
PERCEIVED USABILITY IS IMPORTANT

23. SUS and Customer Loyalty

24. How likely are you to recommend
this product to a friend or
colleague?
MEASURING CUSTOMER LOYALTY - NPS

25. SUS score of 70 will generate an
approximate NPS of 7 and a SUS
score of at least 88 is needed to
be a promoter (9+)
SUS CORRELATES WITH NPS
— measuringu.com

26. • Promoters have an average SUS
score of 82
• Detractors have an average
score of 67
Scores are 0
to 100
— measuringu.com

27. Can predict between 30% and 60% of customer loyalty.
Highly correlated with loyalty scores and can be used to
predict NPS.
PERCEIVED USABILITY IS IMPORTANT

28. How & When to Use It

29. Ask the questionnaire to each user using a Google
Spreadsheet or TryMyUI. Benchmark vs other system and
subsequent tests.
http://www.trymyui.com/blog/2014/10/03/the-system-
usability-scale-a-walk-through-our-newest-user-
testing-feature/
AFTER / DURING EVERY USER TEST

30. Get a pulse on overall usability to make the case for more
testing or to assess what you’re dealing with.
IF YOU CAN’ T DO A FULL USER TEST

31. Use site intercept or email-based surveys to establish a
baseline for an existing software system
IN A SURVEY

32. Coda: SEQ

33. • Measures single task
performance satisfaction
• Versus SUS which measures
overall perceived satisfaction
• Ask after every major test task.
• Helps discriminate about which
tasks caused issues.
SEQ
— measuringu.com

34. • Can we establish a C&A SUS database as a part of our usability offering as a
differentiator?
• SUS scores categorized according to level of fidelity of thing we’re testing, client
industry, user type, etc.
• Over time (6 months?) could have a decent data set to write about, tout to clients,
actually use!
Ideas, Thoughts & Questions

35. Limitations

36. BUT . . .
SUS doesn’t break down
measures granularly into sub-
factors like trust, learnability,
or different UI factors (i.e,
terminology vs functionality)

37. There are other
survey tools
that do this,
often
commercial.
• SUMI http://sumi.uxp.ie
• QUIS http://lap.umd.edu/quis/
• SUPR-Q http://www.measuringu.com/products/suprq
• and many other custom / proprietary ones . . .
Check out measuringu.com