Jonathan Rubin, First Fridays Usability Program. July 10, 2013

1. Howto.gov/firstfridays
Top User Experience
Challenges in Government
Jonathan Rubin
First Fridays Usability Program
July 10, 2013

3. Howto.gov/firstfridays
1. Survey
Results: Gov
UX Challenges

4. Our Survey of UX Activities
1. Survey Monkey with 65 respondents
2. 100% Cabinet agencies
3. Co-created with Jean Fox (+ thanks Axle
Brown)
4. Not scientific or representative
5. Goal:
• Audience research
• Develop web-based and in-person training
• Share aggregate in presentations, etc.
Howto.gov/firstfridaysHowto.gov/firstfridays

5. What we asked
1. Agency (but anonymous)
2. About you – title, UX education, team
size
3. How widespread is UX
4. Challenges to implementing UX
5. Resources
6. Types of UCD methods
7. What would help?
Howto.gov/firstfridaysHowto.gov/firstfridays

6. Good news
• Average team: 2 PT and 2 FT
• Wide range of UX methods being used
• #1 surveys, #2 style guide, #3 tests
• Thirst for training, many self-taught
• 91% doing UX on websites, ⅓ mobile
• Using respected tools (20% eyetracking!)
• Future: More testing, services
• Some agencies have UX institutionalized
Howto.gov/firstfridaysHowto.gov/firstfridays

7. Challenges / Lack of…
• Awareness:
• “[People] refuse to believe the results of
usability tests because they are painful.”
• “We are spending hundreds of thousands of
dollars without one usability test. Web staff
understand the need, but no one is listening.”
• Facilities
• “No facilities” / “Our lab is about 15 years
young”
Howto.gov/firstfridaysHowto.gov/firstfridays

8. Challenges / Lack of…
• Resources
• “Very old models are used for interaction”
• “Army of one”
• Training
• “Figure it out as we go along”
• Standards / Best Practices
• “Applying standard best practices can be
challenging… many offices and components
have different processes and standards.”
Howto.gov/firstfridaysHowto.gov/firstfridays

9. Howto.gov/firstfridaysHowto.gov/firstfridays

10. 34%
17%15%
15%
6%
6%
5%
2%
What best describes what you do?
Website Mngr
Other
Project Mngr
UX Specialist
Web Designer
IA
Researcher
Web Metrics Mng

11. # UX staff
Howto.gov/firstfridaysHowto.gov/firstfridays
Part Time Full Time
Had at least 1
person
86% 59%
Median # of
people
2 2
Average # of
people
2 3
Range of
people
0-10 0-30

12. Howto.gov/firstfridaysHowto.gov/firstfridays
0%
10%
20%
30%
40%
50%
60%
70%
College /
Grad school
Certification Workshops,
webinars
On-the-job
mentorship
Other
Where did your staff get UX training?

13. Howto.gov/firstfridaysHowto.gov/firstfridays
3%
19%
46%
25%
8%
0% 10% 20% 30% 40% 50
op to Bottom UX
Used frequently
Occasional UX
Know but don't
practice
No one knows
UX
How widespread is UX at your organization?

14. Howto.gov/firstfridaysHowto.gov/firstfridays
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
What types of UX products do you work on?

15. Howto.gov/firstfridaysHowto.gov/firstfridays
% Who “Often” or “Always” use
0% 10% 20% 30% 40% 50%
Tests: Mobile
Contextual inquiries
Focus groups
Tests: Remote
Personas
Tests: High Fidelity
User Analysis
Expert Reviews
Task Analysis
Tests: In-person
Surveys
Style guides

16. Howto.gov/firstfridaysHowto.gov/firstfridays
% Who “Never” use these methods
0% 10% 20% 30% 40% 50% 60%
Personas
Tests: Remote
Context. Inquiries
Tests: High Fidelity
Test: Mobile
Card sorts
Tests: Low Fidelity
Affinity diagrams
Tests:: First-Click
Tests: Eyetracking

17. Howto.gov/firstfridaysHowto.gov/firstfridays
How often do you perform usability tests?
0
5
0
5
0
5
Never Rarely Sometimes Often Always Don't Know
In person Remote

18. Asked about UX methods
• Top 3: Surveys, Style guide, Tests
• Bottom 3: Affinity diagrams
Eyetracking, First Click
• 10% never a usability test – bad?
• 16% never a survey
• Task analysis - 8.5% always, 55%
sometimes
Howto.gov/firstfridaysHowto.gov/firstfridays

19. Training Needs
• “UX Training classes - webinars aren't
enough”
• “Software” - Morae, Surveys, CrazyEgg
• “Any and all”
• “Be more aware of findings”
• “How to get buy-in from higher ups “
• Methods: First-Click Testing, Style Guides,
Card Sorts, Surveys, Task analysis
Howto.gov/firstfridaysHowto.gov/firstfridays

20. Plans for the future
Howto.gov/firstfridaysHowto.gov/firstfridays
Asked
about
future
plans:

21. Howto.gov/firstfridaysHowto.gov/firstfridays
Biggest obstacle to using UX?
0% 10% 20% 30% 40% 50% 60% 70% 80%
Tried UX, failed
Other
No UX awareness
Don't get UX ROI
UX not built in
Lack of time
No resources

22. Howto.gov/firstfridaysHowto.gov/firstfridays0% 10% 20% 30% 40% 50% 60%
One-way mirror
Observation room
Testing room
Other
Multi-purpose rooms
No space for testing
Which do you have? (check all)

23. Challenges / Lack of…
• Resources - staff and metrics software
• Awareness – Managers not sold on UX
• Training - Staff and Manager
• Standards / Best Practices
• Testing facilities
Howto.gov/firstfridaysHowto.gov/firstfridays

24. Howto.gov/firstfridays
2. Top Gov UX
Problems on
Sites

25. Two types of gov challenges:
1. Gov UX Strategic and Resource
challenges(Survey)
2. Persistency UX problems on fed
websites (FF test data)
Howto.gov/firstfridaysHowto.gov/firstfridays

26. FF Test Data
1. 29 products reviewed in 2012
2. 130+ significant problems
3. 32 different usability categories -
problems big and small
Thanks to Cynthia, Molly, Josh, and Ruth!
Howto.gov/firstfridaysHowto.gov/firstfridays

27. Top 6 Usability Challenges
1. Navigation difficult, confusing or hidden
2. Search feature hidden or doesn’t work
3. Forms difficult to use
4. Purpose of site unclear
5. Difficulty with Top Tasks
6. Lack of Plain Language
Howto.gov/firstfridaysHowto.gov/firstfridays

28. 1. Navigation difficult to use
Howto.gov/firstfridaysHowto.gov/firstfridays

29. Before After

30. • Average daily visits up 50%
• Mobile Home visits up 32%
• Help Page visits up 1200%
• Desktop site visits up 3.3%
Howto.gov/firstfridays

31. 2. Poor search functionality
Howto.gov/firstfridaysHowto.gov/firstfridays

32. Howto.gov/firstfridaysHowto.gov/firstfridays

33. Howto.gov/firstfridaysHowto.gov/firstfridays

34. Howto.gov/firstfridaysHowto.gov/firstfridays

35. Howto.gov/firstfridaysHowto.gov/firstfridays

36. Howto.gov/fiHowto.gov/f
Before: 7 text items to scan
After: 3 text items to scan

37. 3. Forms require too much effort

38. 3. Forms difficult to use
1. Too much effort
2. Poor terminology
3. Malfunction / deleted answers
4. Too many questions
5. Unclear what’s required

39. 3. Forms difficult to use
1. Too much effort
2. Poor terminology
3. Malfunction / deleted answers
4. Too many questions
5. Unclear what’s required

40. 4. Purpose of Site Unclear

42. 5. Difficulty with top tasks

43. 5. Difficulty with top tasks
1. Can’t find info via any means or know
where to begin

44. 5. Difficulty with top tasks
1. Can’t find info via any means or know
where to begin

45. 6. Lack of Plain Language

47. 6. Lack of Plain Language
1. Insider Speak
2. Technical / scientific terms
3. Jargon
4. Alphabet soup

48. Top 6 Usability Challenges
1. Navigation difficult, confusing or hidden
2. Search feature hidden or doesn’t work
3. Forms difficult to use
4. Purpose of site unclear
5. Can't find what they are looking for
6. Lack of Plain Language
Howto.gov/firstfridaysHowto.gov/firstfridays

49. Wrap up: Opportunities
• Howto.gov
• Us – howto.gov/firstfridays
• Blog.howto.gov – 3,000+ subscribers
• DigitalGov University - howto.gov/training
• User Experience Community of Practice
• Usability Testing Community of Practice

51. Share your greatness
• Email us at firstfridays@gsa.gov or let people
know about:
• Labs or conference space to share?
• Great tools or resources (e.g. Expert Eval tool or
Persona template)
• Best Practices
• Testimonials
• ROI stats
• Speakers available

52. First Fridays Usability Program
Jonathan Rubin
Jonathan.rubin@gsa.gov
Twitter: @govnewmedia
www.Howto.gov/firstfridays
Howto.gov/firstfridaysHowto.gov/firstfridays