Originally delivered at BayCHI in April 2019, a talk on research projects that go beyond user research to answer large, broad questions

1. Researching
where no one
has gone before
Cyd Harrell
BayCHI April 9, 2019

2. 2008: How do people really
use phones in their cars?

3. Research methods & skills:
- Ride-along interviews
- Broadcasting
- Observer management
- Photography
- Video editing

4. We disappeared

5. Center for Civic Design

7. 2012: Field Guides to
Ensuring Voter Intent

8. Research question:
What do voters want to know from
(local govt) election websites?

9. Method hypothesis:
Remote moderated interviews

10. Remote moderated interviews:
- Recruiting plan
- Discussion guide
- Recording protocol
- Synthesis plan

11. So. Many. Ways.

12. Research question:
What do voters want to know from
election websites?
Not-so-side quest:
What’s up with election websites
anyway?

13. Local election officials

14. 50 states
3,144 counties
19,354 cities & towns

15. Constructing a sample

16. A structured way to assess homepages

17. Election site presence check

18. What voters need to know

19. Electiontools.org

20. Research methods & skills:
- Remote moderated interviews
- Volunteer cohort management
- Structured site analysis
- Busywork (presence checks)
- Sample construction
- Conversational synthesis

21. Synthesis is a
conversational practice

22. Epic Journey of American Voters (privileged version)
Dana Chisnell, 2017
https://medium.com/civic-designing/the-epic-journey-of-american-voters-ed07bd0e6c57

23. Epic Journey of American Voters (burdened version)
Dana Chisnell, 2017
https://medium.com/civic-designing/the-epic-journey-of-american-voters-ed07bd0e6c57

24. 2014: Digital Front Door

25. Research question:
How can we help cities run useful
websites?

26. Method hypothesis:
- Citizen survey
- City staff interviews
- Some kind of analytics

27. Municipal service use cases:
pay a parking ticket, attend a council meeting
register for summer camp, find work with the city
get a building permit, call an ambulance, street planning
renew a business permit, find swimming pools, get a library card
report a pothole, find out about parks, report trash on a sidewalk
get a parking permit, leave public comment
pay your property taxes, apply for a special event permit
request a streetlight, find the police non-emergency number
take a CPR training from the fire department
find your recycling day, call your council member

28. Municipal sites serve
200-300 infrequent
transactions.

30. 50 states
3,144 counties
19,354 cities & towns

31. Research questions:
- What are city website teams prioritizing
& why?
- What does the public want?
- What does the public expect?
- How can we define a purpose for a
really general website?

32. Live dashboard showing page titles

33. A starting place

34. A starting place

35. 1,000 resident survey

36. 1,000 resident survey

37. Research questions:
- What are city website teams prioritizing & why?
- How are city staff running websites & what
barriers do they encounter?
- What does the public want?
- What does the public expect?
- How can we define a purpose for a really general
website?

38. Staff web editor interviews

39. Synthesis with policy & IT folks

40. Research methods & skills:
- Moderated interviews
- Survey design
- Map & data science
- Facilitated synthesis
- Web analytics

41. Municipal service use cases:
- Parking - pay a ticket, get a permit
- Jobs - find work with the city
- Permitting - building, planning, business, event
- Recreation - summer camp, swimming pools, libraries
- Maintenance - potholes, streetlights, parks, recycling
- Democracy - comment, meetings, property taxes
- Public safety - police, fire, ambulance

42. 2016: Best Practices in
Digital Transformation

43. Research question:
What are the best practices
for government digital
transformation?

44. Method hypothesis:
- Interviews??
- ¯_(ツ)_/¯

45. FASPBS TTS
ITC

46. Research question:
What makes modern digital
practices stick within a
government entity, beyond a
single innovation project?

47. Assumptions:
- Transformation is not as risky as government
agencies may perceive it to be.
- Transformation doesn’t have an endpoint; you
work on it continually.
- Changing culture to be more open and agile
makes government services more
user-centered and builds trust.
- There’s not one right way to do this work.

48. Interview questions:
- What does digital transformation mean in
government?
- What is the goal or end-state? How do you know when
you’re done?
- What are the biggest obstacles to this work? How can
we help people get past those obstacles?
- Is there one digital transformation process or many?
- How do you make the changes last?
- What resources would be valuable for other agencies
doing this work?

49. Research methods & skills:
- Moderated interviews
- Literature review
- Policy & regulation review
- Cluster recruiting
- Tight consent & confidentiality

50. When companies were going digital
New York Times, 1999
https://www.nytimes.com/1999/12/23/business/toys-r-us-falls-behind-on-shipping.html

51. Cluster recruiting:
Who else in the organization
had an important role in this
moving forward or not?

52. bit.ly/CydBPDT

53. 2018: Self-Help in the
CA Court System

55. Research question:
What’s the actual experience of
people representing themselves in
the civil court system?

56. Non-criminal case types:
- Family law - divorce, parentage, custody
- Small claims - lawsuits under $10,000
- Many restraining orders
- Probate law - estates, guardianship, conservatorship
- Civil law - lawsuits, housing, name/gender changes, more

57. 1 supreme court
6 appellate courts
58 superior courts
254 courthouses

58. Method hypothesis:
Intercept recruiting & interviews
at courts

59. Research questions:
- How do judges work with people
representing themselves?
- Which process steps trip up
self-represented litigants?
- What language do assisters use to
explain legal matters?
- What is people’s mental state at court?

60. Field days at court

61. Going to court without a lawyer

62. “Backstage” at court
Annotated form packets Tools for court clerks

63. How to talk to staff

64. Research methods & skills:
- Website “self-help” inventory
- Courtroom observation
- Shadowing
- Intercept recruiting
- Informal, unscripted interviews
- Taco research
- Synthesis dinners
- Court day summaries

65. Work out
what to ask
the court
Tell
the court
(in writing)
Share
with others
involved
Write up
get signed
& act on
This isn’t as
easy as it
sounds, & help
centers (&
attorneys)
spend quite a
bit of time on
it.
The main step
that forms,
“self help”
assistance,
online doc-
assembly
interviews,
etc, are
intended to
help with.
Service:
unfamiliar
concept for
laypeople.
Basically the
court can’t act
until it knows
that everyone
has the same
information.
There may be
action by the
user in this
step
(appearance,
evidence, etc)
but the point
is the court
does
something.
You, the user,
have to write
up your own
orders - OAH,
judgment, etc
- you’re NOT
DONE when
the judge says
something.
Multiple rounds common here
Court
decision
Agree
with or w/o
help

66. Simplified instructions for serving papers

67. “We’re not just asking
people to be their own
lawyer. We’re asking them
to be a paralegal, legal
admin, process server, and
scheduler.” - Kyanna Williams

68. Institutions act both at
scale and over time.

69. Libraries, schools, hospitals

70. Things are strange in 2019.

71. In the 21st century,
we interact with
long-timescale institutions
via short-timescale
technologies.

72. Public servants make
more design decisions
than the entire design
industry.

73. The best thing we can
do is open our practice
to support everyone
who makes design
decisions.

74. Questions?

75. Thank you so much for
coming!
@cydharrell any time you
want to talk about this