How do you change something as endemic in Australian culture as excessive alcohol consumption? This isn’t a typical consulting job. It requires a design partnership.
Hello Sunday Morning approached Tobias & Tobias to help them better understand their users and find more effective ways of helping them change the relationship they have with alcohol.
In this presentation, we’ll walk through how we’ve been working together to do research, analysis and design:
Forming a design partnership
Immersing everyone in research
Using Behaviour Design to create design hypotheses
Working with multi-disciplinary teams to define and carry out experiments
Evolving the platform
12. Choosing a research cohort
UQ’s recommendations for Future Research included:
‘Conducting blog analysis in relation to consumption data.’
12
World Health Organisation’s ‘Alcohol Use DIsorders Test’ (AUDIT)
Mixpanel’s usage data for level of engagement
13. Recruitment
13
To ensure a high response rate,
we made the first contact a
personal invitation from Jamie.
14. “Have any of you received a letter
from Jamie, followed by some
random guy? Is this legit?”
14
“I got the same thing. Someone
asked to come to my home.
Going to ignore it.”
“WARNING:
Someone’s trying to
scam people on here.”
15. Open interviews
• Context / relationships
• Trigger to use HSM
• Stories of alcohol
• Rationale for drinking
• Journey over time
• Experience with HSM
15
16. Sensitive topics require 100% attention
Protected identities by encoding all files with:
Interview sequence, AUDIT Zone, Engagement level e.g. P06Z3HE
Meetings took place in homes or public, safe spaces
16
30. Dr. Nicholas Carah
Communications & Social Media
University of Queensland
Dr. Carla Meurk
Anthropology
University of Queensland
Dr. Leanne Hides
Clinical Psychology & Substance Abuse
QUT
http://www.pdt.com.au/wp-content/uploads/biomedical-innovation_01.jpg
43. Implicitly segmenting people
Dependent, highly engaged people were
over-eager and confronting to others.
People naturally gravitate to those that they
identify with in age, sex, race, beliefs,
attitudes and behaviour.
43
During on-boarding, suggest ‘people like me’ to follow which we
can qualify from answers to the behavioural criteria and demographic
information.
55. The more you can do these things sober,
the more it pushes the value of alcohol
down and the value of yourself up.
Chris Raine, Founder & CEO
Hello Sunday Morning
55
Notes de l'éditeur
“Hello” Lionel Richie personal beer tap advertisement
<Jamie>
My journey to HSM and why we created a platform that’s non-judgemental, but aims to affect the relationship we have with alcohol.
<Jamie>
Hello Sunday Morning from Queensland, Australia
<Jamie>
This is the simple question we want to explore
<Ash>
HSM is a targeted social network: Facebook for people who want to take a break from alcohol
<Ash>
Of course, alcohol culture is an interesting research topic.
HSM had previously partnered with and had research conducted by UQ, QUT, The University of Newcastle, and The Foundation for Alcohol Research & Education.
<Ash>
The most recent study was a follow-up of Text Analysis Research
‘Measuring blog engagement and social impact’
This was a thematic analysis done with Leximancer: concept engine by Dr. Andrew Smith
Done in 2012 on data from 2011, then followed up with more research and analysis in 2013
There were a number of different studies, but with the same results: a simple, 2-step thematic change
<Ash>
When signing up for HSM, members fill out the WHO AUDIT (Alcohol Use DIsorders Test)
From the AUDIT and website analytics, we created a matrix for a broad spread of qualitative research.
<Ash>
HSM members are an invested community, so we decided not to bias responses with extrinsic incentives. People were intrinsically motivated to help. This also saved on research costs.
Because we made it personal, we had to take a manual recruitment approach.
The call for participants was carefully targeted to remove the ‘Bystander effect.’ And it generated a great response.
Can any of you see the issue in this call for participants when I was going to go out and do the research?
<Ash>
I got my ass handed to me by the HSM community. Hundreds of posts suddenly sprang up warning about this guy who was trying to scam members.
Thankfully, Jamie and Loic immediately jumped in and settled everyone down.
We also underestimated the overhead personal, manual recruitment would bring with it: answering long trails of emails, scheduling time and places, and acknowledging offers to participate that came in even after we’d completed the research.
<Ash>
The interviews were extremely open and started with “Tell me about your relationship with alcohol.” From there, we might chat for 30 minutes or 3 hours. It was entirely up to the participant.
<Ash>
Because this was a sensitive topic, I had to protect identities, give 100% attention, and ensure that I didn’t miss a thing
We bought and tested a range of equipment before settling on:
- A RODE iXY bi-directional mic for iPhone (with pop cover for open / windy locations)
- Gorilla Pod for iPhone (using Hindenburg Field Recorder. RODE app was too buggy).
iPad as backup (using Voice Record Pro).
Cleaned up audio using SoundSoap
We ended up with 16 audio files and 3 email transcripts
<Ash>
I really liked Greg’s metaphor of designers being a virus. We frame ourselves as design partners with all of our clients. We want to help them practice design everyday. That really comes to life when they’re an integral part of the research and analysis.
<Jamie>
My experience with transcription and combing. We took on all transcripting to be part of the research and save on costs.
We didn’t expect it would take so long and be so hard, but we learned things we could never have guessed.
With the combing, we started wondering what it was for or if it would yield anything, but it quickly became apparent that similar behaviours, tasks, and principles were arising.
<Ash>
The transcription and combing was well-timed. As it was occurring, I was able to take my son and wife to Canada for TED and a bit of snowboarding.
<Ash>
And as soon as I got back, all these chunks of information from combing were on Post-its, ready for the next phase…
(Ash and Bel reviewing combed information)
<Ash>
Affinity diagramming.
Because we knew we’d cover the studio with the thousands of Post-it notes we had, we got a number of 2.5 x 1.5m KapaPLEX boards. This also allowed us to shuffle boards with themes and create meta-themes.
Pictured is Loic Golliard, HSM’s Product Manager and my compadre in all of this.
He got well and truly soaked in the research and analysis.
<Ash>
In the end, it took almost 5 sorts to group everything satisfactorily: from loose groups with poor labels, to tighter groups with sub-groups, themes, and meta-themes. Then refactoring everything until we had a well-labelled, ordered taxonomy.
We’d taken a long journey from data to information to knowledge…
<ash>
Then we had to walk the wall to find the wisdom
We had to identify key behaviours and relate common behaviours among and between participants.
<ash>
We looked for participants with multiple related behaviours and were able to classify them by goals.
Filling the void & Single and looking were immediate. Then came Altering state & Part of the crew.
Of course, Altering state disappeared on closer scrutiny, since most people use alcohol to alter their state.
<Ash>
So we used MindManager to run another pass. A concept of maturity appeared and the attitude of ‘Too old for this’ was latched upon.
<Ash>
From that Bel, Simon and I were able to pull out key attributes and stories of participants to draft some narratives.
<Ash>
The narratives became archetype skeletons, up for review and debate. Everyone was so well versed with the research that the team had cogent arguments about many of the aspects we’d attempted to put together.
<Ash>
Eventually, we created the first draft of archetypes. The document is meant to be living and is ordered left-to-right by past (how I got here), present (why I want to change) and future (how HSM can identify and help these people). Four archetypes were created:
1. Reggie - Self-medicating with alcohol to cope with the stresses of work. As he’s maturing his body can’t cope.
2. Marcus - Uses alcohol for Dutch courage and has experienced alcohol fuelled violence.
3. Jen - She likes to be the life of the party and drinks to ‘fit-in.’ She’s a young Mother and is worried about the influence she’s having on her kids.
4. Sarah - She had a traumatic past and now has too much time on her hands. She drinks to forget and has become physically dependent.
<Ash>
Now that we had a first draft, Loic suggested we get together with some of the researchers to discuss interventions and approaches to refine.
<ash>
Bel, Loic and I flew up to the Institute of Biomedical Innovation in Queensland.
I stood up and delivered a presentation outlining our approach. They had no idea who we were or why we were in the meeting. *gulp*
They talked pure quantitative studies. We talked qualitative research guiding and triangulated with quantitative data. They talked 2 year Randomised Controlled Trials (RCTs). We talked 2 week RCTs.
They talked publications, programmes and grants. We talked engagement and impact.
Another rookie mistake. I should have qualified the agenda and purpose, and sent our approach and expectations ahead before flying up. We all walked away confused.
Thankfully, after further discussions, we came up with some options that allow for both pure quant research to attract grants, and rapid experiments to deliver results.
<Ash>
When we were investigating the research cohort, we discovered 53% were dependent! This has subsequently been published in the International Journal of Drug Research http://www.sciencedirect.com/science/article/pii/S0955395914002126
We figured that we needed to know more about dependent people…
<Ash>
Hooray for MOOCs
The Yerkes Research Center of Emory University had a Neuropharmacology course called, ‘The Addicted Brain.’
<Ash>
We got to listen to Dr Michael J Kuhar at high speed as Loic, Bel and myself completed this 7 week course.
<Ash>
We learned about neurodegeneration and the amount of time it takes to recover. How being dependent physically changes the person.
Alcohol abuse impacts the creation and metabolism of a number of neurotransmitters, especially dopamine and serotonin, and physically alters the structure of the brain.
<Ash>
Because humans are variable and subject to changes in impulse and mood, I always advocate for changing the environment to change their behaviour. In this case, the decks are stacked against us because the Person has been changed. The drive or impulse for alcohol is physically present, so it takes a much larger change in Environment to affect the behaviour.
One approach is to start tackling what’s known as ‘comorbidity’ (coexisting health and mental problems) to try and change the Person, as well as changing the Environment - and that’s what we’re doing.
<Ash>
So what’s the point of all this? We’re creating a technical and behavioural ecosystem for experimentation. (The platform was 4 years old and the coding was becoming spaghetti, so the team at Terracoding have been re-writing it.)
We needed to be able to choose cohorts with specific criteria and randomly assign them to an experimental or control group, then track the results of point in time actions, for a span of time, or over a lifetime.
<Ash>
Now that the back-end is being re-architected, the front-end is also receiving some love. There are clear, simple calls to action.
<Ash>
When you sign up, there’s now a proper on-boarding funnel that first prompts you to take a break for either 3 or 12 months. That’s what we call ‘doing an HSM’. Before this, members thought they were doing one just by signing up, or couldn’t find the button to begin. Just putting these options clearly up front has made a significant impact.
<Ash>
In the interest of carrying out experiments, some go through the on-boarding funnel and some don’t. The line that goes up is people who went through on boarding and started an HSM. The line that goes down is people who didn’t go through on boarding and started an HSM.
<Ash>
To identify the behavioural archetypes that apply to people, we have to get them to answer some behavioural questions. We worked hard on the phrasing of the questions and thought carefully about the perceived speed and ease of answering them. As you answer a question, the next swiftly lights up, whisking you along the process.
<Ash>
…and we now display people’s AUDIT zone back to people. Hello Sunday Morning weren’t doing this before as they were worried it might shock people, but clinical research has demonstrated that the simple matter of knowing your AUDIT score has a significant impact on the amount you subsequently drink.
<Ash>
We’ll also soon be targeting goals based on the answers to behavioural criteria questions, and suggesting ‘people like you’ to follow. This last part is critical.
<Ash>
This is because another clinically effective intervention is getting support from people you can relate to. With drinking, this is primarily related to people with shared drinking behaviours and consequences, but there are other cultural and demographic facets people also relate to.
It’s also because highly engaged, dependent members of the HSM community, or people who kept relapsing or sharing dire stories were enthusiastically engaging with and subsequently scaring lighter drinkers.
<Ash>
These are the types of areas our first experiments will be targeting:
comorbidity, aspirational goals and altering environment.
<Ash>
Whilst stated quite plainly, the experiments have to be given detail and context for the developers to set up and measure the outcomes.
<Ash>
We were about to start analysing the data collected after a month of our behavioural criteria surveys being completed when we discovered that Jamie is a stats & SPSS guru…
<Jamie>
How I used SPSS to check correlations, do a factor analysis and cross-cut the data.
<Jamie>
To our surprise, the behavioural correlations T&T manually discovered in the qualitative research quite accurately bore out in the data.
<Jamie>
So we’re still refining and updating the archetypes
<Jamie>
…and still teasing out how best to frame the questions.
<Ash>
All this work has been to understand the current HSM members and create a technical environment that supports the types of experiments we want to perform to affect people’s behaviour with regards to social drinking and addiction.
Although we’ve had a small setback with preparing the environment, we’re almost ready to deploy more targeted experiments.
<Ash>
Many of our experiments, especially to address comorbidity issues, will be made of many sequential components.
We have to break down large goal behaviours into smaller, more manageable steps and discover the ways in which we can help members progress
<Ash>
Of course, each behaviour is a combination of motivation, ability and trigger at the same time.
If we can identify what element is lacking in behaviours that we want to promote, we can address it.
If we want to curb a behaviour, we can try to sabotage one of those elements.
<Ash & Jamie>
Over the next 12 months we’ll be working together to continually test and refine how the service affects people. We aim to drive engagement since feedback on drinking habits and community support, are clinically proven interventions in moderating alcohol consumption. We’ll also be addressing comorbidity, as these are the psychological drivers for drinking behaviours, and nudging members to change their context or environment since people can’t drink if alcohol is not available.