Alastair Simpson discusses using data-informed design to improve products' time to value (TTV). He explains how Atlassian analyzed usage data showing new users spent little time with their products. Combining this data with qualitative customer insights, they identified engagement and conversion as key metrics. Experiments informed by both data and empathy increased conversion by 22%, helping new users realize value faster. Data and testing help optimize, but clear decision-making remains important for product success.

1. ALASTAIR SIMPSON • DESIGN MANAGER • ATLASSIAN • @ALANSTAIRS
Building fast growth into your products
Data driven informed design

2. Time to value
- TTV
- Time to realise value for customer
- Tricky to understand.
- Fast growth = fully understand TTV equation.

3. The Parsonage B&B
San Francisco
- Trip to SF to speak at conference
- Stayed at the Parsonage B&B

4. - Flew from Sydney, Zoe, Frankie
- 1st trip for 6 month old
- Daunted, 15 hour flight
- Stay; safe, quiet, away from other guests

5. - For when my daughter was jet lagged and crying in the middle of the night

6. - Love to host us
- So not disturb other guests, own apartment, private floor, separate room, same cost
- Taxi with car seat
- Personal, honest and tailored to my needs.
- TTV was instant.

7. Time to value
- This example = quick to wow
- Worked hard at this.
- How do you replicate in digital products, at scale?
- How long customer see value?
- Products not built to optimise for TTV
- Built to solve initial problem
- Grow, more features added
- More complexity
- Great for power users
- Terrible for a new customer, lost and confused

8. 50% of new users spend less than 30 minutes
trying our products before leaving.
- Analysed data
- 50% new customers less than 30 minutes with our products.

9. 30 minutes
- Products solve complex problems
- 30 minutes isn’t long enough realise value
- onboarding flow was clunky
- No quick to wow moment
- Very steep learning curve in order to realise value
- Too many just not sticking around
- 10 minutes from now you will walk out of here with a blueprint for how to build fast growth into your products using data informed design

10. ✓ Set up core teams
✓ Clear success metrics
✓ Centralised patterns
Won’t dig into these things; but we:
- Changed org structure. Setup TTV teams, 1 owner, 1 person accountable
- Clear success metrics
- Centralised design patterns that worked and didn’t into our design guidelines, meant we could scale across products.

11. Data driven informed design
- Main thing I want to talk about is data informed, not driven, design.
- Subtle difference.
- Let me illustrate a data driven approach to you.

12. Stand up please
- Everyone stand up please
- Donna kindly gave me some data that she had on attendees.

13. 50% are “UX Designers”
Stay standing
- According to that data 50% of you are UX Designers

14. 55% are “male”
Stay standing
- 55% are male

15. 10% are from Brisbane
Stay standing
- 10% are from Brisbane
- Based on this data driven approach I could make an assumption that you look like this.

16. - Oh…so looking around I am wrong.
- This is a data driven approach…
- Lets imagine I asked a few questions from those left standing
- Where do you shop?
- Do you buy hats very often?
- Do you wear glasses?
- and I build up a small bit of empathy for my customer and better understand what you they look like.

17. - And I now can see that they look like this…
- Please take a seat. Thank you for indulging me.

18. Data Empathy
Gut feelingGreat product design
Data informed design
- Amazing product design
- Understand data what customers are doing
- Empathy for customers problems facing
- Some intuition
- Making decisions is what we are paid to do.
- Lets look at data.

19. AAA Data
- Atlassian we talk about AAA data - what does that mean?
- We love acronyms

20. Accurate
- Data must be accurate.
- Basing product decisions on this, especially at enterprise level. Data must be accurate.
- Ask plenty of questions as sometimes the data is not always what it seems.

21. Accessible
- Data must be accessible for all. It wasn’t in our case.
- To access our data you had to know SQL and have access to the right database to get the data.
- Designers couldn’t get access to it.

22. Actionable
- Data without actionable insights is useless.
- It must be actionable and contain insights.
- So once we had our AAA data, we need to focus on 1 data point, or metric that gave us the best possible proxy for
customer value in our products.

23. Engagement (Time spent)
Conversion
- So for us, our TTV data point was engagement (Time spent) and conversion.
- Having analysed our data these were the best possible proxies for customer value.
- Conversion was also a clear indication of value, as customers had to put CC in at day 7.
- So these were the data points we optimised for.

24. A/B Test
Noun | A/B Test |ˈā-ˈbē ˈtest
An experiment in which a single element of a web
page is altered to create one or more variations of
a page.
- A/B Test
- 2 variations same page to 50/50 split
- Can work out better
- Push directly into production or iterate.

25. Workshop Design + Development Running Analysis
Experiment lifecycle (3 weeks)
- Lean UX methodology
- Workshops, design and dev, run and analyse.
- 3 weeks to get results
- Quick turnaround
- Couple of months
- Tested 6 to 10 different ideas in few months.

26. FIRST ITERATION
-12% engagement
-10% conversion
- Time passed
- Experiments only on data and gut feeling at this point
- No customer empathy
- One experiment high hopes totally bombed. -12% conversion.
- Driven down conversion by in theory improving onboarding

27. - Self sealing logic
- Stuck in our own belief bubble
- Get out and experience customers

28. Data Empathy
Gut feeling
Data informed design
- So empathy, or qualitative feedback from customers
- Get out of office
- Speak to customers
- Anyone can do this
- You don’t have to have a dedicated researcher

29. I’m trying to understand how something
could have that amount of power and be
that difficult to pick up.
(ANOTHER) POTENTIAL CUSTOMER
”
“
- Build up customer empathy within your organisation for problems facing your customers
- Amazing quotes from customers
- Ran Usability testing.
- Customer interviews.

30. - Diary studies. first 10 days tumblr. They were all mini-stories. Organisation and us could not deny them.
- Great insights
- Paired with data, understood what and why
- Brought the pain home that customers experienced when first using our tools.

31. Put these customer journeys up on walls.
Days.
Yellow = customer actions
- blue = questions
- pink = pain
These diaries really helped us build up an understanding of how people used and saw value in our products. It gave us
the why for what we were seeing in our data. It also brought the pain home that customers experienced when first using
our tools.

32. 2. Help me set up and complete critical tasks
3. Show me practical value in everyday work
4. Demonstrate immediate value to subsequent users
1. Identify my problem & potential solution
Time to value conceptual model
- Created TTV model
- Why is a model important?
- No framework = hacking blindly
- Won’t result in fast growth, unless you get very lucky.
- Model = backbone of all future experimentation efforts.

33. Workshop Design + Development Running Analysis
Experiment lifecycle (3 weeks)
Qualitative insights →
- Now we were still running fast, getting results in a matter of weeks.
- But now we had qualitative insights instead of assumptions and just numbers
- We were informed….

34. ~ engagement
+22% conversion
SECOND ITERATION
- Same experiment = -11% result
- Are you ready?
- Made some tweaks based on our increased customer understanding.
- +22% conversion.
- Not easy to add 22% to your bottom line overnight, but we did.
- Released this directly to production within a few weeks
-

35. Data Empathy
Gut feeling
Data informed design
- Lastly gut feeling.
- Some designers worry that data takes way their creative freedom. I disagree I think it enhances it.
- First experiment right track even though the result was negative
- Didn't throw it away.
- The qual feedback gave us insights into why and we iterated and made it a success.

36. Data and A/B tests are valuable allies, and
they help us understand and grow and
optimize, but they’re not a replacement for
clear-headed, strong decision-making.
JULIE ZHUO (FACEBOOK DESIGN DIRECTOR)
”
“
- Julie Zhuo is Facebooks product design director.
- Data / research great but
- No substitute for clear headed thinking and decision making.

37. Stand up if this experiment
was a success

38. - Added a product tour, the type you see on every web app

39. -8% engagement
~ conversion

40. Stand up if this experiment
was a success

41. - Same page. With intro mode on removed ~60% features

42. +22 engagement
+17% conversion

43. Thanks!
design.atlassian.com
ALASTAIR SIMPSON • DESIGN MANAGER • ATLASSIAN • @ALANSTAIRS