My keynote at Product Camp Hyderabad, Feb 2, 2019. In this talk I discussed the topic of how being data-driven could make product management change from "just in case" to "just in time"

1. Data-Driven Product
Management
Tathagat Varma
Country Manager, NerdWallet

2. Disclaimer
These are my personal views and don’t represent my
employer Nerdwallet in any manner.

3. Our Mission
Provide clarity
for all of life’s
financial
decisions.
Our Vision
A world where
everyone makes
financial
decisions with
confidence.

4. PM’s #1 Job?

5. By…

6. But…

7. So, just do it!

8. Product bloats…

9. UX goes downhill…
https://headrush.typepad.com/creating_passionate_users/2005/06/featuritis_vs_t.html

10. Features remain unused…
https://blog.standishgroup.com/post/62

11. Investment is wasted…

12. Though users are ok :)

13. Almost forgot…

14. Meet “HIPPO”
“Highest Paid Person’s Opinion”

15. HIPPOs have an…
http://davechessgames.blogspot.com/2013/07/the-science-of-good-hypothesis.html

16. They’re always right!

17. And data is so wrong!
Pic: https://medium.com/@UnDelay.io/trust-data-not-opinions-955dc1dbc67b

18. So, who are they?
“I can’t say it any better, HiPPO’s rule
the world, they overrule your data,
they impose their opinions on you and
your company customers, they think
they know best (sometimes they do),
their mere presence in a meeting
prevents ideas from coming up.”
https://www.kaushik.net/avinash/seven-steps-to-creating-a-data-driven-decision-making-culture/

19. And they’re real!
https://twitter.com/ScrumAlliance/status/940976932350435328

20. So, what’s the solution?
“…depersonalize decision making, simply don’t make
it about you or what you think. Go outside, get
context from other places. Include external or
internal benchmarks in your analysis. Get
competitive data (we are at x% of zz metric and our
competition is at x+9% of zz metric).” Be incessantly
focussed on your company customers and dragging
their voice to the table (for example via
experimentation and testing or via open ended
survey questions). Very few people, HiPPO’s
included, can argue with a customer’s voice, the
customer after all is the queen / king!
https://www.kaushik.net/avinash/seven-steps-to-creating-a-data-driven-decision-making-culture/

21. Today’s context…
• Time: Quarter? Month? Week? Day? Hours?…
• Money: Never enough!
• Competition: Faster and closer than you think!
• Customers: Merciless, demanding and no loyalty!
• Product: Always evolving, only features not
enough anymore…

22. Data-driven PM?

23. Overarching “Process”
• Vision: Set clear quantitative goals that represent long-term vision
(~1-3y), or “To Be”. This serves as the true north for the product team.
• Strategy: Identify KPIs that help establish “As Is” baseline and track
the trajectory towards goals using “actionable metrics” (~1-3m). Pivot
when strategy won’t lead to the vision.
• Execution: Cull out metrics that help track and optimize operations
(daily, weekly, hourly?). Perform “GPS” style course-correction to
ensure performance is in range. Might also track “vanity metrics”.
• Innovation: Establish “safe to fail” experiments inside sprints (or
shorter!) to test out new ideas and build data to support rolling-in a
new feature (or deprecating old ones!)

24. Goals (become “x” by “y”)
• Marketshare: E.g. Increase from 23% to 31% in 2019
• Revenue: E.g. Achieve topline of $750B by 2022
• Profitability: E.g. Break-even by 2025
• Operations: E.g. Improve unit economics by 30% by 2020
• Customer: E.g. Become the most-preferred brand by 2023
• Product: E.g. Be #1 source for xyz
• …

25. KPIs (x1, x2,…) & Metrics
(where is x right now?)
• Marketshare: Activation, Installed Base, Penetration, …
• Revenue: Free-to-Paid Conversion, Retention, CTR, CPM, ARR/
MRR, …
• Profitability: CAC, ARPU, LTV, …
• Operations: Acquisition, Onboarding, Traffic Sources, UVs, …
• Customer: DAU/MAU, CSAT, NPS, Referral, TS, PV, Bounce rate, …
• Product: Feature parity, Feature usage, Quality, Product Rating, …
• Marketing: Organic vs Paid Traffic, …

26. Goals, KPIs & Metrics
https://www.simplekpi.com/Blog/KPI-Reports-Explained-A-complete-guide

27. Scientific Method
https://www.glassdoor.co.uk/Photos/NerdWallet-Office-Photos-IMG2826084.htm

28. Recap
• Old-school MRD / PRD is too inefficient, too slow, too costly
• Relying on HIPPO could lead to unpredictable, unactionable and
unrepeatable results.
• Set clear hard number long-term goals, and cull out KPIs and interim
milestones to inform PMs what we are after! Link incremental investment!
• The goal of a release, or even a sprint is to move the needle, and not just
to checkbox on the backlog.
• Requirements emerge as “interesting hypotheses” based on
experimental, feedback, anecdotal and intuitive data but must be
rigorously validated before roll-out.
• Execute using a closed-loop system modeled around the scientific
method. Monitor product metrics, track customer KPIs and accomplish
business goals!

29. References
• https://medium.com/@UnDelay.io/trust-data-not-opinions-955dc1dbc67b
• https://www.forbes.com/sites/derosetichy/2013/04/15/what-happens-when-a-hippo-runs-your-company/
#4fb355d940cf
• https://towardsdatascience.com/data-driven-product-management-by-taner-akcok-ffd421cfa8d3
• https://youtu.be/VkMcubu4yMc
• https://roadmunk.com/guides/make-your-product-prioritization-more-data-driven/
• https://useriq.com/data-driven-product-management-roadmapping/
• https://www.emperitas.com/becoming-data-driven-product-manager/
• https://channels.theinnovationenterprise.com/presentations/data-driven-product-development-at-netflix
• https://productcoalition.com/data-driven-product-management-choosing-the-right-metrics-for-your-
product-5b85352b3500
• http://people.groupon.com/2018/data-driven-product-management-at-groupon/
• https://snowplowanalytics.com/blog/2018/02/02/data-driven-product-development-is-more-about-process-culture-
and-people-than-technology/
• https://www.thisisproductmanagement.com/episodes/becoming-data-driven/
• https://www.thisisproductmanagement.com/episodes/data-driven-decision-making/
• https://www.productplan.com/data-driven-product-roadmaps-metrics/
• https://towardsdatascience.com/becoming-a-data-driven-product-manager-f320d0182a9c
• https://towardsdatascience.com/data-driven-product-management-by-taner-akcok-ffd421cfa8d3
• https://www.mindtheproduct.com/category/data-driven-product-management/
• https://alphahq.com/data-driven-product-management/