Main takeaways:
-PMs don't need a lot of data points to prioritize the features for the upcoming sprint. They just need to identify the relevant one's.
-PMs should be skilled to strike the balance between agility in making decisions and accuracy of perceived outcomes
-PMs should be able to prioritize the feature requests with minimum data points available and optimum techniques
7. Sources of Information for a Product Manager
Customer/User
And The Complexity Starts Now...
8. Key Areas where PM takes decisions
1. Product Enhancement OR Product Strengthening
2. Product Vision - Alignment with Organisation Vision
3. Setting Up Product Roadmap
4. Features Development
-What to build next
-Why to build
-Where would be the impact
-Who will be impacted
-When to launch
5. Optimizing the team performance
6. When and why the product should be expanded, killed or revamped
9. Key Areas where PM has to take decisions
1. Product Enhancement OR Product Strengthening
2. Product Vision - Alignment with Organisation Vision
3. Setting Up Product Roadmap
4. Features Development
-What to build next
-Why to build
-Where would be the impact
-Who will be impacted
-When to launch
5. Optimising the team performance
6. When and why the product should be expanded, killed or
revamped
Decision Making would be a lot easier
and accurate if you have :
1. Product KPIs to monitor
2. Quantitative Product Strategic
Roadmap
3. Capability to predict the
product performance
4. the sense of the direction of
the product
5. Capability to read the signals !
10. But Data is Not Everything… Otherwise PM job would have been Automated !
Cool Things Data Can do for a PM
1. Define precisely what is happening
2. Can show the aberrations
3. Can help you in justifying your
decisions
4. Help you in monitoring real time
activities
5. Help you identifying how far you have
come and how far you still have to go
Data Can Mislead you
1. With confirmation Bias
2. By not telling the complete picture
3. Boosting confidence to over confidence
4. By making you deaf of others’ opinions
BUT
Objective Attributes (Data, KPIs) + Subjective Attributes (Emotions, Experience)
=> Rational Decision + Margin of Error
11. Setting Up Basic Data Setup - Required to be a cool PM :)
1. Data collection
a. Mapping optimized user flow
b. Implementing tracking system
c. Listening to user footsteps
d. Listening to user experience
e. Users want you to give what they want but they don’t want to tell you what they want
f. Google Analytics | Google Webmaster | Custom Log files
2. Data organization
a. Reporting System
b. User behavior mapping
c. In product factors- Clicks, Views, Conversion, Referral
d. External factors - SERP, #Users, #Customers, Revenue, Renewal, Online Trends
3. Data analysis
a. WoW Average, MoM Average, Cohort Analysis, A/B Testing, Correlation Analysis,
Frequency Distribution, Benchmarking
4. Feedback and looping the learnings back to the data collection
a. Include in your Sprint Planning
5. Pitfalls
a. Don’t believe on data blindly. Double check with other sources
12. Now you have millions of data points at your service. Next what...
Objective Key Metrics Benchmark Target Tolerance Timeline
Our product should
be 5x usable
compared to the
last year
Annual active
users
10 Million 50 Million +/- 10% 12 months
Annual retention
rate
30% 50% +/- 5 % 12 months
% of users
completed the
journey
3 Million 20 Million +/- 5 % 12 months
Annual product
revenue
$ 10 Million $ 50 Million +/- 20% 12 months
13. Let’s Solve a Practical Problem
(Objective is to think like a data driven product manager)
You have been recently hired as Product Manager for a SaaS based product focussing on B2B domain.
Product: It is a learning app for employee workforce which has self paced courses and evaluation module embedded in the system. New
programs can be added on case basis with customization charges
Team: You have 2 developers and 6 months of time for strategy re-evaluation
Company Vision: To be the default solution for any kind of workforce training and up-skilling
Company Wide Objectives:
1. To increase the client base from current 20 to 30 in 1 year
2. To renew at least 80% of the clients contract for another year
3. To reduce the issues raised by the clients to 50% to reduce the cost
Now your Business Head asked you to come up with next 6-month product roadmap which they have to
include in his business roadmap
14. And Next Thing you know, the outgoing PM gave you the following list and wish you all the best
S. No. Feature
1 An internal tool to address and assign the issues quickly and reduce the TAT of support team
2 Code upgradation to eliminate 30% of the bugs and support new features
3 A feature which is requested by 5 out of 50 clients which your sales team approached
4
A new product which is aligned with company vision and can expand business to European market where company has not
penetrated
5 UI/UX improvements based on data analytics
6 A feature which your biggest competitor has launched recently
7
An AI based course recommendation system which would suggest the employee to pick up the next course based on his
previously acquired skills
8 An admin Analytics dashboard to track the performance of each employee
9 A dashboard for internal Sales and support team to track the usage and renewal metrics
10 A feature which your CEO has thought today and believes that it has to be launched in next week and could be game changer
11
You have a vendor who provide the assessment quiz content via API and they have upgraded their system so, you also need
to upgrade your Backend system
12 A widget which is non functional in your system but 10% of the users use it in their course duration
15. RICE Framework
Reach
Reach is measured in a number of people/events per time period. This factor is aimed to estimate how many people each
feature or project will affect within a specific time period and how many customers will see the changes.
Pay attention to real measurements from product metrics instead of using unclear numbers.
As the example:
● This feature will be used by 800 users per month.
● 1000 start the onboarding and with 70% completion rate only 700 users will see this feature (which is shown right
after completion).
(Source: https://hygger.io/blog/4-powerful-factors-rice-scoring-model/)
16. RICE Framework
Impact:
Impact reflects what contribution does the feature give to the product.
Value is understood in different ways in each product.
For example, in SaaS, the excellent indicator for day 1 retention is 15%. It means that 85% of people simply leave on the second day.
Therefore, here you should think about features that many new users will see as close to the time of registration.
● Help to keep current users
Customers have bought a subscription and now ask to do some feature. We do not “rush” blindly do everything in a row. We accumulate
statistics for each feature – how many customers asked for it. And then we do the most popular features.
● Add values to the product and separate us from competitors
Pro Tip: Focus on scalability of the feature. Product development is one time effort and can give you business for a long time.
Choosing an impact number may seem unscientific. But remember the alternative: a tangled mess of gut feeling.
As the example:
● Project A: For all customers that see it, it will have a huge impact. The impact score is 5.
● Project B: It will have a lesser impact on every customer. So, the impact score is 1.
● Project C: It is somewhere in-between in terms of impact. The score is 2.
(Source: https://hygger.io/blog/4-powerful-factors-rice-scoring-model/)
17. RICE Framework
Confidence
If you think a project could have a huge impact but don’t have data to back it up, confidence lets you control that.
Confidence can be measured with a percentage scale.
As the example
● Project A: PM has quantitative metrics to reach, user research for impact, and an estimate for efforts. So the project
gets a 100% confidence score.
● Project B: PM operates with data to support the reach and efforts, but he/she is unsure about the impact. The project
gets a 80% confidence score.
● Project C: Reach and impact data may be lower than estimated. The effort may be higher. The project gets a 50%
confidence score.
(Source: https://hygger.io/blog/4-powerful-factors-rice-scoring-model/)
18. RICE Framework
Effort
Estimate the total amount of time a feature will require from all team members to move quickly and have an impact with the
least amount of effort.
Effort is estimated as a number of “person-months”, weeks or hours, depending on needs. It is the work that one team
member can do in a specific month. Unlike the other positive factors, more effort is a bad thing, so it divides the total impact.
As the example:
● Project A will take about a week of planning, 2 weeks of design and 3 weeks for engineering, so the effort score is 2
person-months.
● Project B will require several weeks of planning, about a month for design and 2 months for engineering. The score
will be 4 person-months.
● Project C requires just a week of planning, no need for design, and 1-2 weeks for engineering. The score will be 1
person-month.
(Source: https://hygger.io/blog/4-powerful-factors-rice-scoring-model/)
19. % Normalized Score
0-10 0.5
10-20 1
20-30 1.5
30-40 2
40-50 2.5
50-60 3
60-70 3.5
70-80 4
80-90 4.5
90-100 5
Reach Percentage of Customers Normalized to the range of 1-5
Impact Value touch point to each customer in % Normalized to the range of 1-5
Confidence Probability of success based on data analysis Normalized to the range of 1-5
Effort Person-Months required Normalized to the range of 1-5.
20. Applying RICE Framework (R,I and C in % and then normalizing to the range of 1 to 5)
No. Feature Reach Impact Confidence Effort R*I*C/E Priority
1 An internal tool to address and assign the issues quickly and reduce the TAT
of support team
100% ->5 50%->2.5 100%-5 1 62.5 3
2
Code upgradation to eliminate 30% of the bugs and support new features
100%->5 60%->3 100%-5 1 75 2
3
A feature which is requested by 5 out of 50 potential clients
10%->0.5 60%->3 60%-3 1 4.5 11
4 A new product which is aligned with company vision and can expand business
to European market where company has not penetrated
40%->2 80%->4 20%-1 3 2.66 12
5
UI/UX improvements based on data analytics
100%->5 40%->2 80%-4 1 40 5
6
A feature which your biggest competitor has launched recently
60%->3 40%->2 80%-4 2 12 9
7 An AI based course recommendation system which would suggest the
employee to pick up the next course based on his previously acquired skills
100%->5 80%->4 60%-3 3 20 8
8
An admin Analytics dashboard to track the performance of each employee
80%->4 80%->4 40%-2 1 32 6
9 A dashboard for internal Sales and support team to track the usage and
renewal metrics
80%->5 60%->3 80%-4 1 60 4
10 A feature which your CEO has thought today and believes that it has to be
launched in next week and could be game changer
100%->5 60%->3 20%-1 2 7.5 10
11 You have a vendor who provide the quiz content via API and they have
upgraded their system so, you also need to upgrade your Backend system
100%->5 100%->5 80%-4 1 100 1
12 A widget which is non functional in your system but 10% of the users use it in
their course duration
20%->5 40%->2 60%-3 1 30 7
21. Thank You
(For more discussion/comment/feedback etc, I am reachable at https://www.linkedin.com/in/chinmayagupta/ or
chinmayagupta.02@gmail.com