This document discusses predictive analytics as a product and some of the challenges involved. It notes that predictive analytics has become more complex due to demands like monetization opportunities, integration of multiple data sources, and the need for solutions to work across initiatives. Modular, shareable, and monetizable approaches are needed, such as standards like PMML and PFA that allow models to be deployed and scored in different systems. The scaling demands also require platforms that can build solutions once and use them everywhere via application programming interfaces (APIs).
2. Intended for Knowledge Sharing only
Disclaimer:
Participation in this summit is purely on personal basis and is not meant to represent VISA’s position on
this or any other subject and in any form or matter. The talk is based on learning from work across
industries and firms. Care has been taken to ensure no proprietary or work related information of any
firm is used in any material.
3. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
Data Scientist, eh…
3
4. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
FEELS LIKE A ROCKSTAR, DOESN’T IT?
4
http://modernservantleader.com/servant-leadership/narcissism-kills-morale/
5. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
..BUT A KANYE & NOT COLDPLAY
5
https://imgflip.com/memegenerator/7064654/Kanye-West
6. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
So what happened?
6
7. Intended for Knowledge Sharing only
Intended for Knowledge Sharing only
SOME CHALLENGES
7
Unrealistic expectations on RoI.
Operates in siloes, not complemented by user research/other internal or
external data/experimentation results.
Field testing & iterative development still predominantly offline.
Deployment, Post Deployment management & monitoring expensive. Not
easy to turn on/off, tweak, flip, scale.
Predictions driven significantly by historical trends and relationships.
Expectations modeled as simulations.
8. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
Explain it a bit more...
8
9. COMPLICATION 1: PREDICTIVE ANALYTICS IS INTRICATE & COMPLEX
Intended for Knowledge Sharing only 9
Objective
Translation to Analytical
Framework
Data Collection and
Preparation
Analysis, Validation &
Verification
Actionable insights and
impact sizing
A/B Testing
Rollouts
• Understand need, fit with Strategic needs, actionability, stakeholders buy-in, engineering
RoI, project management
• Decide on the Analytical methodology based on nature of the problem, dependent
variable, frequency, sample, time, required precision, actionability
• Hypothesized driver list
• Data Collection: Internal & external sourcing
• Data Preparation: Blending, aggregations
• Data Transformations: Outlier, Missing, math transformation, interactions, redundancy
treatments, variable selections
• Sampling methodology & split
• Model development and validation: In-time, Out-of-time
• Stand alone, ensemble
• Performance diagnostics & cross check with other sources
• Recommendations, impact sizing, cross leverage scores
• Field Testing (Champion vs. Challenger)
• Iteration plan based on user feedback (VOC), performance
• Model deployment, post deployment monitoring & management
• Integration with Product Line– New product,
1
2
3
4
5
6
7
Description
10. COMPLICATION 2: MULTIPLE AUDIENCE, PRIORITIES, DEPENDENCIES
Intended for Knowledge Sharing only 10
Objective
Translation to Analytical
Framework
Data Collection and
Preparation
Analysis, Validation &
Verification
Actionable insights and
impact sizing
A/B Testing
Rollouts
• Analyst & Stakeholder
• Analyst, Data Instrumentation, Data
Manager, Stakeholder
• Analyst, Data Instrumentation, Data
Manager
• Analyst
• Analyst, Stakeholder, Cross Functional
team, Leadership
• Analyst, Experimentation Team, User
Researcher, Developer, Stakeholder
• Analyst, Developer, Stakeholder,
Leadership
1
2
3
4
5
6
7
Who does it?
• Agile and may undergo iteration
• Changes in Strategic goals, newer
initiatives, releases, discoveries, reorgs
• Sourcing/Blending challenges: Data
handovers between systems, blending
challenges
• Scalability/automation
• Data movements/latencies/
teams/approvals
• Evolution of hypotheses, data
changes/errors, success criteria
• Competing priorities, data movements,
Scenario Simulations
• Success criteria, integration with
research/testing tools, iterations
• Integration with host systems,
engineering investment, model
tweaking, monitoring, customization
Key Challenges
11. COMPLICATION 3: OUTPUT OF ONE CAN BE INPUT/ADDITION TO ANOTHER
Intended for Knowledge Sharing only 11
Behavioral
Merchant
Performance
Clickstream/
Ops
Campaign
Performance
VOC/Social/
CRM
• Probability of Engagement/LTV
Growth/Churn/Loyalty
• Life event changes
• Product/Price Migrations
• Probability of Growth/Churn
• Next Best Product/Offer
• Network partners
• Conversion Rate Optimization
• Server Response Times
• Time to Purchase
• Campaign Responses
• Next Best Product/Offer
• Cross Channel target
• Promoter/Detractor & drivers
• Brand Appeal
• Theme/entity of engagement
Data Lake:
Enriched with
predictions
e.g., Uber’s cross
sell platform,
Google Calendar,
VDP
12. COMPLICATION 4: REAL DECISION MAKING NEEDS ADDITIONAL REASONING BEYOND
ANALYTICS
Intended for Knowledge Sharing only 12
Analytics provides insights into “actions”, Research context on “motivations” & Testing
helps verify the “tactics” in the field and everything has to be productized…
Strategy
Data
Tagging
Data
Platform
Reporting
Analytics
Research
Data
Products
Iterative
Loop Why such complexity?
Focus on Big Wins
Reduced Wastage
Quick Fixes
Adaptability
Assured execution
Learning for future
initiatives
Optimization
13. Intended for Knowledge Sharing only
Intended for Knowledge Sharing only
COMPLICATION 5: DEMANDS ON PREDICTIVE ANALYTICS HAVE INCREASED
13
Predictive
Analytics
Behavioral
Analytics
What are the
customers doing?
Voice of
Customer
What are the
customers
telling you?
Platform
Performance
How are you
delivering? Competitive
Are the
customers
buying
elsewhere?
Social Listening
How are
customers
discussing you?
…aaanddd Better, Faster, Cheaper, Monetizable
14. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
So, what do we need then?
14
15. Intended for Knowledge Sharing only
• Extensible
• Scalable
• Flexible
• Easy to integrate with
other techniques
Intended for Knowledge Sharing only
HIGH LEVEL SUMMARY OF NEEDS: MODULAR, SHAREABLE & MONETIZABLE
15
keywordsuggest.org Iconfinder WebPT
• Documentation
• Governance
• Integration with
project management
tools (collaboration)
• Security/Privacy
Management
• Value Abstraction
• API-able
Modular Shareable Monetizable
16. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
Potential Solutions
16
17. TWO DEPLOYMENT SOLUTIONS- PMML & PFA
Intended for Knowledge Sharing only 17
Data Mining Group an independent Vendor Led Consortium that develops Data Mining
Standards has come up with PMML (Predictive Model Mark Up Language) and PFA (Portable
Format for Analytics)
http://www.kdnuggets.com/2016/01/portable-format-analytics-models-production.html
http://dmg.org/
https://www.ibm.com/developerworks/library/ba-predictive-analytics4/ba-predictive-analytics4-pdf.pdf
https://www.ibm.com/developerworks/library/ba-ind-PMML1/
http://www.kdnuggets.com/faq/pmml.html
https://journal.r-project.org/archive/2009-1/RJournal_2009-1_Guazzelli+et+al.pdf
PMML PFA
File XML JSON & YAML
Maturity Mature but expanding Evolving
Nesting/Customization Model Parameters
Control Structures (Type System
of Model Parameters & data -
Callback function allowed)
Flexibility
Standard across most
scoring engines (better
than custom code)
More flexible than PMML but
safer than Custom Code
Scope
Data prep, Modeling,
Scoring, Sharing
+Pre/Post processing,
enforced memory model
18. PMML PROJECTS
Intended for Knowledge Sharing only 18
http://data-informed.com/pmml-puts-big-data-to-work/
19. POSITIONING OF PFA
Intended for Knowledge Sharing only 19
http://data-informed.com/pmml-puts-big-data-to-work/
20. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
Why this, Why now, why here?
20
21. Intended for Knowledge Sharing only
Intended for Knowledge Sharing only
BIGGER TRENDS THAT ARE SHAKING UP THE ANALYTICS WORLD FROM INSIDE OUT…
21
Demand Pressures: Complexity and nature of problems and their solutions,
type of audience & consumption framework evolving
Monetization opportunities- Direct, Indirect, Recurring
Artificial Intelligence, IoE and “Smart”ening of devices/systems faster than
expected.
Evolution of input data sources and integration of multiple insights sources
into decision making (A/B Testing, Research, Predictions/Scores from other
models)
Evolution from Service to Product to Platform (Build Once, Use
Everywhere)
…APIs are eating up our world
22. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
The parting words…
22
23. SUMMARY
Intended for Knowledge Sharing only
Predictive Analytics has stopped being “one-off competitive edge project
exercise” – it’s a necessary survival initiative for organizations
Scale, complexity, breadth of needs (including Monetization) demand
Platform approach.
“Build Once, Use Everywhere” -consumption of predictive analytics outputs
need to be easy to use, integrate, re-use/collaborate across multiple
initiatives
As everything becomes Productized via APIs, together they can become a
business problem solving ANI
23
Streaming Analytics is quickly evolving into Streaming Predictive Analytics
24. Intended for Knowledge Sharing only
Quick recap of what it is
Intended for Knowledge Sharing only
Appendix
25. THANK YOU!
Intended for Knowledge Sharing only
Would love to hear from you on any of the following forums…
https://twitter.com/decisions_2_0
http://www.slideshare.net/RamkumarRavichandran
https://www.youtube.com/channel/UCODSVC0WQws607clv0k8mQA/videos
http://www.odbms.org/2015/01/ramkumar-ravichandran-visa/
https://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a
RAMKUMAR RAVICHANDRAN
26. Intended for Knowledge Sharing only
Disclaimer:
Participation is purely on a personal basis and does not represent VISA,Inc. in any form or matter. The
talk is based on learning from work across industries and firms. Care has been taken to ensure no
proprietary or work related info of any firm is used in any material.
Director, Insights at Visa, Inc.
Enable Decision Making at the
Executives/ Product/Marketing level via
actionable insights derived from Data.
RAMKUMAR RAVICHANDRAN