The Data Unicorns

Copyright@STKI_2019 Do not remove source or attribution from any slide or graph 1
STKI Summit 2019
THE DATA
UNICORNS

Main Themes
2019
for Data and Analytics
DATA-CENTRIC THE DATA DEBT
01
02
03
06
05
08
07
04
Applications, processes &
decisions becoming data-
centric
Data Catalogs proliferation
But Lack of data ownership
and strategy remains
REAL PROBLEMS
Use of Design Thinking
and Empathy concepts to
solve REAL problems
DATA LITERACY
The data “language” in
organizations will
increase
DATA SCIENCE FOR ALL
AI, ML and
Automation will
empower citizen DS
DATA PRODUCTS
“Data product
managers” will manage
the entire lifecycle
DATA TEAMS
Agile-like teams will
collaborate around
data “products”
AUTOMATION
Automation in data
management data
science processes
STKI Summit 2019

ARE WE READY FOR A DATA-CENTRIC REALITY?
Intelligent Automation
Seamless Experiences
AI-fueled processes

Payroll Sales Call center
Software Software Software
Infra Infra Infra
Developers Developers Developers
Users Users
Silo Silo Silo
Application Centric Computing
(systems of transactions)
Customer Facing Computing
(systems of engagement)
DATA Centric Computing
(systems of decisions)
Automation
Revolution
(Preemptive)
AI/ML/DL Data Science
Intelligence
Systems
Human/ Machine Workforce
IoT Process
Engineering
UsersUsers
Digital
(forced)
Transformation
Channels
APIs
AGILE
Customer
Journey
UX
Marketing
Automation
RPA
Data Analytics

of organizations have adopted or have plans to
adopt AI in the next 5 years (IDC)
AI-Driven companies will steal
$1.2 Trillion from competitors
by 2020
The
race
for AI

63% of CEOs think AI will have a
greater impact than the internet
Source: PWC
The
race
for AI

Like it or not, the AGE of AUTOMATION is here
Ratio of human-machine working hours – 2018 vs. 2022
human machinehuman machine

Processes and business operations rely on data
This means future businesses will be
DATA CENTRIC

Source: PWC
10-year old gap!
WHAT DO CEOs SAY ABOUT THEIR OWN DATA GAP?

3 reasons for this gap:
lack of
analytical
Mindset
Data Siloing
Poor data
reliability

DATA DRIVEN
is more of a cultural thing
Being data-driven
means that people’s
decisions & actions
rely on data

DATA LITERACY* is a new
language, and we all need to
be fluent in it
*Data literacy: the ability to read, write and communicate data in context

Source: The data literacy project

POOR DATA LITERACY IS A MAJOR ROAD BLOCK FOR CDOs
Source: Gartner CDO Survey

thedataliteracyproject.org
A global community dedicated to building a data-literate culture for all

The rise of CDOs
29 FTEs reporting
directly to CDOs
25% increase in
CDOs funding
Will be Mission-critical
function in 75% of orgs.

Risk
Mitigation
Cost
Cutting
Value Creation
27% 28% 45%
CDOs time allocation:

63% 28%
DO YOU HAVE A CDO (DATA) IN PLACE?
Source: STKI DATA Survey, 2019Source: Gartner CDO Study YES

CDO survey: Israel
STKI CX
Survey
2019

ONE CDO STRUCTURE >DOESN’T< FIT ALL

Source: Oracle

Source: IBM

WE WANT
DATA
DEMOCRACY

WE WANT
DATA
DEMOCRACY
WHO ARE WE? DATA SCIENTISTS!
WHAT DO WE WANT?
WHEN DO WE WANT IT? NOW!!!
SELF SERVICE!

…and then came the DATA LAKE

MYTH: REALITY:
Data lakes are the answer to
data democratization and
self service.
Let’s upload a lot of data
into the lake as quickly as
possible.

This actually created a big problem.
Data is not harmonized, data lakes are
full of isolated data islands:
Organizations widened their data debt

The DATA DEBT
64% Duplicate data
Missing data - Fields that should
contain values, but do not.
25% data entry errors
No single version of the truth

“80% of data science is
cleaning the data
20 % is complaining
about cleaning the
data”
Source: Kaggle State of Data Science Survey
WHAT ARE DATA SCIENTISTS’ MAIN CHALLENGES?
1. Dirty data
2. No access
3. Privacy issues

20% 33%600B$ 12%
of average
data set is
dirty
is the annual
cost to the
U.S economy
due to bad
data
of company
projects fail
because of
weak data
is the average
annual
revenue loss
(Sources: Springer Link; IBM )
BAD DATA = BAD DECISIONS

STORE
ACCESSDEPLOY
PREPARE
MODEL
6 1
2
3
5
4 Store the data:
DW/ DL/ Data Mart/
Logical DW
Transform
Clean
Understand
DATA CENTRIC ARCHITECTURE
6.DEPLOY 1.ACCESS
2.INGEST
3.STORE4.PREPARE
Model
5.MODEL
Learn
Train
GOVERN INGEST
Run code in operational
processes
Systems of Decisions
Data Dictionary

From Waterfall to Agile, Iterative Processes

WHAT’S THE RIGHT BALANCE
for a DATA-CENTRIC-READY BUSINESS?

Define a data governance strategy
Enforce a “data catalog” policy
Define harmonized data definitions
Create a central COE for DS teams
HARMONIZED DATA PLATFORM
AGILE TEAM
AUTOMATION (DATAOPS)
1
2
3
Create data teams/ squads
Product owner
Automate as many processes as
possible in the DS value chain
Use DataOps/ MLOps as reference

AGILE TEAM
AUTOMATION: DATAOPS
1
2
3
Product owner

What is stopping you from becoming data centric?
Source: Atscale Big Data Maturity Report

Do you need a data catalog?
Yes.

DISCOVERY
Easily Search
and browse
data
WHY DO
YOU NEED
A DATA
CATALOG?
ENABLES
Self service
to DS and
analysts
TAGGING
Data is
described
technical &
business
CURATION
Self service
to DS and
analysts
FEEDBACK
Rating and
reviews by
users
BALANCE
Between
the need
to control
and to
consume
AWARENESS
Be informed
of relevant
and available
data
HARMONIZE
Enable single
version of the
truth

Through to the end of 2022, manual tasks in
data management will be cut by 45% thanks
to ML and automated service-level
management (Gartner)
AUTOMATION
in
Data Management
Cleaning, wrangling, transforming, and loading

AGILE TEAM
AUTOMATION: DATAOPS
1
2
3
Product owner

WANTED: Analytics Engineer
Research tasks
Build/plan models
Statistical languages
Prototype ML
models
DATA
ENGINEER
BUSINESS
ANALYST
DATA SCIENTIST
Ingestion
Storage
Transformation
Preparation
Virtualization
Enrichment
Business Logic
Understand
the impact to
the business
R, Python
Hadoop
Spark
Kafka
ML
Data Visualization
Unstructured data
Business understanding
Communication skills
Storytelling skills
DB Administration
Storage
Visualization
SQL
Data Pipeline
Business understanding
Communication skills
Data Architecture
NoSQL
Data Integration, ETL, APIs

Data Engineer
Business Analyst Operations Data Scientist

AGILE TEAM
AUTOMATION: DATAOPS
1
2
3
Product owner

THE GOAL:
Managing data products

DATAOPS IS NOW A “THING”

The next evolution: MLOps (“A/B Testing” for DS)
ML training (a.k.a model generation, model build or model fit) that generates the model
ML inference (a.k.a prediction, scoring, or model serve) that generates the insights.

Source: EY

Source: EY
Chatbots,
NLP/NLG
and RPA.
Chatbots,
NLP/NLG.
IPA, ML,
NLP/NLG,
RPA
IPA (Intelligent
process automation),
ML and RPA
Deep Learning, ML
and IPA (Intelligent
process automation)

GNS HEALTH: DISCOVERING CAUSAL LINKS
AGRICULTURE : FARMERS RECOMMENDATIONS
GNS applies ML to find overlooked relationships in
patients’ health records. It creates hypotheses to
explain it and then suggests which are most likely.
Result: GNS uncovered a new drug interaction
hidden in unstructured patient notes.
AI system provides real-time recommendations for
farmers on how to increase productivity (which
crops to plant, where to grow, nitrogen in soil…)
Result: farmers happy about the crop yields
obtained with AI’s guidance.
AI solution that improves accuracy of fraud detection.
Monitors millions of transactions daily, purchase
location, customer behavior, IP addresses… to identify
patterns that signal possible fraud.
DANSKE BANK: AI FOR FRAUD DETECTION
AI & ML USE CASE EXAMPLES
Result: Improved fraud detection rate by
50%, decreased false positives by 60%.
Investigators can concentrate efforts on
flagged transactions.

OCEAN MEDALLION
A “LOVE BOAT” EXPERIENCE
DANSKE BANK: AI FOR FRAUD DETECTION
Instead of just alleviating the “friction”
of typical travel experiences (lines,
room keys, paying for things) it will use
data to anticipate what you want to
do, eat, and see.
The medallion can be used to pay;
unlock the door to your room as you
approach; can be used on the ship’s
gambling platform; provide
recommendations based on preferences

35% consider Machine learning models to be ‘black
boxes’ (but feel the models can be explained by
experts – “explainers”).
10% of the participants are confident of explaining
most or all models.

#MyData

What is
“personal data”?
How do you
manage it?
Source: Skillzme

Fair data economy

42% say that lack of trust prevents
them from using digital services
Source: Sitra’s 2018 four country survey (Europe: Finland, Netherlands, France, Germany)
Trust is built by having the power to
influence how your data is used
In a survey for IBM, 75 percent of respondents said they will
not buy a product from a company – no matter how great the
product – if they don’t trust that company to protect their data
“Give me your data”

• Build a data catalog as a “data-lake gatekeeper”
• Tackle point-specific data quality projects
• Assign mixed data teams and an “agile way of
working” for specific dynamic analytic
• Automate as much as possible!
• Define key business questions:
“Start with the problem, not the data”
• Design a data governance strategy
• Establish CDO-IT-LOBs collaborative processes
• Focus on promoting data literacy
• Implement DevOps/DataOps principals

Einat Shimoni
EVP & Senior Analyst
STKI

The Data Unicorns

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à The Data Unicorns

Similaire à The Data Unicorns (20)

Plus de Einat Shimoni

Plus de Einat Shimoni (20)

Dernier

Dernier (20)

The Data Unicorns