This document discusses big data and how companies use it. It provides examples of common big data uses like churn reduction, targeted advertising, and fraud detection. It also outlines an enterprise big data vision including collecting data from multiple sources, exploring and analyzing the data in parallel, and generating real-time insights and periodic analytics reports. Finally, it discusses different patterns for enterprise big data usage including operational data reservoirs, transformational data refactories, and low latency reactive data systems.
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Data Warehouse to Data Science
1. What is Big Data? Big Data Stack
Companies Using Big Data
• Churn Reduction and Customer Retention
• Natural Language Processing and Sentiment Analysis
• Targeted Advertising and Marketing Optimisation
• Personal Recommendation
• Fraud Detection and Prevention
• Social Media and Game Analytics
• Risk and Exposure Analysis
• Real time Insights and Reactive Processing
Industry Use Cases
2. Enterprise Data Lake
Big Data Vision
Centralised High Speed Analytics Hub
Periodic AnalyticsReal-time Insight
Stakeholder Dashboard
N2N4
N1
N3
Multiple Data Sources
3. DIVIDE CONQUER INSIGHT
DATA DROPBOX
Split Data in Block
Replicate and Store
Petabytes of Resilience
DATA EXPLORE
1000s of Parallel Threads
Explore Every Path
Machine Learning
DATA INSIGHT
Real Time Action
Periodic Dashboards
Iterative Evolution
4. ENTERPRISE BIG DATA LAKE
REFINE EXPLORE ENRICH
BATCH INTERACTIVE ONLINE
OPERATIONAL DATA SOURCES
Transactions, Interactions, Observations
time between load to access of data
INSIGHT
Enterprise Big Data Usage Patterns
5. DATASOURCES
Traditional Sources
(RDBMS, OLTP, OLAP)
New Sources
(weblogs, email, social media, forum)
DATASYSTEMS
RDBMS EDW MPP
TRADITIONAL REPOS
ENTERPRISE
BIG DATA
PLATFORM
APPLICATIONS
Business
Analytics
Custom
Applications
Enterprise
Applications
Incumbent Enterprise Data Warehouse
1
2
3
Traditional enterprise data warehousing
“Schema first, data last” approach to
loading data
1 Extract, Transform & Load
2 Schema and Join
3 Deliver
REFINE EXPLORE ENRICH
6. DATASOURCES
Traditional Sources
(RDBMS, OLTP, OLAP)
New Sources
(weblogs, email, social media, forum)
DATASYSTEMS
RDBMS EDW MPP
TRADITIONAL REPOS
ENTERPRISE
BIG DATA
PLATFORM
APPLICATIONS
Business
Analytics
Custom
Applications
Enterprise
Applications
Operational Data Reservoir
REFINE EXPLORE ENRICH
1
2
3
Transform & refine ALL sources of data
“Data first, schema last” approach to
loading data.
Schema created on demand based on case
1 Capture
2 Process
3 Distribute & Retain
7. DATASOURCES
Traditional Sources
(RDBMS, OLTP, OLAP)
New Sources
(weblogs, email, social media, forum)
DATASYSTEMS
RDBMS EDW MPP
TRADITIONAL REPOS
ENTERPRISE
BIG DATA
PLATFORM
APPLICATIONS
Business
Analytics
Custom
Applications
Enterprise
Applications
Transformational Data Refactory
REFINE EXPLORE ENRICH
1
2
3
Leverage “data lake” to perform iterative
investigation for value
“Direct to data” approach to access the data
from applications
1 Capture
2 Process
3 Explore & Visualse
8. DATASOURCES
Traditional Sources
(RDBMS, OLTP, OLAP)
New Sources
(weblogs, email, social media, forum)
DATASYSTEMS
RDBMS EDW MPP
TRADITIONAL REPOS
ENTERPRISE
BIG DATA
PLATFORM
APPLICATIONS
Business
Analytics
Custom
Applications
Enterprise
Applications
Low Latency Reactive Data
REFINE EXPLORE ENRICH
1
2
3
Create intelligent applications
Collect data, create analytical models and
deliver to online applications
“Reactive Data” or “Active Data approach
1 Capture
2 Process & Compute
3 Deliver in Real Time
NOSQL
9. DATASOURCES
Traditional Sources
(RDBMS, OLTP, OLAP)
New Sources
(weblogs, email, social media, forum)
DATASYSTEMS
ENTERPRISE
BIG DATA
PLATFORM
APPLICATIONS
Tool Integration
OPERATIONAL TOOLS
DEV & DATA TOOLS
10. understand customer preferences
embrace diversity and complexity react in real-time
1
3
2
Harness your Data
drive strategic business directioncreate data value
improve customer experience
STAY AHEAD
& INNOVATE
Notes de l'éditeur
Real-time insights, real-time platform
Chandan to explain the process of the data hub