How to Merge the Data Lake and the Data Warehouse: The Power of a Unified Analytics Warehouse

IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
How to Merge the Data Lake and the Data Warehouse
The Power of a Unified Analytics Warehouse
John Santaferraro
Research Director
EMA
Jeremiah Morrow
Senior Product Marketing Manager
Vertica

Watch the On-Demand Webinar
Slide 2
 How to Merge the Data Lake and the Data Warehouse: The
Power of a Unified Analytics Warehouse on-demand webinar:
https://info.enterprisemanagement.com/how-to-merge-the-data-lake-
and-the-data-warehouse-webinar-ws
 Check out upcoming webinars from EMA here:
https://www.enterprisemanagement.com/freeResearch

Featured Speakers
Slide 3 © 2020 Enterprise Management Associates, Inc.
John Santaferraro, Research Director, EMA
John is the research director for analytics, business intelligence, and data management
at EMA. He has 23 years of experience in data and analytics, from startups to executive
positions at Fortune 50 companies. His deep understanding of the industry comes from
years of leadership in implementation, product and marketing organizations, along with
multiple big data imagineering efforts for finance, communications, retail, manufacturing,
healthcare, events, oil and gas, and utilities.
Jeremiah Morrow, Senior Product Marketing Manager, Vertica
Jeremiah is responsible for marketing initiatives involving Vertica’s channel and
technology partners. He has worked in sales, marketing, and analyst relations for
technology companies ranging from infrastructure to software, and he is passionate
about building a strong and collaborative ecosystem to drive value for customers,
partners, and Vertica.

Sponsor
Trusted by AT&T, Cerner, Uber, The Trade Desk, and many other data-driven organizations, Vertica is the
unified analytics warehouse that solves three current market challenges that every organization faces.
Despite the disappointment in Hadoop, HDFS data lakes represent a very significant investment for many
companies, but the value is not equivalent to original expectations. Combined with the explosion of cloud
object storage, organizations struggle even more to unify their data. In addition, organizations favor a multi-
cloud or hybrid cloud and on-premises deployment strategy as they face the reality of cloud vendor lock-in,
costs, and migration challenges. Finally, machine learning is no longer a data science project, but must be
put into production to deliver the predictive analytics information in time to allow proactive actions.
Vertica is a Unified Analytics Warehouse that:
• Unifies HDFS data and Object Storage data lakes to capitalize on storage investments and maximize
business value.
• Unifies a company’s deployment options spanning multi-cloud and on-premises to embrace cloud
innovations, prevent lock-in, and meet regulatory and security requirements.
• Unifies the data science community and the business analyst and IT community, enabling each to
continue to use their preferred tools and languages while operationalizing machine learning at scale for
real-time predictive analytics.
Visit www.vertica.com for more information.

Agenda
• The changing business and data landscape
• Inadequacies of the data lake and data warehouse
• The race for a unified analytics warehouse
• Requirements for a unified analytics warehouse
• Deploying a unified analytics warehouse
• Unified analytics warehouse for cloud and hybrid
• Vertica and the unified analytics warehouse

INDUSTRY ANALYSIS & CONSULTINGSlide 6
The changing business and
data landscape

Digital, Mobile, IoT
Digital Mobile Internet of Things

Evolution of Data Warehouse
Architecture

Business
Intelligence
CRM ERP
Billing
Application Data
Customer
Operational
Financial
ETL
Analytics Database
Transactional
Data
Message Queues
Files
Data Warehouse
Batch
Visualization

Data Lake
1
0
Contextual
Data
Files
Weather
Geo
Low
Latency
Batch
Transactional
Data
Application Data
OLTP/ODS
Batch ETL
or EL with T
done on
mass
storage
Data Prep /
Enrichment
Streaming
Data
Application data
Web clicks
Logs
Sensors
Operational metrics
User tracking
Geo-location
Visualization
Applications
Artificial Intelligence
Cloud
On-Premises
AND / OR
Distributed
Pub/Sub
Distributed
Prepped
Data
Object
Storage
Stream Processing
Data Lake
Mass Storage
Query Engine /
Machine
Learning

Cooperative Data Architecture
1
1
Contextual
Data
Files
Weather
Geo
Low
Latency
Batch
Transactional
Data
Application Data
OLTP/ODS
Batch ETL
or EL with T
done on
mass
storage
Data Lake
Mass Storage
Data Prep /
Enrichment
Streaming
Data
Application data
Web clicks
Logs
Sensors
Operational metrics
User tracking
Geo-location
Visualization
Applications
Import
Export
Query
Cloud
On-Premises
AND / OR
Distributed
Pub/Sub
Distributed
Columnar
Data
Object
Storage
Stream Processing

The Need for a Unified Analytics Warehouse
“Only structured and semi-structured data
combined can deliver on the promise of complete
business insights.”

Inadequacies of the data lake
and data warehouse

The Data Swamp (Lake)
Data Lake
Shortcomings
• Built for semi-structured data
and requires structuring of data
on read to analyze
transactional data.
• Database component lacks
enterprise capabilities and
does not perform will
compared to analytic platforms.

The Data Jailhouse (Warehouse)
Data Warehouse
Shortcomings
• Built for structured data and
requires structuring and
ingestion of semi-structured
data to analyze it.
• Too much effort required and
doesn’t support real-time ad-
hoc analysis of semi
structured data.

The race for the unified
analytics warehouse

From the Data Lake Side
Build On
• Database Technology
• SQL Query Engines

From the Data Warehouse Side
Drill Through
• Semi-Structured Data
• Tiered Storage
• Data Science Tools

Requirements for a unified
analytics warehouse

What is a Unified Analytics Warehouse?
• Unified
• Adequately handles multi-structured
data in a single platform
• Analytics
• Data lake and the data warehouse
are primarily for analytics.
• Warehouse
• Stores multi-structured data in an
organized and accessible manner.

Unified Analytics Warehouse

What is a Unified Analytics Warehouse?
• Unified
• Adequately handles multi-structured
data in a single platform
• Analytics
• Data lake and the data warehouse
are primarily for analytics.
• Warehouse
• Stores multi-structured data in an
organized and accessible manner.

2
5
Contextual
Data
Files
Weather
Geo
Low
Latency
Batch
Transactional
Data
Application Data
OLTP/ODS
Batch ETL
or EL with T
done in
warehouse
Streaming
Data
Application data
Web clicks
Logs
Sensors
Operational metrics
User tracking
Geo-location
Stream Processing
Cloud
Visualization
Applications
Shared
Storage
On-Premises
HDFS HDFS
AND / OR Ingestion/ ELT/
Data Prep / Enrichment
Managed ML
Models
Reporting /
BI
Data Science /
ML
Departmental
Use

Data and the Unified Analytics Warehouse
• Structured Data
• Complex Data Types
• JSON
• XML
• Textual Data
• Streaming Data

Enterprise and the Unified Analytics Warehouse
• Administration
• Management
• Orchestration
• Security
• Privacy
• Performance
• Scalability
• Agility
• Resiliency
• AI Enablement
• Recommendations
- Structure, Schema, Data
Relationships, Tuning
• Automation
- Metadata Generation,
Tuning, Maintenance,
Elasticity, Query Routing,
Data Tiering, Change
Management, Code
Generation

Infrastructure and the Unified Analytics Warehouse
• Analytic Platform
• Distributed File System
• Object Storage

Cloud and the Unified Analytics Warehouse
“53% of all data is
now in the cloud…”
• Single Pane of Glass
• Simple Migration
• Common Foundation

Analytics and the Unified Analytics Warehouse
Embedded Analytics
• Prebuilt
• Common
• High-Performance
• Rapid Deployment
Analytical Performance
• Data Intensive
• Compute Intensive
• High Intensity
• High Concurrency

Users and the Unified Analytics Warehouse
“Users also expect a platform that allows them
to put aside religious beliefs about how analytics
should be done.”

Deploying a unified analytics
warehouse

Best Practices for the Unified Analytics Warehouse
 Choose a vendor with a vision for the UAW
 Choose a platform well into UAW transformation
 Choose a project with high value
 Choose a project with a mix of data and use cases
 Test use case capabilities in a hybrid environment
 Plan for end of life and continued migration

Unified analytics warehouse
for cloud and hybrid environments

3 Necessities of for Cloud and Hybrid Success
• Common Management
• Common Infrastructure
• Interoperability
• Data interoperability
• Query interoperability
• Migration interoperability

Vertica and the unified analytics
warehouse

The Unified Analytics Warehouse

What is Vertica?
3
8
SQL Database
Load and store data in a
data warehouse
designed for blazing
fast analytics
Query Engine
Ask complex analytical
questions and get fast
answers regardless of
where the data resides
Vertica is an advanced analytics platform built for the scale and complexity of today’s data-driven
world. It combines the power of a high-performance, MPP query engine with advanced analytics
and machine learning.
Analytics & ML
Create, train and deploy
advanced analytics and
machine learning models
at massive scale

Remove scale, performance and capacity constraints
3
9
Get data quickly enough to act upon it, explore your data interactively,
and enable everyone to make their own data-driven decisions
Fear of more uses or growing data volumes is a thing of the past
Scale Data Volumes Scale Users
SQL Database
+
Vertica Advanced Analytics Platform
+
Get data quickly enough to act upon it, explore your data interactively,
and enable everyone to make their own data-driven decisions
Analytics & ML Query Engine

2020 McKnight Consulting Group Benchmark
4
0
“At 60 concurrent users, Vertica in Eon
Mode was 1.8x less expensive than
Redshift. The unnamed data cloud
platform consistently had the highest price
performance.”
“Vertica in Eon Mode had 1.14x the QPH
of the next highest database (unnamed
data cloud platform) for the 60
concurrent user workload.”
“Vertica in Eon Mod consistently had the
shortest elapsed time for the longest
running [query] thread across the
concurrency profiles at 250 TB.”
Vertica in Eon Mode, Amazon Redshift, and an Unnamed Data Cloud Platform

Philips Healthcare –
Predictive Maintenance

Machine Learning can simplify business processes and
improve the customer experience
4
2
System
Problem
Customer
Call Dispatch
Onsite
Trouble-
shooting
Remote Monitoring Predicts
Potential failure Service Scheduled Problem Avoided
Parts
Delivery
Repair or
Replace
System
Functional
Reactive Maintenance

High Level Architecture
4
3
Data Lake (Raw Data Archive)
Parallel Extract Transform Load
Legacy
Database
s
CRM &
Business
Systems
Installed Base
Philips Remote Service Network
DSP
(HSI)
Remote
Monitoring Radar
4.0
Remote Service
RSW & CHA
Reliability
Dashboards
R&D Access

Outcomes
4
4
"Remote service provides us with
an engineer online all the time.
They tell us when we've got a fault
before we know we've got a fault.
And not only that, they can fix the
fault before we knew we had a
fault. And that's impressive."
Cobalt Imaging, Gloucester
"Now we will have more uptime
on the scanner and potentially be
able to see more patients…It's a
new level of service for us, with a
greater satisfaction." -
Radiographer, New Stobhill
Hospital in Glasgow
500 TB of data in more than 300 tables. 30 trillion data points
80 different data sources integrated
24/7
months from scratch to production8
live data feeds. Millions of logs per week.
…and marching towards 0 unplanned downtime

EMA Independent Analysis of Vertica and the UAW
45 © 2020 Enterprise Management Associates, Inc.
0
2
4
6
8
10
Data Requirements
Enterprise
Requirements
Infrastructure
Requirements
Hybrid Requirements
Analytical
Requirements
User Requirements
From EMA - The Race for a Unified Analytics Warehouse

Question and Answer: Log Questions in the Q&A panel
© 2020 Enterprise Management Associates, Inc.
Interested in Learning More?
LEARN ABOUT:
DOWNLOAD
EMA – The Race for a
Unified Analytics Warehouse -
https://www.vertica.com/resource/the-race-for-
a-unified-analytics-warehouse/
McKnight Consulting Group – Cloud Database
Performance Benchmark -
https://www.vertica.com/cloud-database-
benchmark-report-2/
Vertica Community Edition -
https://www.vertica.com/try/
Blog – https://www.vertica.com/blog
LinkedIn -
https://www.linkedin.com/showcase
/vertica-co/

How to Merge the Data Lake and the Data Warehouse: The Power of a Unified Analytics Warehouse

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à How to Merge the Data Lake and the Data Warehouse: The Power of a Unified Analytics Warehouse

Similaire à How to Merge the Data Lake and the Data Warehouse: The Power of a Unified Analytics Warehouse (20)

Plus de Enterprise Management Associates

Plus de Enterprise Management Associates (20)

Dernier

Dernier (20)

How to Merge the Data Lake and the Data Warehouse: The Power of a Unified Analytics Warehouse