Powering Self Service Business Intelligence with Hadoop and Data Virtualization

Powering Self Service Business
Intelligence with Data Virtualization
Sean Roberts, Hortonworks
Mark Pritchard, Denodo
January 2017

2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Sean Roberts
Partner Engineering

Data Doubles
Every Two Years
44ZB By 2020
3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

The Old Way
System Centric
Procedural
Hierarchical
Scheduled
Homogeneous
The New Way
User Centric
Agile
Dynamic
Real-Time
Contextual
Data Doubles
Every Two Years
44ZB By 2020

Capture
streaming data
Deliver
perishable insights
Combine
new & old data
Store
data forever
Access
a multi-tenant data lake
Model
with more data
DATA AT RESTDATA IN MOTION
ACTIONABLE
INTELLIGENCE
Perishable Insights Historical Insights

Secure
Real-time
Streaming
Integrated
Hortonworks DataFlow for Data in Motion
Powered by Apache NiFi, Kafka, and Storm

Hortonworks Data Platform for Data at Rest
Powered by Open Enterprise Hadoop
Open
Interoperable
Ready
Central

Claims
Optimization
Social
Sentiment
Cohort
Selection
Bill Shock
Physician
Notes
Device
Monitoring
R & D
Quality
Benchmarks
Patient
Experience
Seasonal
Staffing
Net
Promoter
Score
Supply
Chain
Sentiment
Analysis
Patient
Outreach
360°
Patient
View
Patient
Throughput
Customer
Churn
Analysis
STARS
Ratings
Genomics
Remote
Monitoring
Drug
Diversion
Census
Proactive
Maintenance
Preventative
Medicine
Inventory
Medication
Safety
OPEX
Reduction
Lab Notes
Archive
Mainframe
Offloads
Device
Data
Ingest
Rapid
Reporting
Digital
Protection
Data
as a
Service
Fraud
Prevention
Real-time
Decision
Support
INNOVATE
RENOVATE
E X PLO R E O PTIMIZE TR A NS FO R M
ACTIVE
ARCHIVE
ETL
ONBOARD
DATA
ENRICHMENT
DATA
DISCOVERY
SINGLE
VIEW
PREDICTIVE
ANALYTICS
HEALTHCARE
Care-path
Best
Practices
OR
Optimization
HCAHPS
Scores
Staffing
Predictions
Proactive
Outreach
Legacy
System
Data
Imaging
Archive
Historical
Patient
Records
Improved
Drug
Yields

Actionable Intelligence Makes Healthcare Precise and Personal
Patient
Records
Lab Data
Pharmacy
Data
Patient
Locations
Wearables
Intra-Network
Data
Sensor
Data
Claims
Data
Social
Media Physician
Notes
Patient
Satisfaction Data
Clinical
(EMR) Data
SINGLE VIEW OF
PATIENT
REAL-TIME VITAL
SIGN MONITORING
BILLING &
REIMBURSEMENTS
EMR
OPTIMIZATION
SUPPLY CHAIN
OPTIMIZATION

Mercy’s Journey
Mercy Medical System Sought a Data Lake for a Single View of its Patients –
“One Patient, One Record”
Existing platform impeded goal of enriching Epic data for 1 million patients
over 35 Hospitals and 500 clinics
Moving Epic EMR data to Clarity EDW took 24 hours and was “never going
to enable real-time analytics”. Now that takes 3-5 minutes with HDP.
Improved billing processes resulted in $1M additional annual revenue
from newly documented secondary diagnoses and care

Better Health Through “One Patient, One Record”
ACTIVE
ARCHIVE
Lab Notes
DATA
DISCOVERY
OPEX Efficiency
SINGLE
VIEW
Billing
DATA
DISCOVERY
Vital Signs
SINGLE VIEW
Single Patient
Record
ACTIVE
ARCHIVE
Epic EMR
Replication
ACTIVE
ARCHIVE
Privacy
Database
DATA
ENRICHMENT
Epic Enrichment
PREDICTIVE
ANALYTICS
Device Data
Ingest
Move to Clarity wouldn’t
enable real-time analytics
Existing platform
impeded goals
Data enrichment needed
for 1 million patients
Move off Epic
took over 24 hours
SITUATION
3-5
Minutes
$1M Additional
Annual Revenue
From “Never”
to “Seconds”
900x Faster
move data off Epic to
Clarity with HDP
from improved
billing process
accelerated
researcher insight
ingest of ICU
vital signs
PREDICTIVE
ANALYTICS
Preventive Care
ETL OFFLOAD
Medical Decision
Support

Interoperable with Leading Technologies
Partners

Information spread across different
systems
IT responds with point-to-point data
integration
Takes too long to get answers to
business users
The Self-Service Challenge
 Data Is Siloed Across Disparate Systems
MarketingSales ExecutiveSupport
Database
Apps
Warehouse Cloud
Big Data
Documents AppsNo SQL
“Data bottlenecks create business
bottlenecks.”
– Create a Road Map For A Real-time, Agile, Self-Service Data
Platform, Forrester Research, Dec 16, 2015

Denodo and Modern Data Architecture

Powering Self Service Business
Intelligence with Data Virtualization
Mark Pritchard
January 2017

Agenda1.Data Virtualization and Benefits
2.Case in Point – Vizient
3.Modern Data Architecture
4.Self-Service BI on Distributed Datasets
5.Denodo Platform for Data Virtualization
6.Q&A

Data Virtualization and Benefits

18
-Source: “Gartner Market Guide for data virtualization – 2016”
Data virtualization technology can be used to create virtualized and
integrated views of data in memory (rather than executing data movement
and physically storing integrated views in a target data structure), and
provides a layer of abstraction above the physical implementation of data.”
Data Virtualization – Definition

19
Data Virtualization
Real-time Data Integration
“Data virtualization integrates disparate data sources in real time or near-real time
to meet demands for analytics and transactional data.”
– Create a Road Map For A Real-time, Agile, Self-Service Data Platform, Forrester Research, Dec 16, 2015
Publishes
the data to applications
Combines
related data into views
Connects
to disparate data sources
2
3
1

20
Denodo and Modern Data Architecture

21
Benefits of Data Virtualization
“Get it Real-time and Get it Fast!”
Better Data Integration
Lower integration costs by 80%.
Flexibility to change.
Real-time (on-demand) data services.
Complete Information
Focus on business information needs.
Include web / cloud, big data,
unstructured, streaming.
Bigger volumes, richer/easier access to
data.
Better Business Outcome
Projects in 4-6 weeks.
ROI in <6 months.
Adds new IT and business capabilities
“Benefits of Data Virtualization: get it real-time and get it fast!”
– William McKnight, President, McKnight Consulting Group

23
Who is Vizient?
Network for non-profit hospitals and alliance of academic medical centers
Network of not-for-
profit healthcare
organizations to
improve
performance and
efficiency in clinical,
financial and
operational
management
Combination of
VHA, University
HealthSystem
Consortium,
Novation,
MedAssets Spend,
Clinical Resources
Management and
SG2
Experts with
purchasing power,
insights and
connections that
accelerate
performance for
members

24
Purpose, Mission and Strategic Aspirations
Mission
To connect
members with the knowledge, solutions
and expertise that accelerate performance
Strategic Aspirations To become an indispensable partner to
healthcare organizations
Purpose
To ensure
members deliver exceptional, cost
effective care

25
Vizient delivers brilliant, data-driven resources and
insights — from benchmarking and predictive
analytics to cost-savings — to where they’re needed
most.
Empowering Brilliant Connections

27
Modern Data Architecture
HTTP
FTP
HL7
Flat File
Share
Subscribe
Attach
Initial
Event
Share
Persist
Event
Share
Ongoing
Event
Usage
Purchase
Data
Open
Data
RAW CLEAN AGGREGATED ENRICHED
RDBMS
RDBMS
ODS
DW
STAGESOURCE PROCESS PERSIST SERVE
Rules
Batch
Human
Process
Hadoop
Hadoop
Lake
Machine
Learning
Analytics
Consulting

28
Modern Data Architecture
HTTP
FTP
HL7
Flat File
Share
Subscribe
Attach
Initial
Event
Share
Persist
Event
Share
Purchase
Data
Open
Data
RAW CLEAN AGGREGATED ENRICHED
RDBMS
RDBMS
ODS
DW
STAGESOURCE PROCESS PERSIST
Ongoing
Event
Usage
SERVE
Rules
Batch
Human
Process
Hadoop
Hadoop
Lake
Machine
Learning
Analytics
Consulting
DataVirtualization

Powering Self-Service Discovery with Data
Virtualization

30
Financial Data Mart
Primary Use Case
o Unify disparate accounting and finance data marts across various legacy
organizations into a shared repository
Secondary Use Cases
o Provide a unified source for key BI initiatives like the GPO Dashboard
o Support reporting needs as legacy systems are migrated or replaced during
integration of Vizient and L-MDAS (dbVision, etc.)
o Provide a final resting place for archived legacy sources like Solomon, Epicor,
etc.
VHA
MedAssets
UHC
Vizient

31
Financial Data Mart
Architectural Approach
Denodo was selected as the data platform in order to utilize the following
features of the software:
o Data Virtualization allows sources in various mediums and locations to be
integrated without physically moving the data
o Data Abstraction allows data to be represented consistently within the
DataMart while data sources are moved or replaced behind the scenes
o Data Integration allows for a single seamless view to be created across a
subject area (e.g. “Supplier Sales”) with varied data transformation rules
for each data source within the subject area (PRS, dbVision).

32
GPO Dashboard
Primary Use Case
o Provide a consolidated view of supplier sales data across all customers of
legacy Vizient & Med Assets organizations.
o Financial DataMart (on Denodo) for data source
o Denodo TDE Exporter Tool for daily data extracts to Tableau:
 Report Data
 Report User Security
o Tableau for report development and distribution

33
GPO Dashboard
Key Challenges
o Balance between data timeliness and report performance
 Tableau reports performed best utilizing the TDE format (cached/extracted dataset) as
opposed to a live connection
 This meant that the report caches required daily refreshes, and data extraction had to
be appropriately tuned
 Denodo features such as dataset statistics and indexing greatly contributed to this
performance tuning
o Provisioning user security at cell level
 The requirement for some internal report users to be restricted to the
members/customers to which they are assigned meant that a new report security
approach was needed
 Reliance on TDEs for report data necessitated the integration of security in the reporting
layer
 Tableau’s “data blending” feature allows user security to be specified within a separate
dataset
 This also supports reuse of the security view in other reporting environments.

34
Contract Sales Actualizer Dashboard
Primary Use Case
o Integrate Member Spend and Supplier Sales data from all Vizient organizations to identify
opportunities for increasing contract utilization
Other Use Cases
o Maintain consistency (Single Source Of Truth) with GPO dashboard regarding:
 Supplier Sales Data
 Dimension Data
 User Security
o Data source utilizes Denodo to reuse overlapping datasets (sales, dimensions, security) while
allowing separate virtualized views to be created for new datasets (member spend) which can
be also be reused by future projects
o Reporting components match approach used by GPO Dashboard

35
Contract Sales Actualizer Dashboard
Key Challenges
o Successful integration of Exadata RDM as a data source for Denodo.
 Approach utilizes the strength of Exadata RDBMS for aggregating large
quantities of data quickly
 Denodo to integrate the data with similar legacy SQL Server data sources to
create a comprehensive view of Vizient member spend
o Scalability/Configuration Management
 Advances were made to support parallel development of this project and
continued efforts on GPO dashboard
 Compartmentalization features within Denodo allow for code changes in each
project to be version controlled and assessed for dependencies
 Process guidelines are being authored to allow for multiple development efforts
on the same datasets

Denodo Platform for Data Virtualization

37
Accelerate Your Fast Data Strategy with Denodo Platform 6.0
New Release of Denodo Platform Delivers Breakthrough Performance, Accelerates
Adoption, and Expedites Business Use of Data
Breakthrough
Performance
Dynamic Query Optimizer
delivers breakthrough performance
for big data, logical data
warehouse, and operational
scenarios.
Data Virtualization
In the Cloud
Denodo Platform for AWS
accelerates adoption of data
virtualization.
Self-service Data
Discovery and Search
Self-service data discovery
and search expedites use of
data by business users.
“Very happy with Denodo version 6. Well done!”
– Claudia Imhoff, President, Intelligent Solutions

38
Common Data Virtualization Use Cases
Data Virtualization
BIG DATA, CLOUD INTEGRATION
 Advanced Analytics
 Data Warehouse Offloading
 Big Data for Enterprise
 Cloud / SaaS Integration
AGILE BUSINESS INTELLIGENCE
 Logical Data Warehouse
 Virtual Data Marts
 Self-Service BI
 Operational BI / Analytics
SINGLE VIEW APPLICATIONS
 Single Customer View - Call Centers, Portals
 Single Product View - Catalogs
 Single Inventory View - Inventory Reconciliation
 Vertical Specific - Single View of Wells
DATA SERVICES
 Unified Data Services Layer
 Logical Data Abstraction
 Agile Application Development
 Linked Data Services

39
Why Denodo Data Virtualization?
Data Virtualization
Exceptional pre-sales service and
customer support
 Technical expertise and response time rated best
by costumers.
 Enhancement requests typically made available in
days / weeks.
Lower overall TCO and ROI  Address more use cases with a single platform.
 Aggressive “all-in-one” pricing for entire platform;
flexible pricing models.
Broad functionality and winning
innovation in a fully integrated
platform
 Flexible and broad use cases.
 Integrated purpose-built for data virtualization –
very easy to develop / deploy / change models.

Thanks!
www.denodo.com info@denodo.com
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical,
including photocopying and microfilm, without prior the written authorization from Denodo Technologies.

Powering Self Service Business Intelligence with Hadoop and Data Virtualization

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Powering Self Service Business Intelligence with Hadoop and Data Virtualization

Similaire à Powering Self Service Business Intelligence with Hadoop and Data Virtualization (20)

Plus de Denodo

Plus de Denodo (20)

Dernier

Dernier (20)

Powering Self Service Business Intelligence with Hadoop and Data Virtualization