Contenu connexe
Similaire à Why Data Virtualization? By Rick van der Lans (20)
Why Data Virtualization? By Rick van der Lans
- 1. Copyright © 2018 R20/Consultancy B.V., The Netherlands. All rights
reserved. No part of this material may be reproduced, stored in a
retrieval system, or transmitted in any form or by any means,
electronic, mechanical, photographic, or otherwise, without the
explicit written permission of the copyright owners.
Why
Data Virtualization?
Rick F. van der Lans
Industry analyst
Email rick@r20.nl
Twitter @rick_vanderlans
www.r20.nl
- 2. Copyright © 2018 R20/Consultancy B.V., The Netherlands 2
Rick F. van der Lans
Rick F. van der Lans is an independent consultant, lecturer, and author. He specializes in data warehousing, business
intelligence, database technology, and data virtualization. He is managing director of R20/Consultancy B.V.. Rick has been
involved in various projects in which data warehousing, and integration technology was applied.
Rick van der Lans is an internationally acclaimed lecturer. He has lectured world wide professionally for the last twenty
five years. He has been invited by several major software vendors to present keynote speeches.
He is the author of several books on computing, including his new Data Virtualization for Business Intelligence Systems.
Some of these books are available in different languages. Books such as the popular Introduction to SQL is available in
English, Dutch, Italian, Chinese, and German and is sold world wide. He also authored The SQL Guide to Ingres and SQL for
MySQL Developers.
Ambassador of Kadenza: Rick works closely together with the consultants of Kadenza in many projects. Kadenza is a
Dutch consultancy company specializing in business intelligence, data management, big data, data warehousing, data
virtualization, and analytics. Our joint experiences and insights are shared in seminars, webinars, blogs, and white papers.
Affiliate to SimplicityBI: SimplicityBI and Rick have independently promoted the use of data virtualization technology for
years. To support the market better, they have decided to work more closely together. In the role of affiliate, Rick
presents seminars and webinars, writes blogs for the SimplicityBI website, and assists the SimplicityBI specialists.
R20/Consultancy B.V. is located in The Hague, The Netherlands, www.r20.nl. You can get in touch with Rick via:
Email: rick@r20.nl
Twitter: @Rick_vanderlans
LinkedIn: http://www.linkedin.com/pub/rick-van-der-lans/9/207/223
- 4. Copyright © 2018 R20/Consultancy B.V., The Netherlands 4
Data hasn’t changed,
it’s just more of the same
- 5. Copyright © 2018 R20/Consultancy B.V., The Netherlands 5
Data usage has changed
Self-service BI
Embedded BI
Supplier- and Customer-driven BI
Applied AI in Text, Image, Video Analysis
Edge Analytics
Data Marketplace
Data Science
Automated decisions
…
- 6. Copyright © 2018 R20/Consultancy B.V., The Netherlands 6
Data for the Happy Few Only
- 7. Copyright © 2018 R20/Consultancy B.V., The Netherlands 7
Business Intelligence Has Come a Long Way
- 8. Copyright © 2018 R20/Consultancy B.V., The Netherlands 8
Specifications
Source
system
s
Analytics & reporting
From Data to Dashboards
Data structure specifications
Integration specifications
Transformation specifications
Data security specifications
Data cleansing specifications
Analytical specifications
Visualization specifications
Data privacy specifications
- 9. Copyright © 2018 R20/Consultancy B.V., The Netherlands 9
Source
system
s
Analytics & reporting
The Implementation (on Powerpoint)
Data structure specifications
Integration specifications
Transformation specifications
Data security specifications
Data cleansing specifications
Analytical specifications
Visualization specifications
Data privacy specifications
Data Warehouse
- 10. Copyright © 2018 R20/Consultancy B.V., The Netherlands 10
Source
system
s
Analytics & reporting
The Implementation (in Real Life)
Data structure specifications
Integration specifications
Transformation specifications
Data security specifications
Data cleansing specifications
Analytical specifications
Visualization specifications
Data privacy specifications
Data
Warehouse
Data
MartsStaging Area
- 11. Copyright © 2018 R20/Consultancy B.V., The Netherlands 11
The Data Ware House Architecture
Is Like a Rigid Assembly Line
- 12. Copyright © 2018 R20/Consultancy B.V., The Netherlands 12
ETL ETLETL
Source
system
s
Data martsStaging
area
Analytics &
reporting
Data
warehouse
Metadata Specifications Everywhere
Data structure specifications
Integration specifications
Transformation specifications
Data cleansing specifications
Analytical specifications
Visualization specifications
- 13. Copyright © 2018 R20/Consultancy B.V., The Netherlands 13
Yesterday: Data Warehouse and Data Usage
Developers
IT specialists
Development Styles
Pre-programmed, auditable,
governable, formally tested
Report Types
Batch and online business
reports
Consumers
Business users
Legislators
- 14. Copyright © 2018 R20/Consultancy B.V., The Netherlands 14
Today & Tomorrow: Data Warehouse and Data Usage
Developers
IT specialists
Business Users
Development Styles
Pre-programmed, auditable,
governable, formally tested
Self-service, investigative
Pre-programmed
Self-service, investigative
Report Types
Batch and online business
reports
Customer-facing apps
Ad-hoc reports
Simple data retrieval
Ad-hoc reports
Data mining, statistics
Dark data analysis
Consumers
Business users
Legislators
External parties
Consumers
Business users
Business users
Business users
Data scientists
Business users and IT
Streaming analytics Business users, machines
- 15. Copyright © 2018 R20/Consultancy B.V., The Netherlands 15
Data Virtualization to the Rescue
- 16. Copyright © 2018 R20/Consultancy B.V., The Netherlands 16
Data Virtualization Overview
production
application website
analytics
& reporting
mobile
App
internal
portal dashboard
Data Virtualization Server
SQL
databases
streaming
databases
social
media data
Hadoop,
NoSQL
databaseESB
messaging
unstructured
datalegacy
database
cloud
applications
private
data
applications
- 18. Copyright © 2018 R20/Consultancy B.V., The Netherlands 18
DataVirtualizationServer
Virtual table pointing to source
Virtual table:
May contain row selections, column selections,
column concatenations, transformations,
column and table name changes, groupings,
aggregations, data cleansing, …
Data consumer
Developing Virtual Tables
Source
- 19. Copyright © 2018 R20/Consultancy B.V., The Netherlands 19
Layers of Virtual Tables
Enterprise data layer
Data consumption
layer
Data source
layer
DataVirtualizationServer
- 20. Copyright © 2018 R20/Consultancy B.V., The Netherlands 20
Caching to Mimimize Access of Data Stores
Virtual table
with cache
Virtual table
without cache
Data source Data source
- 21. Copyright © 2018 R20/Consultancy B.V., The Netherlands 21
The Data Delivery Platform with Data Virtualization
Data sources
ETL ETL Cached Cached
Data Delivery Platform – Data Virtualization
- 22. Copyright © 2018 R20/Consultancy B.V., The Netherlands 22
The Logical Data Warehouse Architecture
ETLETL
Source
system
s
Staging
area
Analytics &
reporting
Data
warehouse
Social
media data
Open data
Spreadsheets
Logical Data Warehouse Architecture
Big data
DataVirtualizationserver
- 23. Copyright © 2018 R20/Consultancy B.V., The Netherlands 23
The Logical DWA is Metadata Driven
ETLETL
Source
system
s
Staging
area
Analytics &
reporting
Data
warehouse
Social
media data
Open data
Spreadsheets
Logical Data Warehouse Architecture
DataVirtualizationserver
Repository
- 24. Copyright © 2018 R20/Consultancy B.V., The Netherlands 24
Use Cases Physical versus Logical Data Warehouses
Physical Data Warehouse:
• standard reporting
• internal data
• sources with no history
• IT-dominated development
Logical Data Warehouse:
• self-service BI and data science
• internal and external data
• systems with history
• Includes physical data warehouse
• speedy development
• operational reports
• new data storage technology
• IT & Business combined
development
- 25. Copyright © 2018 R20/Consultancy B.V., The Netherlands 25
Use Cases of Data Virtualization
• Logical data warehouse architecture
• Logical data lake
• “Servicing” existing applications for external use
• E.g., developing REST interfaces on source systems
• Managed self-service BI
• Democratizing data
• Making data from any kind of source available for every user
• BYOBIT: Bring Your Own BI tool
• Sharing of meta data specifications by data virtualization server
• 360 degree view of customers
• Cloud integration
• And many more …
- 26. Copyright © 2018 R20/Consultancy B.V., The Netherlands 26
Summary
• Organizations want to become more data-driven
• Data usage is changing
• They have to unlock all their data
• The traditional data warehouse is too restrictive
• Data virtualization is mature and agile integration
technology
• It’s all about abstraction
• Data virtualization is the preferred technology for
developing a logical data warehouse