Watch full webinar here: https://bit.ly/3RXRi3m
Vincent Fages-Gouyou, EMEA Product Management Director, détaille les nouveautés produits de Denodo Platform.
Lors de cette session, vous découvrirez :
- d’où vient Denodo et pourquoi Denodo porte cette vision auprès des organisations modernes
- quels sont les challenges Data des départements IT et des métiers
- quelles ont été les réponses apportées et pourquoi elles ont été insuffisantes
- quelle est la vision de Denodo pour résoudre enfin rapidement les problématiques data
Lunch and Learn ANZ: Shaping the Role of a Data Lake in a Modern Data Fabric ...
Denodo 2022 : le meilleur time-to-Data du marché
1. Vision & Orientation Stratégique
Futures Evolutions
N o u v e a u t é s D e n o d o 2 0 2 2
Vincent Fages-Gouyou,
Directeur Produit, EMEA
Toward Denodo 9
3. 4
Data Quality Data Science ML / AI Visualisation
Enterprise’s Data Delivery Architecture
Locations
Data Sources
OLAP
4. 5
Data Quality Data Science ML / AI Visualisation
Enterprise’s Data Delivery Architecture
Locations
Data Sources
OLAP
Governance, Metadata Management, Data Mart
Security
Data Access
Data Virtualization Data Services
5. 6
Enterprise’s Data Delivery Architecture
Governance, Metadata Management, Data Mart
Security
Data Access
Data Virtualization Data Services
Federation
Transformation
Abstraction
Data Service Dynamic Query
Optimization
Cost Based
Optimizer
Query
Rewriting
Caching MPP
Security &
Governance
Lifecycle
Management
Data Catalog
Discover
Collaborate
Query
Categorize
6. 8
The Data Citizen Personas
Application-to-Application
Data Developer
Administration &
Operations
Business User
& BI Analyst
Data Scientist
Denodo Proprietary and Confidential
7. 45
Want big impact?
Use big image
Denodo Proprietary and Confidential
9
One Logical Platform for:
Data Integration,
Management & Delivery
Denodo Proprietary and Confidential
8. 10
INTEGRATE
Disparate data in
any location, format
or latency
Files
Cubes
Cloud
Stores
Traditional
DB & DW
Data Lake &
NoSQL
Hybrid/
Multi-Cloud
Security &
Governance
Al/ML
Recommendations
Advanced
Semantics
MANAGE
Related data with a universal
semantic model and AI / ML
functionality enabling vital
data governance and query
acceleration through MPP
Query
Optimization &
Acceleration
Data Catalog
Discover / Explore /
Document
BI Tools
SQL / MDX
Data Science
Tools
Data as a Service
RESTful / Odata
GraphQL/ GeoJSON
DELIVER
And democratize data using
BI & data science tools,
data catalogs, and APIs
Denodo Platform: ONE Logical Platform for All Your Data
Source: Forrester Total Economic ImpactTM
of Data Virtualization, 2021
83% reduction
Faster Time-to-revenue
67% reduction
Reduced shadow IT
65% decrease
Delivery times over ETL
Denodo Proprietary and Confidential
9. 11
Denodo 8 Empowers Data Delivery
Application-to-Application
Denodo Developer
Administration &
Operations
Business User
& BI Analyst
Data Scientist
Denodo Proprietary and Confidential
10. 12
A Modern Data Platform Architecture
CONNECTIVITY
Query
Optimization
Security
AI/ML Governance
Semantic
Layer
DATA OPPS
Deployment
Cloud PaaS
Containers/K8
On-Prem
Monitoring
Scheduling
Version Control
DEVELOPMENT
MODELING
DELIVERY
SOLUTION MANAGER
Real Time
Smart Query
Acceleration
Caching
MPP Engine
LOGICAL
DATA
FABRIC
SOURCES
150+
data
adapters
Apps Streaming SaaS
Files
OLAP
Hadoop
& NoSQL
Cloud
Stores
Traditional
DB & DW
DATA CATALOG
Discover - Explore - Document
{ API ACCESS }
RESTful / OData
GraphQL / GeoJSON
SQL
MDX Access
Denodo Cubes
CONSUMERS
Denodo Proprietary and Confidential
11. 13
3rd Party
Connectors
Data Science / Prep Tools
Technology - Ecosystem
MDM & DQ Tools
Data Governance Tools
Cloud Service
Providers
Big Data/NoSQL Sources
HDInsight
BI & Reporting Tools
Data Modeling Tools
Security & Privacy
Traditional Data Sources
SaaS/Cloud Applications
Cloud Data Warehouses
API Management / ESB
Denodo Proprietary and Confidential
13. 16
Semantic Modeling, Bridging IT & Business
Integration & Optimization
Business Views & Security
Application Layer & Interfaces
Data Services
Connectivity
Denodo Proprietary and Confidential
14. 17
Semantic Modeling, Bridging IT & Business
Integration & Optimization
Business Views & Security
Application Layer & Interfaces
Data Services
Connectivity
Data
infrastructure
Data Developer Administration &
Operations
Domains
Application-to-Application
Business User
& BI Analyst Data Scientist
Denodo Proprietary and Confidential
15. 18
Traditional Data Virtualization
Examples:
• BI & analytics
• Ad-hock queries
• Realtime query
acceleration
• SQL enable any sources
• RDBMS, Data Lake
• API, noSQL
• Data source
protection
• Hybrid / multi-cloud
• Datawarehouse
offloading
• Join Internal /
External data
On-demand live execution
Cache
&
Acceleration
Denodo Proprietary and Confidential
16. 19
Integrated ETL / ELT Support: Remote Tables
Examples:
• Data Lake management
• Load data in lake
when needed
• Materialize data in
different zones
• Data Science
• Move data to Spark
after initial analysis for
model creation and
training
• Cloud Migrations
• Replicate and refresh
data to cloud system
… and from data science notebooks
SCH
Often triggered from
Scheduler …
Create table in any location
Load with data from any other data source
Denodo Proprietary and Confidential
17. 20
Streaming
Examples:
• Event enrichments
• Real time context to
events
• IoT Data
• Access streamed
and historical data
Instead of a request-response, execution is
triggered asynchronously by an incoming message.
Denodo acts as a listener to a queue
A query is executed using the message data
as input values (e.g. filter by device ID)
Results can be placed back in a queue
Or persisted in any target system
Denodo Proprietary and Confidential
18. 21
Data as a Service Provider
Transactional Services
API Gateway
Other
Enterprise
Applications
Denodo exposes the semantic data layer
through APIs with zero coding
• It includes security controls to show
different data depending on the user/role
• You can add complex workload
management policies, including quotas
(e.g. 100 queries/hour)
Denodo supports a wide range of protocols
and options
• GraphQL
• GeoJSON (geospatial APIs)
• OData 4
• OAuth 2.0, SAML and SPNEGO
authentication
• OpenAPI (pka Swagger) documentation
Direct
Denodo Proprietary and Confidential
19. 22
Data Governance with Denodo Platform
Integrate
Data Sources
& Semantic modeling
Secure
Advanced Security Policy
& Governance
Self Service
Data Market Place
Automatic Documentation
Execute
Optimization / MPP
Query Acceleration
VQL REST GraphQL OpenAPI
SQL
Design Studio
Low Code / No Code
Tag Based Policies &
RBAC / ABAC
Data Science
Notebook & Python
BI / Analytics Self-Service Data Catalog
API Data Services
Operational EDW Data Lakes
Files
SaaS APIs
Data Developer
Administration &
Operations
API
Business User
& BI Analyst
Data Scientist
Denodo Proprietary and Confidential
20. 23
Executing a Decentralized Data Mesh’s Organization
SQL
Operational EDW
Data Lakes Files
SaaS APIs
REST GraphQL OData
Event
Product
Customer Location Employee
Customer Management Event Management Human Resources
Domains create
virtual models in
separate schemas.
Execution servers
can be scaled
independently
1
Domains can
choose and
autonomously
evolve their data
sources
2
Domains can share standardized
definitions. Products can be used
to define other products
3
A central team can
set guidelines and
enforce global
security, quality and
governance policies
in the virtual models
4
Denodo Proprietary and Confidential
21. 24
1. Continued evolution of Denodo’s core components to
adapt to new sources/consumers and requirements of
modern Dev/Ops strategies
2. Enrich functionality of the Data Catalog
3. Extended support for multiple data integration methods
beyond the data virtualization foundation
4. Integrate MPP processing engine
5. Embrace SaaS and automated deployment models
6. Integrate AI engine for automation across all components
7. Add Semantic capabilities for Security and Governance
Completing the Vision
Denodo Proprietary and Confidential
23. 26
Security – Tag-based policies in Denodo
§ Denodo’s tags, available for tables and
columns
§ Definition of semantic, tag-based security
policies
§ Completely abstracted from specific tables
§ Easier to manage and less error prone
§ E.g mask the #SSN with *** for HR and
Finance
§ Allows for implementation of semantic security
rules across the data landscape, independent
of technologies underneath
Denodo Proprietary and Confidential
24. 27
Denodo’s Role & Tag Based Data Access Protection
API / TSL
Protected Source
Protected Source
N
1
N
2
JDBC
REST/JSON
KMIP / TSL
Finance
HR
K1 K2
Open Source
Java Client Library
ABE Crypto
Engine
Confidential Data
Intelligence Platform
Secure Enclave
KMS K1
K2
N
1
N
1
N
2
Attributes
N
2
N
1
Custom Function & Policy
• Get User Key uid
• Build JSON Policy
• Build JSON Access Policy
• Request Master Key
• Request User Key
• Encrypt with Attributes
• Decrypt
Denodo Proprietary and Confidential
25. 28
Connectivity - Native Kafka consumer/producer
Kafka
subscriber
123
Client Id: 123
Order Price: 1500$
Kafka
producer
SELECT * FROM Client, contract WHERE id = 123 …
{
“clientId”: “123”,
“customerName”: “Vincent”
“contract”: “Premium”,
“email”: “vincent@mail.com”,
“cashback” : “15$”
}
CRM database MDM
Denodo Proprietary and Confidential
26. 29
Embedded MPP Execution Engine
Denodo embeds its own MPP engine, based on Presto:
§ Easy, efficient access to data lake content
(Parquet file in S3/ADLS)
§ No need for additional engine
§ Integrated security and management
§ Out-of-the-box MPP options for caching and
acceleration capabilities
§ Simplified management of “data offloading” use
cases
§ Efficient integrated store for large volumes of
active metadata / query history to enable
upcoming AI capabilities
Logical Layer
Traditional
DB & DW Cloud Excel
Distributed FS
Data Lakes
MPP Engine
Denodo Proprietary and Confidential
27. 30
Embedded MPP execution engine
Denodo Proprietary and Confidential
Object Store configuration Object Store browsing
28. 31
Denodo + MPP Refence Architecture
Denodo
Virtualization
Server
Denodo
Data Catalog
Denodo
Web Services
On-prem
data
Snowflake
Warehouse A
Warehouse B Data Science
(e.g. Snowpark)
Other Apps
IdP
Denodo
MPP
Denodo Proprietary and Confidential
29. 33
Denodo 8 Empowers Data Mesh
Application-to-Application
Data Developer
Administration &
Operations
Business User
& BI Analyst
Data Scientist
Denodo Proprietary and Confidential
30. 34
Logical Data Fabrics in the Data Mesh
API
Business User
& BI Analyst
Data Scientist
Denodo Proprietary and Confidential
31. 35
Key Takeaways
Data Centralization does not drive
Data Centric Enterprises
The Data Mesh based on Logical Data Fabric
play a crucial role in current data management
topics like advanced analytics, data science,
API strategy and move to the cloud.
Logical Data Fabric
Empowers Data Centric Initiatives!
Denodo Proprietary and Confidential