Oracle Data Integration CON9737 at OpenWorld

Oracle Data
Integration
OVERVIEW | VISION | ROADMAP
Jeff Pollock
Vice President, Product Management
October 2015
Oracle Open World 2015Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Oracle Open World 2015 2

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 4
Oracle Data Integration Core Business Use Cases
Analytics & Business Intelligence
• Big Data Integration & Governance
• Data Warehouse / Mart Integration
IT Data Integration & Governance
• Enterprise Shared/Common Data Services
• Move, Migrate, Transform, Profile, Cleanse,
Glossary and Metadata Management
Business Continuity
• Active-Active for Maximum Availability
• Zero Downtime Migrations
24 x 7 x 365
Cloud
On Premise

NoETL Engine
100% Native Data
Transformation
Data
Integrator
Big Data
Preparation
GoldenGate
Data
Quality
Data Service
Integrator
Metadata
Management
Non-invasive CDC,
Realtime streaming
data delivery
Profile, Cleanse,
Match, and
Remediate Data
Prepare, Secure,
Enrich and Publish
Unstructured Data
Catalog, Trace and
View Models across
the Enterprise
Federate Data
Across DBs, Services
and Applications

Oracle is Different from Other Data Integration Vendors
BETTER WITH OPEN SOURCE
MODERN DATA ARCHITECTURE
MORE GOVERNANCE COVERAGE
HYBRID DEPLOYMENT
90+
…tools supported by
Business Glossary
and Metadata
Management
15more than
different certified open
source technologies
Zero…additional
hardware for
data replication
or transformation Public | Managed | Private
GoldenGate
Data
Integrator
Data Quality
Data
Preparation

Our Goal: Lead by Innovation with Open Source
Data Preparation:
• Product: Oracle Big Data Preparation
• The Market: Red-hot subcategory of Big Data
• Unique About Oracle: Semantics!
Data Ingestion:
• Product: Oracle GoldenGate for Big Data
• The Market: Core aspect of Big Data market
• Unique About Oracle: Streaming from DBMS!
Data Transformation:
• Product: Oracle Data Integrator Advanced Big Data
• The Market: ETL Offload in Big Data
• Unique About Oracle: 100% Native & Decoupled!
Metadata Governance:
• Product: Oracle Enterprise Metadata Management
• The Market: Data Governance
• Unique About Oracle: 70+ harvesters for 3rd party!
1st
1st
1st
1st
to move beyond Machine Learning
and to embrace Semantics (NLP &
Graph) for Data Preparation
to deliver DBMS transactions in
realtime to Big Data Streaming
platforms like Flume and Kafka
to support 100% native Spark based
ETL using a graphical, logically de-
coupled design studio
to include wide range of Big Data
metadata harvesting fully supported
as part of 70+ technology bridges

Changing Concepts About Modern Data Management
Polyglot
Fit for Purpose Data
Lambda
Speed Layer
Batch Layer
Data
Sources
Data
Services
Dataflow
Data
Services
Data PipelineData
Sources

Staging
Modern Platform for Next Generation Solutions
Sqoop
HDFS
Hive
Flume
Data StreamingKafka (MPP Pub/Sub)
Storm and Trident
Spark Streaming
HBase
Unstructured and
Semi-Structured Data
Structured Data
Lambda/Kappa Speed Layer
Lambda Batch Layer
Apps
Analytics
& DWs
Oracle Data Governance Foundation
Oracle Data Integrator
Oracle Enterprise Data Quality
Oracle Enterprise Metadata Management with Business Glossary
Oracle GoldenGate
Oracle Data Preparation

Oracle Data Governance Foundation
Staging
100% Open and Heterogeneous Solution
Sqoop
HDFS
Hive
Flume
ETL Offload / Transformation
Data StreamingKafka (MPP Pub/Sub)
Storm and Trident
Spark Streaming
HBase
Discovery Sandbox/s
R
UnstructuredStructured Oracle Data Integrator
Oracle Enterprise Metadata Management with Business Glossary
Oracle GoldenGate
Oracle Data Preparation

BIG DATA PREPARATION

Oracle Big Data Preparation for Data Without Schema
Big Data Preparation Cloud Service
Data Preparation
 Designed for data domain
experts, not programmers
 Focused on cleansing, enriching
and transforming unstructured
business data
 Operationalize data flows into
ETL or Business Intelligence
Key Benefits
 Easy to get started with
browser based application
 Better recommendations
engine combines machine
learning with semantic
technologies
 Integrated into Oracle Cloud

Oracle Big Data Preparation Core Capabilities
Publish
• On Demand or
Scheduled
• Source / Target
Definition
• Restful APIs
• Event Driven
Govern and Monitor
• Dashboards
• Automated Alerts
• Reusable user policies
• System Controls
• Security
• Stats & Metrics
Ingest
• Import & Ingest
• Detect Schema
• Cleanse, Normalize
& De-duplicate
• Detect & Mask
Sensitive Data
Enrich
• Profile
• Annotate
• Data Classification
• Semantic
Enrichments
• Missing Data
Interpolation
• Unified solution to prepare
unstructured data
• Simple to use tooling designed
for non-programmers
• Unique technology approach
combines Machine Learning
(ML) with Natural Language
Processing (NLP) engine
• Powered by Apache Spark,
Hadoop, and UIMA
• Cloud operated from the Oracle
Public Cloud

Load Data for Oracle Business Intelligence Cloud Service
Easy Publishing to Oracle BICS
 Pre-integrated RESTful service
connectivity
 Shared single sign on access
across Oracle Cloud
 Common operational support
from OPC services
Key Benefits
 Self Service access from non
technical users
 Cloud based applications can
bypass need for extensive IT
support for on premise tech
 Accelerated Value by enabling
business users to quickly ingest
data for operational BI
BDP-CS BICS

Highly Differentiated from Other Data Prep Tools
Better Recommendations Engine
 Only Data Prep/Wrangling tool
to combing Natural Language
Processing (Apache NLP) with
Machine Learning (Spark ML)
 Leverages Linked Open Data
graph of domain knowledge
Key Benefits
 More Efficient Mapping by
leveraging a more effective
recommendation service
 Higher Quality automation
“gets it right” more often
 Data Enrichment leveraging
domain data for enriching
sparse data sets
Hadoop
Oracle
Cloud
Semantics
based
Knowledge
Graph
Spark
Machine
Learning
Natural
Language
Processing

GOLDENGATE

Announcing: Oracle GoldenGate New Capabilities
Zero Downtime Upgrades
& Data Migration
High Availability
& Disaster Recovery
Application and
Report Offloading
Big Data, Data Warehouse
and Data Marts
Message Bus,
SOA/JMS, Coherence
Global Data Synchronization
Across the Enterprise
Real-time Analytics
& Data Streaming
New Capabilities
 GG Studio for easy design
 Metadata in Trail File
 Delivery to Kafka
 Enhanced Monitoring w/Lag
 Performance Improvements
 MySQL 5.7. SQL Server SE, ALO
Key Benefits
 Reliability most proven CDC on
the planet – don’t lose data!
 Performance when low latency
matters most – GG is the best
 Breadth of coverage on all your
targets, Oracle, Non-Oracle, Big
Data and NoSQL
Capture
Trail
Route
Deliver
Pump
GoldenGate

Streaming Big Data
Batch Big Data
NoSQL
Capture
Trail
Route
Deliver
Pump
NEW: Oracle GoldenGate for Big Data
GoldenGate
GoldenGate for Big Data
 Same trusted architecture used
by 1000’s of customers
 Data delivery to Big Data
 Lambda/Kappa streaming data
Key Benefits
 Less invasive on Source DBs
when compared to Batch
Processing such as Sqoop or
ETL processes
 Simple ingestion for 1:1 data
architecture for populating
“raw data” zones
 Real-time data delivery for
streaming analytics/apps
 Reliable, proven at scale

Database as a Service
Hadoop as a Service
3rd Party Cloud
Capture
Trail
Route
Deliver
Pump
NEW: Oracle GoldenGate Cloud Service
GoldenGate Cloud Service
 Available in the Public Cloud via
Subscription or Hourly basis
Key Benefits
 Oracle Database Cloud Service
delivery via GoldenGate
 Exadata Cloud Service delivery
via GoldenGate
 Big Data Cloud Service delivery
to Hadoop and NoSQL
3rd Party Cloud
 More Choices run as BYOL on
other Clouds for delivery to any
supported Database
On Premise
Database
Cloud Service
Exadata
Cloud Service
Big Data
Cloud Service
GGCS
GGCS
GoldenGate

DATA INTEGRATOR

Proprietary ETL
engines die circa
2015 – folded into
big data
Proprietary ETL is Dead. Apache-based ETL is What’s Next
Scripted
SQL
Stored
Procs
ODI for
Columnar
ODI for
In-Mem
ODI for
Exadata
ODI for
Hive
ODI for
Pig & Oozie
1990’s
Eon of Scripts and PL-SQL Era of SQL E-LT/Pushdown Big Data ETL in Batch Streaming ETL
Period of Proprietary Batch ETL Engines
Informatica
Ascential/IBM
Ab Initio
Acta/SAP
SyncSort
1994
ODI for
Spark
ODI for
Spark Streaming
Warehouse
Builder

Announcing: Oracle Data Integrator New Capabilities
On Any DB/DWOn Exadata On Big Data
0
50
100
150
200
250
300
350
42X
FASTER
ETL
INFA/IBM
 Best ETL for any
Exadata App
 Run ODI on the
Exadata Host
 ODI Optimized
for SmartScan
 Proven w/1000’s
of customers
 Native SQL for
3rd Party DB’s
 Faster ETL and
lower TCO
 First to market
with Spark
 Reuse maps
across languages
 Executable runs
anywhere
New Capabilities
 Lifecycle Management updates
to core versioning systems
 Simplify setup and deployment
of Big Data transformations
 Performance improvements
across all E-LT platforms
Key Benefits
 Open Source friendly – run E-LT
on Apache Spark/Hadoop
 Best Performance – take
processing to the data
 No Hardware required – run on
the DB host or on Apache big
data cluster

Oracle Data Integrator Advanced Big Data Edition
SQOOP
OGG
SQOOP
+ Native
Loaders
API/File
Flume/Kafka
OLH/OSCH
Big Data SQL
Data Integrator for Big Data
 Batch data ingestion with
Sqoop or native loaders
 Generate data transformations
in Hive, Pig or Spark
 Extract data into external DBs,
Files or Cloud
Key Benefits
 NoETL Engine designed from
scratch for E-LT execution
 Non-invasive does not require
any install on cluster
 Decoupled mappings allow for
portable runtime execution,
saving development time, and
simplifying debugging

Oracle Data Integrator for Cloud
Data Integrator for the Cloud
 Deploy ODI anywhere and
execute ETL on the data
Key Benefits
 Integrated with OPC
 Flexible Deployment
architecture that can run on-
premise, Java Cloud Service or
Big Data Cloud Service
 Broad Connectvity for sourcing
data from any Apps or DBs
3rd Party Cloud
 Flexible run ODI with BYOL
on other clouds
On Premise
ODI
SaaS Apps
ODI/EDQ
Java
Cloud Service
ODI
Big Data
Cloud Service
Database
Cloud Service
Exadata
Cloud Service
Storage
Cloud Service
Business Intelligence
Cloud Service

Oracle Data Integrator Call for Streaming ETL Beta
Contact Us Now!
 Beta is open to existing ODI
customers, try out Spark
Streaming ETL with right now!
Key Benefits
1. Shorter batch windows
• Processing before staging
(vs after)
2. Multi-use from the engine
• In line analytics on the
stream (not possible w/older
ETL engines)
• Data pipeline for Log data
ingest (not possible at scale
w/older ETL engines)
EDWData Transformation
Batch Process
EDW
Micro Batch
Batch Engine
HDFS
vs. Batch ETL
Streaming ETL with Spark Streaming
Events
Analysis
Staging
Staging
Data Transformation
BI
BI

METADATA MANAGEMENT

Announcing: Oracle Metadata Management 12.2.1
1. Business Glossary – simple tools to catalog, link and
collaborate on business terms
2. Metadata Management – horizontal and semantic
data lineage for all big data sources
Business/Metadata Catalog
 Report to Source Lineage
 Impact Analysis
 Audit, Versioning & Diff Reports
 Social/Collaboration Features
 Annotations and Tagging
 Automatic Stitching
Comprehensive Harvesting
 3rd Party BI Metadata
 3rd Party ETL Metadata
 3rd Party DB Metadata
 3rd Party Modeling Tools
 Hadoop & NoSQL Metadata
 Metadata Standards

Data Lineage in Three Crucial Varieties
Vertical Lineage
 For the Data Stewards
 Links a business friendly set of
terms to the IT metadata and
operational assets
 Links Business Glossary, Taxonomy,
Ontology, Conceptual Models to
Technical Metadata (Columns, Files,
Objects etc)
Horizontal Column Level
 For the Architects
 Links the data fields from Business
Intelligence Dashboards or Reports
back to the Source DB/App
Columns
 Schemas, BI View Layers, ETL
Transformations, Calculations, etc.
Business Lineage
 For the Business Users
 Links a business friendly set of
Glossary terms to the BI reporting
models (eg; Semantic Models,
Universe, etc.)
 Links Business Glossary, Taxonomy,
Ontology, Conceptual Models to
Reporting Fields
BIZ
TERM
BIZ
TERM
REPORT
FIELD
DB
COLS REPORT
FIELD
SOURCE
COLS

Open Source, Standards and Third Party Integrations
 Adaptive
 Altova
 Apache HCatalog
 Apache Hive/HQL
 Borland
 CA ERwin
 Cloudera Impala
 COBOL Copybook
 DataStax
 Embarcadero
 EMC ProActivity
 GentleWare
 Google BigQuery
 Grandite
 Hadapt Hive
 Hortonworks Hive
 IBM Cognos
 IBM DB2
 IBM DataStage
 IBM Discovery
 IBM Federation Server
 IBM Lotus Notes
 IBM Netezza
 IBM Rational Rose
 IBM Rational Architect
 Informatica Metadata Manager
 Informatica PowerCenter
 CoSORT
 ISO SQL Standard (DDL)
 MapR Hadoop Hive
 MicroFocus
 Microsoft Access
 Microsoft Office Excel
 Microsoft Visio
 Microsoft SQL Server
 Microsoft SSIS
 Microsoft Visual Studio
 Microstrategy
 Magic Draw
 OMG CWM Standard
 OMG UML Standard
 Oracle BI Answers
 Oracle BI Enterprise Edition
 Oracle BI Server
 Oracle DAC
 Oracle Data Integrator
 Oracle Data Modeler
 Oracle Database
 Oracle Designer
 Oracle Hyperion Applications
 Oracle Hyperion Essbase
 Oracle Warehouse Builder
 Pivotal Greenplum
 PostgreSQL
 QlikView
 SAP BO Crystal Reports
 SAP BO Designer
 SAP BO Desktop Intelligence
 SAP BO Repository
 SAP BO Data Integrator
 SAP BO Data Steward
 SAP Master Data Management
 SAP Sybase PowerDesigner
 SAP Sybase ASE Database
 SAS Data Integration Studio
 SAS BI Server
 SAS Information Map
 SAS Metadata Management
 SAS OLAP Server
 Select
 Sparx Architect
 Syncsort
 Tableau
 Talend
 Teradata
 Tigris
 Visible
 W3C DTD & XSD Schema
Metadata Harvesting (Glossary, Lineage & Impact Analysis)Key Standards

DATA QUALITY

Announcing: Oracle Enterprise Data Quality 12.2.1
New Capabilities
 Enhanced High Availabilty
 WebLogic Clustering
 DB Failover Support
 Improved Matching Flexibility
 Scoring for Matching
 Easier Definitions
 Hadoop/Hive Certification
 Audit Framework Support
 REST Web Services
Key Benefits
 Simpler tools to very efficiently
create match rules
 Accuracy of scoring models and
easier access to parameters
Fusion DQ (EDQ)
EDQ 12c
Profile Audit Transform Match Parse
WebLogic 12c
DaaS DQ (EDQ)
EDQ

Pragmatic Operational Governance Focus
Profile and Cleanse Data Flows
Quality KPIs Case
Management
Governance Cockpit for Data Stewards & Stakeholders
Exception
Review
Metadata
Management
Business
Glossary
Prepare & Enrich
Data Integration

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 33
Data Quality Core Strategy is Two-Pronged
2. BEST FOR ORACLE RED-STACK1. BEST OF BREED DATA GOVERNANCE
1. EDQ for Oracle MDM –
Customer Hubs
2. EDQ for Oracle MDM –
Product Hubs
3. EDQ for Oracle Fusion
Applications
4. EDQ for Oracle SaaS
Applications
5. EDQ for Oracle DaaS
Services
6. EDQ for Oracle Database
7. EDQ for Oracle Data
Integration & Governance
Oracle Open World 2015

DATA FEDERATION

Industry Broadest Data Federation Capabilities
Oracle Big Data
Spatial and Graph
B B B
Oracle Data Service Integrator Oracle Big Data SQL
DBAsApp Dev
Transactional
Data Federation
Analytic
Data Federation
From a single
Oracle SELECT
statement…
getObject()
Web Services
/ XQuery

VISION & VALUE

Data
Integrator
Big Data
Preparation
GoldenGate
Data
Quality
Data Service
Integrator
Metadata
Management

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 39
DataflowML – A Data Integration Fabric with Smarts
Come see me at Oracle
OpenWorld for more
information about this
one!!

Our Vision: ALL Capabilities on the Cloud
Data Integration
SaaS Data
Quality
DaaS Data
Quality
ODI on Java
Cloud Service
EDQ on Java
Cloud Service
MM on Java
Cloud Service*
Big Data
Preparation
GoldenGate
Cloud Service
Big Data
Cloud Service
GG on Big Data
Cloud Service*
ODI on Big Data
Cloud Service
* Roadmap

Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

Presen-
tations
on:
Oracle Confidential 42
Data Integration Solutions Program - tinyurl.com/DISOOW15
Demo
Stations:
Hands-
on labs:
Oracle
Enterprise
Metadata
Management
Oracle
Enterprise
Data Quality
Oracle
GoldenGate
Oracle
Data
Integrator
Oracle
Big Data
Preparation
Cloud Service
Oracle
Enterprise
Data Quality
HOL10464
Oracle
GoldenGate
Deep Dive
HOL10460
ODI and OGG
For Big Data
HOL10463
Big Data
Preparation
Cloud Service
HOL10467
Oracle Data
Integrator
Deep Dive
HOL10462
Middleware
Demoground
- Moscone South
Oracle Cloud Platform and
Infrastructure Showcase
- Moscone West
Big Data
Showcase
- Moscone South
Database
Demoground
- Moscone South

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 43
Data Integration Solutions Program - tinyurl.com/DISOOW15
Monday, Oct 26
• Oracle Data Integration Solutions—Platform Overview and Roadmap
[CON9737]
• Oracle GoldenGate Product Update and Strategy [CON9720]
• How Oracle GoldenGate Helped Increase Data Availability for
Financial Close by 600% at Humana [CON4398]
• Into the Cloud and Back with Oracle Data Integrator 12c [CON3506]
Tuesday, Oct 27
• Oracle Data Integrator Product Update and Strategy [CON9601]
• Oracle GoldenGate 12.2 New Features Deep Dive [CON8737]
• The Role of Data Governance in Data Integration [CON9736]
• Oracle Fusion Middleware: Meet This Year’s Most Impressive
Innovators [CON10374]
• The Oracle Enterprise Data Quality Platform—Product Overview and
Roadmap [CON9587]
• Enabling Real-Time Data Integration with Big Data [CON9724]
• Solving the “Dirty Secret” of Big Data with Oracle Big Data
Preparation Cloud Service [CON9615]
Wednesday, Oct 28
• Oracle Data Integration Solutions: the Foundation for Cloud Integration
[CON9614]
• Oracle Data Integration Product Family: a Cornerstone for Big Data
[CON9609]
• The Essential Core of Data Governance with Oracle Enterprise Data
Quality [CON9593]
• Accelerate Cloud Onboarding Using Oracle GoldenGate Cloud Service
[CON9717]
• Introducing Oracle GoldenGate Studio for Faster Design and
Deployment [CON9713]
Thursday, Oct 29
• Integrating Big Data: Balancing the Data Warehouse and the Data
Reservoir [CON8768]
• Zero-Downtime Upgrading Using Oracle GoldenGate [CON7892]
• Best Practices for Migrating to Oracle Data Integrator [CON9604]
• Oracle GoldenGate Performance Tuning and Monitoring [CON8736]
• Cloud Data Quality: Lessons Learned from Oracle’s Journey to the
Sales Cloud [CON9595]
• Oracle Enterprise Metadata Management and the Cloud [CON9612]
• Developer Best Practices for Oracle Data Integrator Lifecycle
Management [CON9611]
• Oracle Management Pack for Oracle GoldenGate Update [CON9716]

Connect with Oracle Data Integration
@OracleDI
Blogs.oracle.com/DataIntegration/
Oracle Data Integration
Oracle Data Integration

Oracle Data Integration CON9737 at OpenWorld

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (13)

Similaire à Oracle Data Integration CON9737 at OpenWorld

Similaire à Oracle Data Integration CON9737 at OpenWorld (20)

Plus de Jeffrey T. Pollock

Plus de Jeffrey T. Pollock (9)

Dernier

Dernier (20)

Oracle Data Integration CON9737 at OpenWorld