Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
SOA Summit 2014
1. Better Business Results with
Open Source Integration
Kenneth Peeples
JBoss Technology Evangelist
2. Agenda for SOA Summit
●
Overview of Open Source Integration with Data
Virtualization
●
●
2
Preview of JBoss Middleware Integration Offerings
Integration examples that will help you get started
RED HAT JBOSS MIDDLEWARE
4. Red Hat is…
“By running tests and executing numerous examples for specific teams, we were able to prove […] not
only would the solution work, but it will perform better & at a fraction of the costs.”
MICHAEL BLAKE, Director, Systems & Architecture
4
RED HAT JBOSS MIDDLEWARE
5. Better Business Results through Open Source Integration
Overview of Open Source Integration
with Data Virtualization
6. SOAs and abstraction
Decoupling is the key concept
in SOAs achieved through
abstraction based on
service interfaces
Business processes and
services make use of
business information
Business information is
exposed to business
services as data services
6
RED HAT JBOSS MIDDLEWARE
7. Bridging the Gap
The need to bridge data gaps in a more straightforward, streamlined
and scalable way is due to:
●
●
Growing need for consolidated views of information
●
7
More and more data
Drive toward interoperability and standards
RED HAT JBOSS MIDDLEWARE
8. Data Services through Data Virtualization
Key requirements of data services
●
●
Expose integrated information in one or more desired formats,
even if the original data are in different formats
Expose information through one or more consistent, standard
interfaces, even if the original data are accessed through
different interfaces
The requirements are achieved by
●
●
8
Modeling to define the required format of data, integrated from the
underlying sources
A query engine for processing these abstract definitions efficiently,
exposing the integrated information through one or more
interfaces
RED HAT JBOSS MIDDLEWARE
9. Data Virtualization essential to a SOA
Data Virtualization provide the necessary interface to data for all
business services. Data Virtualization:
●
●
●
Expose all data through a single uniform interface
Provide a single point of access to all business services in the
system
Expose data using the same paradigm as business services - as
"data services"
●
●
Provide a uniform means of exposing/accessing meta-data
●
Provide a searchable interface to data and meta-data
●
Expose data relationships and semantics
●
9
Expose legacy data sources as data services
Provide uniform access controls to information
RED HAT JBOSS MIDDLEWARE
10. Data Virtualization and SOA Goals
●
●
●
●
●
10
Create an enterprise data foundation for streamlined development,
better integration and more efficient data governance
Improve delivery of consistent, complete information from any data
source to the application and business processes
Externalize business knowledge buried in applications and
databases
Expand automation, moving beyond reports to active analysis and
decision-making
Build business processes in code that are actually followed by the
organization
RED HAT JBOSS MIDDLEWARE
11. What is Data Virtualization?
Data Virtualization provides a service that:
●
Encapsulates data-oriented logic
●
Enable loose coupling between the application and the data stores
●
Manages real-time connectivity to data stores
●
●
●
11
Offers a higher level of data abstraction compared to interacting
directly with data stores
Provides appropriate views of data
Allows views of data to be cached based on performance
requirements
RED HAT JBOSS MIDDLEWARE
12. Translators and Resource Adaptors
A translator
●
●
●
provides an abstraction layer between the run time engine and
physical datasource
Knows how to issue query commands into source specific
commands
Execute them using the resource adapter
A Translator also defines the capabilities of a particular source, like
whether it can natively support query joins (inner joins, cross joins
etc) or support criteria.
A Translator along with its Resource Adapter must be configured on a
Source Model
12
RED HAT JBOSS MIDDLEWARE
13. Translators and Resource Adaptors (Continued)
A Resource Adapter provides the connectivity to the physical data
source
This also provides way to natively issue commands to the source and
gather results
Translator capabilities define what processing each translator/source
combination can perform.
For example, most relational sources can process joins and unions,
whereas when processing delimited text files these operations
cannot be performed by the resource adapter or the "source" (in this
case, the file system).
13
RED HAT JBOSS MIDDLEWARE
14. Modeling Environment for Data Virtualization
Virtual Databases contain two primary varieties of model types - source and
view.
Source models represent the structure and characteristics of physical data
sources, whereas view models represent the structure and characteristics
of abstract structures you want to expose to your applications.
Models used for data integration are packaged into a virtual database (VDB).
The models must be in a complete and consistent state when used for data
integration.
That is, the VDB must contain all the models and all resources they depend
upon.
Source models must be configured with a Translator and a Resource Adapter
with them before a VDB is tested in Designer or deployed for data access
14
RED HAT JBOSS MIDDLEWARE
15. Runtime Environment for Data Virtualization
●
●
Optimize performance
●
Manage data security
●
Take advantage of run-time system management features
●
15
Connect to multiple, heterogeneous data sources
Audit data service usage
RED HAT JBOSS MIDDLEWARE
16. Better Business Results through Open Source Integration
Preview of JBoss Middleware
Integration Offerings
17. Innovate faster, in a smarter way
A family of a lightweight, enterprise-grade products
that are ideal for open hybrid cloud environments.
17
RED HAT JBOSS MIDDLEWARE
18. There is a better way – Red Hat JBoss
Integration Product Line
Red Hat JBoss A-MQ
● A small-footprint, high-performance messaging platform
●
●
Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/amq/
Developers Download the Code - https://www.jboss.org/products/amq.html
Red Hat JBoss Fuse
● A small-footprint, flexible enterprise service bus (ESB)
●
●
Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/fuse/
Developers Download the Code - https://www.jboss.org/products/fuse.html
Red Hat JBoss Fuse Service Works
● A service design, development and integration platform that enables
organizations to transition to an open hybrid cloud
●
●
Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/fuse-service-works/
Developers Download the Code - https://www.jboss.org/products/fsw.html
Red Hat JBoss Data Virtualization (Only Open Source Data Virtualization
product on the market)
● A lean, virtual data integration that unlocks trapped data and delivers it as
easily consumable, unified, actionable information
●
●
18
Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/data-virtualization/
Developers Download the Code - https://www.jboss.org/products/datavirt.html
RED HAT JBOSS MIDDLEWARE
21. What does Data Virtualization software do?
Turn Fragmented Data into Actionable Information
Data Virtualization software virtually
unifies data spread across various
disparate sources; and makes it
available to applications as a single
consolidated data source.
DATA CONSUMERS
BI Reports
The data virtualization software
implements 3 steps process to bridge
data sources and data consumers:
●
●
●
21
Connect: Fast access to data from
diverse data sources
Compose: Easily create unified
virtual data models and views by
combining and transforming data
from multiple sources.
Consume: Expose consistent
information to data consumers in
the right form thru standard data
access methods.
SOA Applications
Easy,
Real-time
Information
Access
Virtual Consolidated Data Source
•
•
•
Data Virtualization Software
Consume
Compose
Connect
Oracle DW
SAP
Hadoop
DATA SOURCES
RED HAT JBOSS MIDDLEWARE
Salesforce.com
Virtualize
Abstract
Federate
Siloed &
Complex
22. Turn Fragmented Data into Actionable Information
Mobile Applications
ESB, ETL
BI Reports & Analytics
SOA Applications & Portals
DATA
CONSUMERS
JBOSS
DV
Design Tools
Standard based Data Provisioning
JDBC, ODBC, SOAP, REST, OData
Consume
Dashboard
Unified Virtual Database / Common Data Model
Compose
Unified Customer
View
Unified
Product View
Easy,
Real-time
Information
Access
Unified
Supplier View
Optimization
Caching
Virtualize
Abstract
Federate
Security
Connect
Native Data Connectivity
Metadata
DATA
SOURCES
Hadoop
22
NoSQL
Cloud Apps
Data Warehouse
& Databases
Mainframe
RED HAT JBOSS MIDDLEWARE
XML, CSV
& Excel Files
Siloed &
Complex
Enterprise Apps
23. JBoss Data Virtualization:
Supported Data Sources
Enterprise RDBMS:
• Oracle
• IBM DB2
• Microsoft SQL Server
• Sybase ASE
• MySQL
• PostgreSQL
• Ingres
Enterprise EDW:
• Teradata
• Netezza
• Greenplum
23
Hadoop:
• Apache
• HortonWorks
• Cloudera
• More coming…
Office Productivity:
• Microsoft Excel
• Microsoft Access
• Google Spreadsheets
Specialty Data
Sources:
• ModeShape
Repository
• Mondrian
• MetaMatrix
• LDAP
RED HAT JBOSS MIDDLEWARE
NoSQL:
• JBoss Data Grid
• MongoDB
• More coming…
Enterprise & Cloud
Applications:
• Salesforce.com
• SAP
Technology
Connectors:
• Flat Files, XML Files,
XML over HTTP
• SOAP Web Services
• REST Web Services
• OData Services
24. Key New Features and Capabilities for v6
●
Data connectivity enhancements
–
–
NoSQL (MongoDB – Tech Preview) and JBoss Data Grid
–
●
Hadoop Integration (Hive – Big Data),
Odata support (SAP integration)
Developer Productivity improvements
–
–
Enhanced column level security,
–
●
New VDB Designer 8 and integration with JBoss Developer Studio v7
VDB import/reuse, and native queries
Simplify deployment and packaging
–
–
●
Requires JBoss EAP only; included with subscription
Remove dependency with SOA Platform
Business Dashboard
–
24
New rapid data reporting/visualization capability
RED HAT JBOSS MIDDLEWARE
25. ●
JBoss Data Virtualization – Use Cases
Self-Service
Business
Intelligence
The virtual, reusable data model provides business-friendly representation of data,
allowing the user to interact with their data without having to know the complexities of their
database or where the data is stored and allowing multiple BI tools to acquire data from
centralized data layer. Gain better insights from Big Data using JBoss Data Virtualization to
integrate with existing information sources.
360◦
Unified
View
Deliver a complete view of master & transactional data in real-time. The virtual data layer
serves as a unified, enterprise-wide view of business information that improves users’ ability
to understand and leverage enterprise data.
Agile SOA
Data
Services
A data virtualization layer deliver the missing data services layer to SOA applications. JBoss
Data Virtualization increases agility and loose coupling with virtual data stores without the
need to touch underlying sources and creation of data services that encapsulate the data
access logic and allowing multiple business service to acquire data from centralized data
layer.
Regulatory
Compliance
Data Virtualization layer deliver the data firewall functionality. JBoss Data Virtualization
improves data quality via centralized access control, robust security infrastructure and
reduction in physical copies of data thus reducing risk. Furthermore, the metadata
repository catalogs enterprise data locations and the relationships between the data in
various data stores, enabling transparency and visibility.
25
RED HAT JBOSS MIDDLEWARE
26. There is a better way – Red Hat JBoss
Integration Product Line
JBoss A-MQ
A small-footprint messaging platform, eliminating batch
delivery
“JBoss A-MQ for real-time messaging”
Additional Capabilities
JBoss Fuse
(Includes JBoss AMQ)
A Integration/ESB platform for integrations that can extend
outside the data center
“JBoss Fuse for integration/ESB requirements”
Additional Capabilities
JBoss Fuse Service
Works
(Includes JBoss Fuse)
26
A Service design, development and integration platform that
enables responsive and innovative enterprise
“JBoss Fuse Service Works for
integration/ESB+SOA+Orchestration+Governance
requirements”
RED HAT JBOSS MIDDLEWARE
27. JBoss A-MQ v6.1
New in 6.1:
●
●
Full support of AMQP 1.0 – provides wire-level compatibility across connections
Improved high availability (HA) – embedded message store for shared-nothing
HA
27
RED HAT JBOSS MIDDLEWARE
28. JBoss Fuse v6.1
New in 6.1
• Full support of AMQP 1.0 – provides wire-level compatibility across connections
• Vast library of connectors – over 200 out-of-the-box-connector via Apache Camel
• Managed integration routes - start, stop, measure and trace Camel routes on-premise or in
the Cloud
• Improved high availability (HA) – embedded message store for shared-nothing HA
28
RED HAT JBOSS MIDDLEWARE
29. JBoss Fuse Service Works v6
Enables organizations to Simplify integrations and focus on business innovation
29
RED HAT JBOSS MIDDLEWARE
30. SwitchYard Application Overview
Composite Service
ComponentBinding
Composite
Reference-Binding
Service-Binding
Component-Service
Component
30
Component-Reference
RED HAT JBOSS MIDDLEWARE
31. Design Time and Run Time Governance
31
RED HAT JBOSS MIDDLEWARE
32. Better Business Results through Open Source Integration
Integration examples to help you
get started
33. Sample 1: Big Data with
Data Virtualization
●
Determine if sentiment data from the
first week of the Iron Man 3 movie is a
predictor of sales
Problem:
–
●
Cannot utilize social data and
sentiment analysis with sales
management system
Consume
Compose
Connect
Solution:
–
JBoss Data Virtualization
Leverage JBoss Data Virtualization to
mashup Sentiment analysis data with
ticket and merchandise sales data on
MySQL into a single view of the data.
https://github.com/DataVirtualizationByExample/Mashup-with-Hive-and-MySQL
33
Consumer 2 : DV
Dashboard to analyze
the aggregated data
Objective:
–
●
Consumer 1: Excel
Powerview to analyze
the aggregated data
Hive
SOURCE 1: Hive/Hadoop
contains twitter data
including sentiment
RED HAT JBOSS MIDDLEWARE
SOURCE 2: MySQL data
that includes ticket and
merchandise sales
46. References
Data Virtualization Datasheet http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-data-virtualization-datashe
et
Fuse Service Works Datasheet
http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-fuse-service-works-datash
eet
Red Hat JBoss Data Virtualization for financial services Whitepaper
http://www.redhat.com/resourcelibrary/whitepapers/data-virtualization-fsi
Red Hat JBoss Fuse Service Works technology overview Whitepaper
http://www.redhat.com/resourcelibrary/whitepapers/red-hat-jboss-fuse-service-works-tech
nology-overview
Turn data into actionable information with Red Hat JBoss Middleware Whitepaper
http://www.redhat.com/resourcelibrary/whitepapers/turn-data-into-actionable-information-w
ith-red-hat-jboss-middleware
46
RED HAT JBOSS MIDDLEWARE
