1. This Presentation Courtesy of the
International SOA Symposium
October 7-8, 2008 Amsterdam Arena
www.soasymposium.com
info@soasymposium.com
Founding Sponsors
Platinum Sponsors
Gold Sponsors Silver Sponsors
SOA Symposium
October 7-8, 2008, Amsterdam
ESB Reference Architecture and
Product Selection Guidelines
André Tost
Senior Technical Staff Member, SOA Technology
IBM Software Services for WebSphere
© 2008 IBM Corporation
1

2. SOA Symposium
Agenda
 Introduction
 Definition / Reference Architecture
 Patterns
 Federation
 Product Selection Criteria
 Sample Scenarios
3 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Introduction
4 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
2

3. SOA Symposium
SOA Addresses Many Perspectives
Business Services SOA Entry Points Infrastructure &
Management
in Support of SOA
Service
Management
Service
Security
Business Process
Management Service
Virtualization
SOA Governance and
Service Lifecycle Management
5 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
SOA Entry Points Help You Get Started
Both Business Centric and IT Focused
What is it? Value
Improved productivity and flexibility by enabling targeted user
Deliver role-based interaction and collaboration interactions for improved business operations and collaboration
People through services
Achieve business process innovation through Greater innovation and flexibility
treating tasks as modular services through faster deployment and
Process modification of business processes
Provide trusted information in business context Better business operations, more informed decisions and reduced
by treating it as a service risk with information delivered in-line and in-context
Information
Service-enable existing assets and fill portfolio Lower risk and faster time to market
gaps with new reusable services by leveraging proven, time-tested functionality
Reuse
Connect systems, users, and business channels Reduced maintenance costs and
Connectivity based on open standards greater reliability and consistency
through flexible, any-to-any linkages
6 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
3

4. SOA Symposium
What are some of today's business challenges that SOA
Connectivity can address?
How do you…?
 Ensure seamless flow of information from
anywhere at anytime using anything
 Execute broad business processes that
span your company and business
partners?
 Build trusted relationships with your
partners?
 Scale your business to grow smoothly?
 Deliver a consistent user experience
regardless of channel or device?
7 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
How do you connect the assets and services that support your
business process?
Connectivity needs
Enable "any-to-any" linkage and communication inside and
beyond your company
Simplify connectivity by ensuring secure, reliable, and
scaleable pipeline of information
8 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
4

5. SOA Symposium
Business Value… …Through SOA Connectivity
…by exposing the same process
Deliver a secure, consistent user
through new business channels and
experience…
devices
Strengthen trading partner …with managed Service-based
relationships… connections
… over custom-built or traditional
Potential savings of 2X-4X…
connectivity*
Minimize complexity in your
… through any-to-every connectivity
organization…
…through scaleable access to
Increased business flexibility and
information in the right format and at
responsiveness…
the right time
…by extending IT assets rather than
Eliminate redundancy…
duplicating them
Decreased development and … through decoupling of function
maintenance costs… and interface
*Software Strategies
“Enterprise Integration Challenge” 2005
9 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Three examples of SOA connectivity
Deliver a process
across multiple
business channels
Connect to
Connect external service
business providers
systems
1. Internal connectivity based on open standards
2. Deliver an existing process through new business channels
3. Securely connect to external 3rd parties and trading partners
10 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
5

6. SOA Symposium
ESB Definition
11 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Core Principles of the ESB Architectural Pattern
Service Service
Requestor
ESB Provider
 ESB inter-connects requester and provider
– Interactions are decoupled
– Supports key SOA principle – separation of concerns
 ESB provides Service Virtualization of
– Identity via routing
– Interaction via conversion
– Interface via transformation
 ESB also enables Aspect Oriented Connectivity
– Security
– Management
– Logging
– Auditing
– …
12 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
6

7. SOA Symposium
An ESB-centric view of the Logical Model
Business Services
IT Management Services
Security
Interaction Process Information
Development Services
ESB Registry
Management
Business Services
Partner Business App Access
 Outside ESB  Tightly coupled to ESB
– Business Logic (Business Services) – Service Registry
– ESB does contain integration logic or connectivity logic – Registry is a Policy Decision Point for ESB
– Criteria: semantics versus syntax; aspects – ESB is a Policy Enforcement Point for Registry
– But, Registry has a broader scope in SOA
 Loosely coupled to ESB
– Security and Management
 Tooling required for ESB
– Policy Decision Point outside the ESB – Development
– ESB can be Policy Enforcement Point
– Administration
– Configuring ESB via Service Registry
13 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Expanded view of the ESB
Enterprise Service Bus
Mediation Flows Mediation Patterns
Message Models Meta models
Communication Protocols Interaction Patterns
 Communication Protocols
– Supply basic connectivity to requesters and providers
– Impact QoS (e.g., reliable delivery, transactions)
– Supply inherent Interaction Patterns (e.g., request/reply, one-way, pub/sub)
 An ESB leverages underlying communication fabrics of SOA infrastructure
– ESB provides on-ramps and off-ramps
 Typical requirements
 Standards are important – HTTP (SOAP/HTTP, XML/HTTP)
– MQ (SOAP/JMS/MQ, XML/MQ, text/MQ, …)
– Adapters (legacy, EIS)
– WS-I, WS-Security
– RAMP
14 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
7

8. SOA Symposium
Expanded view of the ESB
Enterprise Service Bus
Mediation Flows Mediation Patterns
Message Models Meta models
Communication Protocols Interaction Patterns
 Message Models
– Describe messages exchanged with requesters and providers
– Based on Meta-models
– Fundamental means of describing messages  Typical requirements
– For example, XML Schema language – XML schema definition
– Content models define specific messages – Industry specific content models
– For example, XML schema
 An ESB supports one or more message meta-models
 An ESB supports multiple message content models
– Can include industry standard models as well as enterprise specific models
– Can include weakly-typed models
15 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Expanded view of the ESB
Enterprise Service Bus
Mediation Flows Mediation Patterns
Message Models Meta models
Communication Protocols Interaction Patterns
 Mediation Flows
– Process messages exchanged between requester and provider via ESB
– Large grained
– Moderately reusable
– Constructed from Mediation Patterns
– Mediation Patterns define processing “steps” of a mediation flow
– Small to middle grained
 Typical requirements
– Highly reusable
– Dynamic routing
– ESB products include pre-built “mediation primitives”
– Logging
16 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
8

9. SOA Symposium
ESB Mediation Flows and Mediation Patterns
 Mediation enables Service Virtualization of
– Identity via routing
– Using basic mediation patterns (context, content, contract)
– Using composed mediation patterns (retry, failover, Distribution/aggregation, …)
– Dynamic, driven by metadata in registry
– Impact aspects of QoS (e.g., SLA, failover)
– Interaction (of protocol and pattern) via conversion
– Protocol conversion inherent with support for more than one transport protocol
– Impact aspects of QoS (e.g., reliable delivery, transactions)
– Interface via transformation
– Using specific mediation patterns
– Using adapters
– Impact aspects of QoS (e.g., performance)
– NOTE: other forms of mediation should be agnostic to interface (weakly-typed processing)
 Mediation enables Aspect Oriented Connectivity
– Security & Management
– Logging
– Auditing
– …
 Provided by a mediation framework
– Offering pre-built mediation pattern (primitives) support
– Enabling mediation pattern composition
17 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Mediation Patterns - Examples
Request / Request / Multi Event
Response Response Propagation
Protocol
Switch Transform Enrich Route Distribute Monitor Correlate
+ + +
Canonical Transform – Log - Route Gateway
Adapter
18 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
9

10. SOA Symposium
Adapters for Existing Systems Integration
 An ESB must allow access to existing Enterprise Information
Systems
 Adapters typically used, and may or may not be part of the ESB
– Technology
– Application
– Legacy
 The following are the placement options for adapters, based on
domain where adapter configuration managed:
– Outside of the ESB, and inside the EIS domain
– Outside the ESB, and the EIS domain
– On the boundary of the ESB
– Inside the ESB
19 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
ESB Patterns
20 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
10

11. SOA Symposium
ESB Patterns
Global ESB
 All services in one namespace
 All services have visibility to each other
 Centrally Administered
 Used by departments or small enterprises (or for development & test environments)
Requester Provider
ESB
Requester Provider
21 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
ESB Patterns
ESB Gateway
 Controlled and Secure service interactions at internal or external domain boundaries
Requester Provider
ESB ESB ESB
Requester Provider
22 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
11

12. SOA Symposium
ESB Patterns
Brokered ESB
 Bridge services that selectively expose requesters or providers
 Each ESB manages its own namespace.
 Service interactions between ESBs facilitated through a common broker that
implements the bridge services.
 Used by departments that develop and manage their own services, but share a few of
them
Provider
Requester ES ES ES
B B B Provider
Requester
ES Provider
B
Provider
23 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
ESB Patterns
Federated ESB
 One master ESB with several dependent ESBs
 Service requestors and providers connect to master ESB or one of the
dependent ESBs
 Used to federate a set of autonomous organizations
Requester ES
Requester B
Provider
ESB
Provider
Provider
ESB
Provider
Provider
ESB
Provider
24 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
12

13. SOA Symposium
ESB Federation
25 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Observation
 Federation is really about service visibility
– One domain
– Several domains
– All domains
 A service registry defines service visibility “policy”
– In the desired domain(s)
– With the desired monitoring and management
– With the desired security
 A service bus simply enforces “policy”
 We use this to analyze approaches to federation
26 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
13

14. SOA Symposium
Some analysis – Service Virtualization
1 3
Service requester … implemented
invokes a service… by Service Bus
mediation…
2 4
… which is a … delegating to the
Virtual service … Actual service
Pattern:
Service Virtualization
27 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Service Virtualization with Service Registry
Virtual Service Actual Service
Implemented by the virtual service
Service Bus delegates to
A‟ A
Advertise
3.a Use Registry content to
virtual services
0
implement smart mediations
Metadata about services
A‟ A in Registry for use by
Service Bus
Services in Registry
advertised for use by Pattern: Dynamic
Requesters Service Virtualization
28 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
14

15. SOA Symposium
Federated Service Virtualization
Virtual Virtual Service Actual Service
Service the virtual service the virtual service
delegates to delegates to
A‟‟ A‟ A
A‟‟ A‟ A‟ A
Pattern: Recursive
Service Virtualization
29 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Federated ESB interaction patterns
Domain
D Remoted ESB Y
virtualization
C C‟ D‟
Remote D
virtualization C
B B‟ Recursive B‟‟ B
virtualization A
A A‟
Local
Domain virtualization
ESB X
C D‟ A‟
B D
A B‟‟
Patterns:
B‟ C‟
Private in X
“Federated”
Private in Y Public in Y
Public in X
30 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
15

16. SOA Symposium
Federated ESB interaction patterns
 In reactive situations, with “black box” service buses, addition of a „backbone‟ may be
necessary
“Introspection” of existing service
registries required to populate
„global‟ service registry
31 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Some analysis – Service Management
Virtual
B
Service
A Observes
endpoints
Uses Endpoint Offers Interaction
2
Status Information 1 Status Information
A B Endpoint Status
Service Monitor &
Manager
Service Registry Change & Configuration
Metadata Management DB
32
Federation
ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
16

17. SOA Symposium
Some analysis – Service Security
Virtual Virtual Service Actual Service
Service the virtual service the virtual service
delegates to delegates to
A A‟‟ A‟ A
Identity Identity
Identity
Mapping
33 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Some analysis – Service Security
 In reactive situations, with “black box” service buses, addition of a „backbone‟ may
be necessary
Identity Identity
Identity
Mapping
34 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
17

18. SOA Symposium
ESB Product
Selection Criteria
35 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Key Criteria for ESB selection
 Communication Protocols & Interaction Patterns
– Critical (e.g., MQ, SOAP/HTTP)
– Accommodated
– “Associated” standards (e.g., WS-Security)
– APIs (e.g., JMS)
– Adapters
 Message Models
– Meta-models (e.g., XML)
– Content models (e.g., ACORD), including Weak-typing
– “Associated standards” (e.g., SOAP headers)
36 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
18

19. SOA Symposium
Key Criteria for ESB selection (cont.)
 Mediation Flows and Mediation Patterns
– Conversion
– Transformation engine(s)
– Routing
– Security and Management integration
– Logging and auditing integration
– Registry integration
– Breadth of pre-built mediation primitives
– Custom mediation capability and Programming model
– Weakly-typed processing
37 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Key Criteria for ESB selection (cont.)
 Qualities of service
– Heterogeneous transaction coordination
– Reliable/assured delivery
– Performance
– Scalability
– Reliability
 Non-functional
– Affinity to SOA environment (e.g., WebSphere Process Server)
– Affinity to IT environment (e.g. J2EE application server)
– Development tooling capabilities and affinity to current tools
– Configuration and administration tooling capabilities
– Existing and required skill set (e.g., J2EE skills)
– Ease of integration with management (e.g., monitoring, security) environment
– Product maturity and comfort level with leading edge products
– Price and total cost of ownership
38 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
19

20. SOA Symposium
Sample ESB
Scenarios
39 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Case 1 – Connectivity between web service requester and an
EIS system
Tooling Domain of interest -
Intranet
SAP Adapter
EIS
Backend
Web Services
requester ESB
Registry
40 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
20

21. SOA Symposium
Case 1 Description
 Customer environment
– This customer is leading adopter of technology. Comfortable with sophisticated solutions
– WebSphere Application Server customer
 Business Requirements
– The customer wants to provide web service access to functionality in an Enterprise Information
System such as SAP R/3, PeopleSoft, or Oracle Financials
– Intranet environment doesn‟t require complex security considerations
– The integration is based on message exchange/data replication scenarios - there is no business
process or data synchronization between clients and EIS systems
– Support moderate volume of requests
 Technical Requirements
– The targeted integration is one-to-one, i.e., one ESB virtual service maps to one EIS application,
although multiple EISs can be exposed as web services at the same time
– Data transformation should use XSLT; development tooling important
– Log the messages as they flow through the hub – asynchronously to a file
 Architecture Decisions
– J2EE environment preferred
– Use available adapter product to simplify development
41 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Case 1 Analysis
 Infrastructure Pattern
– Service virtualization
– Domain isolation unimportant
 ESB Product selection
– Transport Protocols
– JCA adapter supported
– SOAP/HTTP
– Synchronous Request/response sufficient
– Message models
– XML metamodel desired
– Content model supported
– Mediation flows
– XLST transformation and supporting tooling
– QoS and Non-functional requirements
– J2EE foundation (WebSphere Application Server)
– XSLT transformation skills available
– Adequate throughput and/or response time
– Leading edge adopter
42 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
21

22. SOA Symposium
Case 2: Connectivity between heterogeneous requesters and
providers (Multi-protocol Exchange)
Tooling
Domain of interest -
Intranet
Registry
Portal SOAP/JMS SOAP/JMS J2EE provider
SOAP/HTTP SOAP/HTTP
.NET Client .NET provider
XML/HTTP XML/HTTP
Some Client ESB Some provider
XML/MQ COBOL Copybook/MQ
XML/MQ Client CICS
Text/MQ
Text/MQ Client
43 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Case 2 Description
 Customer environment
– Customer is a leading adopter of technology. Comfortable with sophisticated solutions
 Business Requirements
– Any provider must be accessible via multiple heterogeneous requestors
– Support moderate volume of requests
– Intranet environment does not require complex security considerations
– Global transactions across multiple heterogeneous transaction managers
 Technical Requirements
– ESB must support
– Communication protocol conversion, but not adapters
– Flexible data model conversion, with acceptable performance and adequate tooling
– Enterprise class persistent messaging backbone
 Architecture Decisions
– Canonical data model(s) used in ESB
– Consumers and providers must adapt to the service definition supported by the ESB
44 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
22

23. SOA Symposium
Case 2 Analysis
 Infrastructure Pattern
– Significant service virtualization
 ESB Product selection
– Transport Protocols
– Multiple protocols
– Synchronous and asynchronous Request/response
– MQ provides enterprise class persistent messaging backbone
– Message models
– Multiple built-in message metamodels
– Content models supported
– Mediation patterns
– Significant protocol conversion capability
– Significant transformation capability with supporting tooling
– Supports advanced correlation of asynchronous request/response
– Extensive runtime configuration options
– QoS and Non-functional requirements
– Global transactions across multiple heterogeneous transaction managers
– Mature product which supports throughput and/or response time
– Sophisticated transformation skills available
45 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Example of Federated ESB (1)
Retail POS/Store
Store Applications
Point of
Sale
Terminal
Lightweight
commercial
Point of Java-based MQ
Sale ESB
Terminal Headquarters
MQ
MQ Mainframe
Full-featured DBMS
Retail POS/Store
ESB with
Store Applications
richer
mediation & SOAP/
connectivity HTTP TP
Point of MQ Monitor
Sale SOAP/
Terminal HTTP MQ JMS
Lightweight MQ
commercial Enterprise J2EE Server
Point of Java-based Applications
Sale ESB
Terminal
46 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
23

24. SOA Symposium
Example of Federated ESB (2)
Enterprise Secure Zone
ESB 1
Subscribers Publishers
(distribute
MQ MQ
events)
MQ
Demilitarized
Zone
SOAP J2EE
Partner SOA Appliance
ESB 2 Web services
SOAP/ HTTP SOAP/ HTTP
(Dynamically
Route and log Web services
Web Web services) for CICS
Application
Registry Federated IM
47 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Example of Federated ESB (3)
External
Services/Consumer Red – service requestors/providers
s Blue – part of the ESB WAS
Legacy Applicatio
SOAP/HTTP/ Application n
WS-Security
BPEL Process
FTP/Files Process Server
Legacy BPEL Process
.NET XML/MQ
Application Application Process Server
SOAP/ SCA
JMS Binding
Sockets MQ
DataPower Departmental Departmental
Device Adapter PM4Data
Service Bus Service Bus
Message Broker WebSphere ESB
SOAP/HTTP MQ
MQ MQ MQ
Backbone Service Bus
MQ and Message Broker MQ
Partner
Services Registry/Repository Gateway
EDI / AS2 /
SOAP/ SOAP/HTTP
WAS XML/HTTP XML/MQ HTTP
Applicatio
Business
Business
n .NET Departmental Business
Application Service Partner
Application Partner
Partners
48 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
24

25. SOA Symposium
Summary
 ESB provides Service Virtualization
 ESB also enables Aspect Oriented Connectivity
– in support of an SOA Connectivity Entry Point
 ESB is an architectural pattern
– various types
– federated ESBs become more wide-spread
 We can identify a number of federated ESB patterns
 The ESB ref arch provides the basis for product selection
49 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
SOA Symposium
Thai
Traditional Chinese
Russian
Gracias Spanish
Thank English
You Merci
French
Obrigado
Brazilian Portuguese
Arabic
Grazie Danke
Italian German
Simplified Chinese
Japanese
50
50 ESB Reference Architecture and Product Selection Guidelines © 2008 IBM Corporation
25