In this presentation will talk about how one of the world's leading Financial Institutions, leveraged WebSphere DataPower to provide a set of centralized consumer profile management services. This central service would be leveraged by internal and external applications, and would align with enterprise marketing capabilities. The solution included a complex security model which included the following products: Tivoli Directory Server, Tivoli Access Manager and Tivoli Federated Identity Manager. We will describe how to build complex orchestrations in WebSphere DataPower, and also go through some of the performance tuning options we implemented to achieve a high degree of efficiency.
Nell’iperspazio con Rocket: il Framework Web di Rust!
Creating a Centralized Consumer Profile Management Service with WebSphere DataPower
1. Creating a Common Consumer Profile
Management Service with WebSphere
DataPower
Speakers:
Brian Backer - Business Leader, Enterprise Architecture(MasterCard)
Bin Tang – Consultant, Enterprise Architecture(MasterCard)
Prithvi Srinivasan – SOA/Integration Practice Director(Prolifics)
Date: May 1st 2013
2. Agenda
• Introductions
• Business Case for a Common Consumer Profile Service
• Architecture Overview
– IT Landscape before the CCRS Implementation
– CCRS and the Future-state IT Landscape
• CCRS Implementation – Developing Complex Composite Services
– Logical Architecture
– Service Orchestration Overview
• WebSphere DataPower Performance Testing and Capacity Planning
– Approach
– Test Cases & Results
• WebSphere DataPower Tuning and Optimization Options
2
3. Agenda
• Introductions
• Business Case for a Common Consumer Profile Service
• Architecture Overview
– IT Landscape before the CCRS Implementation
– CCRS and the Future-state IT Landscape
• CCRS Implementation – Developing Complex Composite Services
– Logical Architecture
– Service Orchestration Overview
• WebSphere DataPower Performance Testing and Capacity Planning
– Approach
– Test Cases & Results
• WebSphere DataPower Tuning and Optimization Options
3
4. Common Consumer Registration Service (CCRS)
Page 4
• Centralized enterprise
capabilities for managing
consumer users
• Benefits
– Centralized management of
consumer identities
– Single identity across
multiple applications
– Security compliance
– Eliminate redundant
solutions
Overview / Problem Statement
CCRS represents the next evolution of shared services. Orchestration of a common business
process from the ESB.
5. Agenda
• Introductions
• Business Case for a Common Consumer Profile Service
• Architecture Overview
– IT Landscape before the CCRS Implementation
– CCRS and the Future-state IT Landscape
• CCRS Implementation – Developing Complex Composite Services
– Logical Architecture
– Service Orchestration Overview
• WebSphere DataPower Performance Testing and Capacity Planning
– Approach
– Test Cases & Results
• WebSphere DataPower Tuning and Optimization Options
5
7. • Point to Point connection to credential management
service
– Inconsistent implementation
• Localized User Profile Information
– Consumer profile information spread across
multiple databases
• Lead generation activities submitted via batch file
– Lead generation profiles not matched with login
profiles
Landscape Prior to CCRS on the ESB
Page 7
9. • Common service to manage authentication credentials and
user profile information
• Common user profile repository leveraging master data
management platform
• Service extensible for supporting traditional and single sign-
on models
• Service accessible from external gateway for Lead
Generation and Pre-Registration activities
– Provides common consumer profile across lead gen and
login models
• Leverages ESB security and monitoring models
CCRS on the ESB Landscape
Page 9
10. Agenda
• Introductions
• Business Case for a Common Consumer Profile Service
• Architecture Overview
– IT Landscape before the CCRS Implementation
– CCRS and the Future-state IT Landscape
• CCRS Implementation – Developing Complex Composite Services
– Logical Architecture
– Service Orchestration Overview
• WebSphere DataPower Performance Testing and Capacity Planning
– Approach
– Test Cases & Results
• WebSphere DataPower Tuning and Optimization Options
10
14. Agenda
• Introductions
• Business Case for a Common Consumer Profile Service
• Architecture Overview
– IT Landscape before the CCRS Implementation
– CCRS and the Future-state IT Landscape
• CCRS Implementation – Developing Complex Composite Services
– Logical Architecture
– Service Orchestration Overview
• WebSphere DataPower Performance Testing and Capacity Planning
– Approach
– Test Cases & Results
• WebSphere DataPower Tuning and Optimization Options
14
15. Performance and Load Testing Approach
Testing should focus on observing the appliance under
• Normal anticipated traffic load
• Anticipated traffic spikes
• And finally complete saturation (un-anticipated traffic spikes)
Appliance statistics should be studied including
• Number of connections
• CPU/load behavior
• Consumed memory
• Response times
• Interactional throughput at saturation
15
16. What to Monitor ?
• XSLT
– XSLTs is probably the most common reason for low performance. It is where
complex business transformation logic is written, for instance the url-open
extension function.
• Services
– Service objects like WS-Proxy/MPGW/XMLFW…
• Message Flows : The message flows are categorized in 4 types:
– Message: The time taken to process the request message received from
client, plus processing time by the server, plus time taken to process the
server response by the device (The full transaction cycle).
– Request: The time taken by DataPower to process the request message
before sending it to the server.
– Server: The time taken by the server to process the request message sent
to it by WebSphere DataPower.
– Response: The time taken by WebSphere DataPower to process the
message received from the server before sending it back to the client.
16
17. What to Monitor ? (Contd.)
• Connections
– You can get a snapshot of inbound, outbound and internal TCP
connections, you can also watch services listening on specific ports.
• Device Utilization
– It is also beneficial to watch the device metrics (CPU, Memory and
System Usage) under different situations. This is usually done via
periodic SNMP polling in most environments.
**Monitoring System Load/Usage provides a better metric of appliance
load than CPU usage.
• System Usage
• CPU Usage
• Memory Usage
• Transaction Rate
• Receive and Transmit Throughput17
20. Agenda
• Introductions
• Business Case for a Common Consumer Profile Service
• Architecture Overview
– IT Landscape before the CCRS Implementation
– CCRS and the Future-state IT Landscape
• CCRS Implementation – Developing Complex Composite Services
– Logical Architecture
– Service Orchestration Overview
• WebSphere DataPower Performance Testing and Capacity Planning
– Approach
– Test Cases & Results
• WebSphere DataPower Tuning and Optimization Options
20
21. Tuning Options
• Caching : Caching is the most important option for performance tuning, think of
the time used for network communication, retrieval of WSDL documents or
compiling XSLT files.
– XSLTs
• Objects > XML Processing > XML Manager > XML Manager ->XSL cache
size
– Documents
• Objects > XML Processing > XML Manager > XML Manager -> Document
Cache Size/ Document Cache Count
– WSDLs stored in WSRR
• Control Panel > Web Service Proxy > Your Proxy->WSDL tab->WSDL
Subscription->Refresh Interval
– AAA Cache
• Objects > XML Processing > AAA Policy > Your Policy-> AuthN/AuthZ tab-
> Cache Lifetime
21
22. Tuning Options (Contd.)
• HTTP Persistent Connections : HTTP Persistent connections means reusing
opened connection for sending multiple messages instead of
opening/closing a connection for each message, this option decreases CPU
cycles, network traffic and latency.
– Control Panel-> Your WSProxy->Advanced Proxy Tab->Persistent
Connections = "On“ and specify persistence timeout
• Streaming : It provides many advantages, one of which is decreasing
memory usage.
– DP supports many types of streaming such as
• execution streaming (using SAX when executing XSLTs), attachment
streaming,
• message streaming
• context streaming
• WebSphere MQ Queue Manager
• When connecting to MQ, use "dpmq://" instead of direct MQ
connection "mq://",
22
23. Tuning Options – (Contd.)
• Processing Rules :
– PIPE and NULL Contexts
• Whenever applicable use PIPE as Input and Output between two
contiguous action nodes
• Use NULL as Input or Output of any action that doesn't need to
produce or use predecessor actions' results
– Asynchronous Rules
• Limit the usage of asynchronous rules because as you can
imagine parallel processing will increase CPU Utilization (Context
Switching)
– Reusable Rules
• Reusable rules are often called many times from parent
rules, this may introduce redundancy in action node execution
– XSLT Code
• Avoid using "//"; specify the node you are targeting using its
absolute tree path
• Limit the number of backend calls like url-open23
24. 24
Self Balancing: Self balance across a cluster of appliances
Replace front-end IP load balancer
New support (introduced in firmware version 4.0.2) enables connections to be
preserved, without loss, during failover scenario
Dynamic and Intelligent Load Distribution to backend systems
Replace backend load balancer
Front-end IP load
balancers not needed
Self balancing (IP
spraying)
Application Optimization
25. 25
Provides application-aware Intelligent Load Distribution
Auto-discovers application targets and distributes load using dynamic feedback
mechanism
Topology learning for WAS ND and VE
Uses intelligent weighted distribution algorithms based on current server load
Weighted Least Connection load balancing algorithm
Provides several options for enabling Session Affinity
DataPower performs dynamic back-side
routing and load distribution (leveraging
dynamic information from back-ends)
Failure of target appliances
are masked by appropriate
weighted distribution
Application Optimization
26. REST
2
6
DataPower XC10 As a Side Cache
User
1
5
3
2 4
Client
Provider
1. Client submits application request.
2. DataPower XI parses request and queries XC10. On a hit, skip to step 5.
3. On a miss, XI forwards request to target Provider.
4. XI adds application response to XC10.
5. Client receives response from XI.
Easily integrates into the existing business process
– No code changes to the client or back-end
application
– Simply add the side cache mediation at the ESB laye
Significantly reduces the load on the back-end system by
eliminating redundant requests
Response time from elastic cache is in milliseconds
Improved
Response Time
ImprovedLoad
DataPower XC10
DataPower XI Appliances
Large Response Time