The document discusses challenges in integrating new technologies like APIs, microservices, and cloud services with existing core systems. It emphasizes the need for consistency across the organization to integrate old and new systems effectively. Some key challenges discussed include transaction management across systems, data consistency, operations, and integrated reporting. The document also provides examples of integration patterns and technical enablers that can help with impedance matching between new and old systems.

1. Old Meets New
Managing Transactions
on the edge of the New World
API Days London
October 2020
Paul Lucas | IBM Distinguished Engineer
CTO UKI Financial Services Sector
Email: paul_lucas@uk.ibm.com
May 2020

2. ©2020 IBM Corporation1
Context
Cloud services, Containers, APIs, Streams, Microservices, DevSecOps. are all
now firmly mainstream IT: our default tool box for today.
All organisations are on a journey to a Hybrid Cloud architecture – intentional
or unintentional – monocultures have gone.
Much of the IT that has grown up over the last 50 years is still around, and
central to business. Core workloads are only in the very early days of moving
on to the new stack.
The new stack is no longer the leading edge of
fashion - time to make it boring and predictable,
and to work well with the broader IT estate.

3. ©2020 IBM Corporation2
What’s the problem?
We need consistency for the organisation
For many years we had established ways to
solve “job X”:
• Need a Transaction Monitor? Run it under
CICS
• Assured delivery of your messages ? Two
Phase Commit via XA transactions across
your Queue Manager and RDBMS
• High availability – HA/DR stack with cross
site synchronous data replication
We return to our Pattern book
• Consistency helps everyone – the Industrial
revolution didn’t happen because everybody
invented their own screw threads.
• We need to converge on a common solution to truly
get the benefits for the enterprise – implementors
must make many decisions
• API based vs Stream Based?
• Workflow or Event Driven? Both?
• How do I deal with technical failures?
• …
• New models will not “win” and completely replace
the old world.

4. ©2020 IBM Corporation3
With all major shifts there are two sides to the
coin:
• Problems they solve
• Challenges they create

5. ©2020 IBM Corporation4
Heads, I win
Providing scaling to support
transformation:
• Rapidly exposing capability in
a quick and light weight way
• Protecting Systems of Record
• Data Caches
• Circuit Breakers
New approaches to:
• Always On
• Elastic Scaling
• Graceful degradation of
service
All improve the presentation of my
core to the outside world.
System of
Record 1
API G/W
Digital
Channels
Mediated
Channels
…
Customer
Read
Data
Store
Orchestration & Integration
System of
Record n
Orchestration & Integration
Gateway
Industry
Connectivity
System of
Record 1
API G/WDigital
Channels
Mediated
Channels
…
Customer
Orchestration & Integration
System of
Record n
Orchestration & Integration
Gateway
Industry
Connectivity

6. ©2020 IBM Corporation5
Tails, I’ve got some work to do
Matching old and new worlds is not a new
problem:
• Real time banking and payments with a batch
core banking platform?
• Integrate a CRM system with a billing system
- both of which think they own the Customer
record?
In electronics, impedance matching is the practice of
designing the input impedance of an electrical load or
the output impedance of its corresponding signal source
to maximize the power transfer or minimize signal
reflection from the load.
Source: Wikipedia
I need Macro level strategy considering:
• End to end transaction management
• Data consistency across old and new
• End to End operations
• Integrated Reporting
• Ability to progressively transform

7. ©2020 IBM Corporation6
An Example
A modern Product System to serve a new customer segment,
for example a new pre-paid card, sitting on to the side of a
“traditional” core Banking system.
• New product platform manages customer channel
interaction, loyalty scheme, pre-paid balance.
• Functions such as Available Core Banking, Funds,
Customer Data Master, Accounting and Payments
components remain unchanged.
Questions:
• How much do you put in the integration layer,
and how long do you expect it to live?
• The level of coupling can vary - depending on
the granularity of calls between old and new
world.
Integration Core Platform
Customer
Channels
Core Banking
Platform
Data Transform
File Handling
Routing
Customer Master
APIs / StreamsNew Product
Platform
1
2
3
1
2
3

8. ©2020 IBM Corporation7
Detailed Considerations
Solution Features:
• Data Transfer and Availability
• Non-Functional Characteristics
• Data Mapping & Transformation
• Security
• Transactionality
• Operations
Transient Features:
• Roadmap for change
• Deployment
• Environments & Testing
Integration Core Platform
Customer
Channels
Core Banking
Platform
Reporting
Data Transform
File Handling
Routing
Customer Master
Data
Warehouse
APIs / Streams
New Product
Platform

9. ©2020 IBM Corporation8
Patterns
Eventual Consistency is the name of the game
• Saga: a series of local transactions - with compensating transactions for failures
• Developers must write more code - everything you do needs an undo
• We no longer have read consistent across the whole transaction
• Failed compensation actions leave you “in doubt”
• Orchestration Based vs Choreography Based
• Where should I read data from to get the right outcome?
• Pessimistic for my Available Funds? Don’t want to spend twice …
• Optimistic for Customer Data?
• How am I going to deal with
• In doubt?
• Retries?
• Drawing an edge in my data?

10. ©2020 IBM Corporation9
Technical Enablers
We have a rich tool box
• z/OS Connect EE
Unlocking mainframe capabilities with REST APIs –
inbound and outbound.
• Transaction managers
New external Transaction Managers are starting to
appear.
• Queue Managers / Bridges
Solutions such as Apache Kafka Connect enables
bridging to traditional queueing technologies such as
MQ
• Change Data Capture
CDC is available for almost all Database systems –
enabling reliable replication and reconstruction on
different target.
Outbound data transfer using MQ + Kafka connect
Outbound data transfer using z/OS Connect EE

11. ©2020 IBM Corporation10
Summary
• Think about longevity as well as MVP
• Think about how the whole will work, not solely the challenge of the today
• Assume your solutions, and all their integration may still be there in 20
years.
• Guidelines
• Isolate and separate new solution from integration solution
• Manage roll out to minimise co-existence
• Understand the current state as a basis for the co-existence Integration
Services
• Minimise impact or changes to current applications/infrastructure.