PureApp Hybrid Cloud - Mark Willemse ING Presentation 11th September 2014
S424. Soa Mainframe Practices Best And Worst
1. S424. Mainframe Service
Architecture and Enablement
Best and Worst Practices
• Michael Erichsen,
Chief Consultant, CSC
• Stockholm, 8 June 2011
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 0
2. Main points
• Why are we doing this?
– We have done integration projects in decades
– Service orientation can be a qualitative improvement
– The new aspect is open, standardized, loosely coupled interfaces
• Best practices are mostly found in the technicalities of service
enablement
– Case studies of seven different ways of doing it
• Best and worst practices are found in the service modelling and in the
implementation of models and architectures
– Some principles and patterns based on best practices
– Worst practices illustrated by ”anti-patterns” and research in what can go wrong
and how
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 1
3. A Conceptual, Technical View of Service Enablement
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 2
4. A Conceptual, Modelling View of Service Enablement
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 3
5. SOA Mainframe Practices – Best and Worst
To Begin with the
Conclusion
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 4
6. Why are we doing this?
Which problems are we trying to solve?
• The world is fast and messy
• Businesses and authorities need to
adapt very quickly to survive
• IT systems are generally
heterogeneous
• It is a mess - and it will remain so
• Service interfaces hope to remove
some of the mess – but also
introduce new kinds of mess
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 5
7. Reuse?
Oh no, not again!
• Software reuse is the process of
creating software systems from
existing software rather than
building software systems from
scratch
• First proposed in 1968
• Four dimensions:
– Abstraction
– Selection
• Classification
• Retrieval
• Exposition
– Specialization
– Integration
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 6
8. Reuse?
Promises, promises, promises…
• Over time
– Component Libraries with automated
customization
– Cut and paste of COBOL code
– Application Generators
– Object orientation
– Frameworks
– Service interfaces
• Reuse is not a technology, but a
culture
– Why not reward for reuse and
reusability?
• Otherwise a vendor might earn more
money by not reusing
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 7
9. Overarchitecting and Overengineering
• You need an enterprise
architecture, an enterprise data
model and good business process
descriptions to get the full business
and economic advantage of such a
project
• Reuse depends on a higher level
of abstraction
– But too many levels of abstraction
could mean that you can’t use it at all
in the first place
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 8
10. Extending or replacing Mainframe systems?
• The mainframe platform is
continuously under siege
– IBM's monopoly makes it difficult to
forge alliances
– The only practical solution to the age
problems seems to be offshoring
– The best way to protect the big
investments in mainframe systems is
to open them as equal partners in
modern architectures
• Not COBOL or Java/.Net, but strategy
and interfaces
– Using zSeries for SOA Infrastructure
interesting, but not unproblematic
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 9
11. The real economy of zIIP’s and zAAP's
• MIPS come cheaper, but they are
certainly not free
– Processor costs
– Other operation costs
• Investment in real storage
– Much more expensive than other
memory hardware
– Java Virtual Machines need very
large heap sizes for many
applications, including SOA
infrastructure like ESB’s and
BPMS’es
– Memory overcommit perhaps only
1,5 on new workloads?
• Calculate economy very carefully
to compare with other platforms
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 10
12. SOA Mainframe Practices – Best and Worst
Lewis et al, SEI-CMM:
SMART: The Service-
Oriented Migration and
Reuse Technique
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 11
13. SMART: Defining a service
• A service is a coarse-grained, discoverable, and self-contained
software entity that interacts with applications and other services through
a loosely coupled, often asynchronous, message-based communication
model
– Quoted from Brown et al, Rational: Using Service-Oriented Architecture and
Component-Based Development to Build Web Service Applications
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 12
14. SMART: Defining a SOA
– A SOA is a collection of services with well-defined interfaces and a shared
communications model. A system or application is designed and implemented
to make use of these services
• 1. Service interfaces are added to existing enterprise information systems for
applications to use, while these systems remain unchanged for internal users
• 2. Service-specific code is written to provide functionality for applications to use
• 3. Services written by third parties and deployed elsewhere are used within applications.
– SOAs also offer the promise of enabling existing legacy systems to
expose their functionality as services, without making significant changes
to the legacy systems themselves
• This is one of the most attractive features of SOA to many organizations that do
not wish – and cannot afford – to walk away from their investment in legacy
systems or redevelop the same capabilities as services from scratch
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 13
15. SOA Mainframe Practices – Best and Worst
Technical Best Practices:
Seven Case Studies
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 14
16. Seven Case Studies
1. A Tactical Service Enablement
2. A Tactical Service Enablement Evolving into Strategy
3. COBOL IMS in a large Scale SOA Environment
4. International, multilateral, heterogeneous and data replicating
5. XML Services over MQ
6. WebSphere ESB on z
7. Service Enablement of a Legacy Application Generator System
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 15
17. Case #1: A Tactical Service Enablement
• Back end system
– VSE, CICS, a 4GL and a non-relational data base
• The business need
– Update requests from business partners used in nightly batch runs
• No requirement for online updates of the mainframe system
• Web Services a mandatory protocol for communication
– Security required using TLS and X.509 certificates
– Not a general interface to the entire system, but a tactical solution for a single
demand
• Technical challenges
• VSE native SOAP and XML features did not really work for us
• No cryptographic hardware on the VSE box
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 16
18. Case #1: The Solution
The easiest solution was to
use a WebSphere
Application Server on
Windows
Daily updating Web Service
requests from partners
Weekly master data extracts
from mainframe for each
partner
Quarterly and semi-annually
control data extracted by
mainframe
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 17
19. Case #2: A Tactical Service Enablement
Evolving into a Strategic Solution after a successful
Proof-of-Concept
• Back end system
– An ”i-Case” tool generating COBOL in CICS at the back end and a fat Java
client at the front end with required runtimes and a proprietary communication
layer
– A generic callable interface to the application was already in place
• The business need
– Online updates from business partners’ own systems
– Web Services a mandatory protocol
• Technical challenges
– Operation and field names had to be mediated between existing WSDL-
standard and COBOL restrictions
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 18
20. Case #2: Technical
Solution
CICS Web Support (now superseded
by Web Services for CICS)
An existing COBOL adapter to the
case tool generated code had to be
modified to a dynamic wrapper
Mapping definitions hand crafted
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 19
21. Case #2: A Proof of Concept
• A Proof of Concept was designed to explore possible problems like how
does
– Complexity and verbosity of the mandatory XML dialect fit with the tools
available?
– Internal security fit with this channel of connections?
– Digital certificates work in real life in this configuration?
– Service Enablement tools fit with exiting interfaces?
– Data from different parts of the back end system match to build a meaningful
response?
• The customer provided the XSD-definitions
• CSC built the adapter modules based on these and on the existing
structures in the legacy system
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 20
22. Case #2: Interface Design
• Discrepancies between XML data concepts and COBOL data concepts
had to be solved
• In real life it seems that no interface contract is perfect
• Data was defined in DB2, in COBOL, and in XML
• Optional and Variable Fields (0..1) (0..n) (0..*)
• Strings
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 21
23. Case #2: COBOL and XML Definitions
Type COBOL XML
String PIC X() String
Float COMP-1 Float
Double COMP-2 Double
Packed decimal COMP-3 Decimal
Binary COMP-4 Long, short, int
Date PIC X(10) in the application, 2000-01-12
Content: 20061016 2000-01-12T12:13:14Z
2000-03-04T23:00:00+03:00
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 22
24. Case #2: COBOL and XML Field Examples
Field COBOL XML
a 9(8) Date
b S9(5) unsignedInt totalDigits value="5"
c S9(11) COMP-3 Decimal totalDigits value="11"
fractionDigits value="2"
d S9(10) Double maxInclusive value="9999999999"
e x(30) String maxLength value="30"
f S9(8) unsignedInt totalDigits value="8"
g S9(4) String pattern value="[0-9]{4}"
h S9(10) Double maxInclusive value="9999999999"
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 23
25. Case #2 Version 2:
Back End Modernized,
Interface Simplified
Ongoing project to migrate case
tool to runtime-free COBOL and
Java to be maintained in Rational
Developer for z (RDz)
Back end web service interface
created bottom-up from
generated COBOL code using
RDz
Proprietary client communication
code changed to Web Services
for CICS
Front end client communication
using Web Services over HTTPS
with generated communication
code adapted to the WS4C
format
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 24
26. Case #3: COBOL IMS in a large scale
SOA Environment
– Back end system
• Large IMS COBOL system
– The business need
• Modernizing the entire heterogeneous enterprise portfolio based on SOA integration and
supporting a large number of protocols and formats
• Service enabling IMS systems with DB2 data bases
– Technical challenges
• Reusing existing web enablement skills and technology
• The new enterprise architecture regards all components on all platforms as both service
providers and service providers
– Must support both ingoing and outgoing Web Services requests
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 25
27. Case #3: The Solution
Simple interface to IMS
applications
Reusing large application
”motors”
OTMA (Open Transaction
Manager Access)
IMS Connect
WebSphere Application
Server with the AXIS
framework
Also coding new, outgoing
Web Services requests from
IMS COBOL applications
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 26
28. Case #3: The challenges
– There are limits on the protocol of 400.000-500.000 bytes
– Legacy applications used screen oriented logic
• Handled by adding new logic on top of legacy applications
– Existing web enablement coded in Visual Age for Java using the now obsolete
Record Framework
– Advantages
• The continuing existence of many skilled programmers and designers with deep
business knowledge
• Experience from an existing, well performing, and very successful web interface to the
IMS and DB2 systems
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 27
29. Case #4: International, Multilateral,
Heterogeneous and Data Replicating
• Back end system
– An IMS DB2 MQ system
• The business need
– International, multilateral exchange of data, going from data base replication to
a service oriented architecture
• Technical challenges
– Many different data types with big variances
– Large binary data items
– Consistency checking between central and national data bases
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 28
30. Case #4: The Original
Solution
A Web Services Gateway
implemented as a simple
servlet on WebSphere
Application Server on z
MQ communication with IMS
COBOL programs handling
XML and Unicode, storing
data in DB2
Consistency checks became
too complicated and too
difficult to make well
performing
Local data base solution
superseded by direct service
requests to the central
system
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 29
31. Case #4: Taming the CPU Hog
• Technical Challenges
– IMS COBOL XML Unicode program was originally coded only for functionality
using XML PARSE, not for performance
– It used vast amounts of CPU
– A code review did not show anything unsound
– Transactions were profiled using Compuware STROBE to split CPU-usage into
handling of MQ, XML, Unicode and DB2
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 30
32. Case #4: Untuned system
0.50 CPU-second per transaction
MODULE SECTION
NAME NAME PROCEDURE/FUNCTION NAME CPU TIME Interpretation
Runtime Storage
.LELIB CEEBINIT CEEVGTSI GET A STACK INCREMENT 51.91 allocation
IGZCHXP COBOL LIBRARY
.COBLIB IGZCPAC SUBROUTIN 31.69 XML
CEEVOGTS XTND USR STK/DSA Runtime Storage
.LELIB CEEBINIT OPLINK 7.10 allocation
CUNMUNI 1.64 Unicode
.SVC SVC 228 USER SVC 1.64 IMS
CEEVOGSX XTND USR STK/DSA Runtime Storage
.LELIB CEEBINIT OPLINK 1.09 allocation
RPS000 .55 The Program
RPS000 .55 The Program
System Storage
.SVC SVC 120 GETMAIN/FREEMAIN .55 allocation
.SVC SVC 013 TERMINATION .55 System
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 31
33. Case #4: COBOL Working Storage Reduced
1M to 100K
0.06 CPU-second per transaction
MODULE SECTION CPU
NAME NAME PROCEDURE/FUNCTION NAME TIME Interpretation
CEEVGTSI GET A STACK Runtime Storage
.LELIB CEEBINIT INCREMENT 36.17 allocation
IGZCHXP COBOL LIBRARY
.COBLIB IGZCPAC SUBROUTINE 31.91 XML
CEEVOGTS XTND USR STK/DSA Runtime Storage
.LELIB CEEBINIT OPLINK 10.64 allocation
.XES IXLR1ALR ALTER W/RECORD REQUEST 4.26 SysPlex Services (DB2)
System Storage
.SVC SVC 120 GETMAIN/FREEMAIN 4.26 allocation
.SVC SVC 013 TERMINATION 2.13 System
.NUCLEUS CSVEXPR CONTENTS SUPERVISION 2.13 System
CEEVOGSX XTND USR STK/DSA Runtime Storage
.LELIB CEEBINIT OPLINK 2.13 allocation
.DB2 DSNIDM DATA MANAGEMENT DRIVER 2.13 DB2
.COBLIB IGZCPAC IGZCVMO VARIABLE LENGTH MOVE 2.13 COBOL Move
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 32
34. Case #4: Further tuning of LE/370
Measurements from the LE/370 Storage Report
Before After
• The important parameter is
“Number of segments allocated:”,
Initial stack size: 131072 1048576 which should be 0 or 1
Increment stack size: 131072 102400
• Used parameters after tuning:
Largest used by any thread: 1010896 1007944
Number of segments
•ALL31=(ON),
allocated: 3 1 STACK=(1M,100K,ANYWHERE,
KEEP,512K,128K),
Initial heap size: 32768 921600
HEAP=(900K,128K,ANYWHERE,
Increment heap size: 32768 131072 KEEP,512K,128K),
Total heap storage used
(sugg. initial size): 818184 818152
ANYHEAP=(32K,8K,ANYWHERE,
Number of segments KEEP),
allocated: 2 1 STORAGE=(NONE,NONE,NONE,
Initial anyheap size: 16384 32768 0K)
Increment anyheap size: 8192 8192
Total heap storage used
(sugg. initial size): 25648 26456
Number of segments
allocated: 2 1
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 33
35. Case #4: Language Environment Parameters Tuned
0,03 CPU-second per transaction
MODULE SECTION
NAME NAME PROCEDURE/FUNCTION NAME CPU TIME Interpretation
IGZCHXP COBOL LIBRARY
.COBLIB IGZCPAC SUBROUTINE 46.15 XML
.COMMON .COMMONX EXTENDED COMMON AREA 7.69 System
CSQQSTUB QUIESCE AFT SYS LOG
RPS000 .MQSRIES ERR 5.13 MQ
CUNMUNI 00D1C0 5.13 Unicode
.SVC SVC 013 TERMINATION 5.13 System
.DB2 DSNXGRDS RDS ACCESS MODULE GENER 5.13 DB2
.DB2 DSNIDM DATA MANAGEMENT DRIVER 5.13 DB2
.DB2 DSNGEDM DATA MGT DBD/SKCT RTNS 5.13 DB2
RPS000 RPS000 00B380 2.56 The Program
.SVC SVC 228 USER SVC 2.56 System
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 34
36. Case #5: XML services over MQ
• Back end system
– A merger has created a heterogeneous environment with CICS systems, IMS
systems and Wintel systems
• The business need
– A very big SOA program aims at having a new midrange industry system as the
front end and gradually replacing the legacy systems
– Service enabling CICS systems
• Technical challenges
– Generic callable interfaces to CICS and IMS systems are already in place
– MQ has been the integration channel for CICS and IMS for a number of years
– The enterprise service bus is implemented as Microsoft Biztalk
• Tools do not support SOAP over MQ, but SOAP over HTTP and XML over MQ
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 35
37. Case #5: The Solution
A modified Web Services for
CICS implementation
Reuse existing Biztalk tooling
Use RDz tooling to generate
WSBIND files for the existing
callable interface
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 36
39. Case #5: Dummy SOAP Header Handler Program
GET CONTAINER(PI-DFHFUNCTION)
IF PI-RECEIVE-REQUEST THEN
GET CONTAINER(PI-DFHREQUEST)
Remove existing <?XML tag
Add a dummy SOAP envelope around the payload
PUT CONTAINER(PI-DFHREQUEST)
DELETE CONTAINER(PI-DFHRESPONSE)
ELSE IF PI-SEND-RESPONSE THEN
GET CONTAINER(PI-DFHRESPONSE)
add <?XML tag
Test each tag in Response-Data to find if it starts with <soap or
</soap and then remove
PUT CONTAINER(PI-DFHRESPONSE)
Move "text/xml" to MEDIATYPE-DATA
PUT CONTAINER('DFHMEDIATYPE')
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 38
40. Case #6: WebSphere ESB on z
• Back end system
– An IMS COBOL MQ system
• The business need
– Integration between applications on z, Oracle/Unix and Wintel using Web
Services and MQ
• Technical challenges
– The need for real storage is much larger for JVM’s than for traditional workload
– Recycling JVM’s costs hundreds of MIPS in quite extended periods
• Even though most of it is offloaded to z**P processors
• Z**P processors are not free either
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 39
41. Case #6: WebSphere
ESB on z Challenges
After a long quiet night the
first transactions in the
morning could take minutes
causing client timeouts
Difficult to monitor –
compared to CICS, IMS, MQ
etc.
Workload Manager policies
quite complicated
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 40
42. Case #7: Service Enablement of a Legacy
Application Generator System
• Back end system
– CICS, DB2, originally coded in the 4GL CSP
• The business need
– A 20 year old legacy application still represents the core of the business and a
long-term investment
– Back office and staff can use web and cloud applications, which need access to
the core system and its data, both inquiries and updates
• Technical challenges
– The application was coded in CSP and still has a 3270 user interface
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 41
43. Case #7: The Solution
Web Services for CICS
developed using EGL tooling
in Rational Business
Developer (RBD) and RDz
The first service was used by
a web form
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 42
44. Case #7: The Solution
Web Services for CICS
developed using EGL tooling
in Rational Business
Developer (RBD) and RDz
The first service was used by
a web form
The second service was
used by a Salesforce cloud
application
The integration from
Salesforce was done using
a Cast Iron SOA appliance
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 43
45. Case #7: The Solution
• The first service with a single operation just updated a single field in the
core data base
– The front end is a simple web form, which calls a web service
– Very easy to implement – took 20 minutes to code and generate
• The second service with half a dozen operations ran into all kinds of
technical problems
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 44
46. Case #7: Modelling the Service
• Describe the existing user interaction with the 3270 application
• Model this as a business process
• Describe the operations needed in the new service to support this
process
• Define the programs needed to implement the operations and the
interface as message structures
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 45
47. Case #7: Restrictions when developing
Web Services for CICS
• If you need to have a structure as a response from an EGL Web Service
for CICS the request must be the same structure (Input == output)
• A configuration file called the “deployment descriptor” must not have the
same name as any EGL program
– RBD will quietly generate a COBOL program with the name of the deployment
descriptor to overwrite the legitimate program just in case you also wanted to
have an outgoing web services request
• The predecessors to EGL were COBOL-like in their structures and
culture, while EGL is java-like in its structures and culture
– This difference can lead to great difficulties when service enabling legacy
programs
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 46
48. Case #7: Restrictions when developing
Web Services for CICS
• Do you have arrays in the externally exposed structure definition in the
service programs?
– Then the structure needs to be an unstructured record to have arrays generated
correctly into the WSDL document
– The field definitions must not be preceded by level numbers
• Otherwise the array will be “flattened” with all occurrences of each subfield
• {a1 a2 a3 b1 b2 b3 c1 c2 c3} in stead of {a1 b1 c1 a2 b2 c2 a3 b3 c3}
– Note that unstructured records are dynamic
• If you do not allocate them yourself they will be empty or even null
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 47
49. Case #7: Restrictions when developing
Web Services for CICS
• A static array like this:
01 MYARRAY ARRAYRECORD[10];
Is allocated in storage like a COBOL program would
• A dynamic array, distinguished by not having level numbers:
myArray arrayRecord[10];
Is not allocated, but must be explicitly defined to avoid null pointers
• EGL generation for COBOL does not support multidimensional, dynamic
arrays
• All this is not considered an error, but a design, by IBM, even
though it is deeply unintuitive for a legacy programmer
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 48
50. Summing up the case studies
• Mainframe service enablement is probably our most important area of
mainframe application development in recent years
• There are many ways to service enable an existing application
• Use your existing skills, tools and methodologies where you can
• Make the service interface completely hide what is behind it
• Remember:
– To be reusable it has to usable in the first place!
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 49
51. SOA Mainframe Practices – Best and Worst
Best and Worst Practices
in Service Modelling and
Architecture
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 50
52. Best and Worst
Practices
•Mark Skilton, CSC: Three simple
rules for definition what is a Service
are
•Forrester: North American SOA
Systems Integrators
•Gartner: The 13 least wanted
•Ang, Cherbakov & Ibrahim, IBM:
SOA Antipatterns
•Tarak Modi, Unisys: SOA
Antipatterns
•IBM on SOA Governance
Nice to have research to quote
Rather than admitting your own
mistakes
"The secret to creativity is knowing
how to hide your sources.“
(Albert Einstein)
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 51
53. SOA Mainframe Practices – Best and Worst
Mark Skilton, CSC:
Service Oriented
Architecture
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 52
54. Mark Skilton, CSC: Service Oriented Architecture
• Three simple rules for definition
what a Service is
• Five key areas to consider when
looking at the granularity of
business and IT services
• Three key operating principles of
SOA/POA underpin these choices
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 53
55. Mark Skilton, CSC:
Three simple rules for definition what a Service is
• It is strategic
– The capability is critical to your organisation
• Composable
– The service can be run in its own and be combined with other services to build
new services not originally specified
• Rule of Three
– If an interface and/or message is used in more than three systems
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 54
56. Mark Skilton, CSC:
Five key areas to Consider when looking at the
Granularity of Business and IT Services
• Performance and size
– Neither too big nor too small
• Transactionality and state
– This audience should know all about
that!
• Business suitability
– One operation for one business step
• Safe operations
– Read only or reentrant
• Idempotency of operations
– Can be called again without
changing the result
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 55
57. Mark Skilton, CSC:
Three key operating principles of SOA/POA
underpin these choices
• Process Integrity
– Active
– Safe
– Idempotent
• Data Integrity
– Transactionality and states
– Data Synchronisation
– Logic
• Security Integrity
– Security policies, processes and
technologies
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 56
58. SOA Mainframe Practices – Best and Worst
Forrester:
North American SOA
Systems Integrators
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 57
59. Forrester: North American SOA Systems Integrators
• In a typical project that uses SOA strategically, there are typically five
steps that organizations need to follow to successfully complete the
project
1. Develop a future state for the business to evolve toward
2. Map existing business services to current business processes
3. Map business services to IT solutions
4. Map IT assets to IT solutions
5. Design SOA governance and infrastructure
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 58
60. SOA Mainframe Practices – Best and Worst
Gartner:
The 13 Least Wanted
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 59
61. Gartner: The Thirteen Least Wanted
1. Starting too big 8. Excessive centralization
2. Starting in the wrong place 9. Irrational SOA exuberance
3. Leaving SOA to the "techies“ 10. Forgetting to consider the data
4. Underestimating the technical 11. "Not invented here" syndrome
(and infrastructure) issues 12. Allowing nonshareable services
5. Assuming that everyone thinks to proliferate
like you 13. "I already have 200
6. Selling SOA before you're ready services, now what?”
7. Choosing anarchy or
dictatorship as leadership styles
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 60
62. SOA Mainframe Practices – Best and Worst
Ang, Cherbakov &
Ibrahim, IBM:
SOA Antipatterns
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 61
63. Patterns and Anti-Patterns
• Patterns are a popular way of
illustrating best practices
– SOA Patterns are used very much in
discussions and literature
• Anti-patterns are being used more
and more to slow down the hype
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 62
64. Ang, Cherbakov & Ibrahim, IBM:
SOA Antipatterns (1)
• Adoption
– Technology Bandwagon
– So, What’s New?
– The Big Bang
• Design
– Blob
– Poltergeists
• Structure
– Spaghetti code
– Stovepipe systems
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 63
65. Ang, Cherbakov & Ibrahim, IBM:
SOA Antipatterns (2)
• Technology
– Wolf ticket
– Continuous obsolescence
• Reuse
– Cut-and-paste
– Golden hammer
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 64
66. Ang, Cherbakov & Ibrahim, IBM:
SOA Antipatterns (3)
• Identification & Design
– Web service = SOA
– The Silo Approach
– Misbehaving Registries
• Realization
– Chatty Services
– Point-to-point Services
– Component-less Services
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 65
67. SOA Mainframe Practices – Best and Worst
Tarak Modi, Unisys:
SOA Antipatterns
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 66
68. Tarak Modi, Unisys: SOA Antipatterns (1)
• The "Same Old, Same Old"
Phenomenon
• The "Big Bang" Approach
• Service Fiefdoms
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 67
69. Tarak Modi, Unisys: SOA Antipatterns (2)
• Technophilia
• Bloated Services
• Anorexic Services
• Hyper Brokerage
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 68
70. SOA Mainframe Practices – Best and Worst
IBM on SOA Governance
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 69
71. IBM on SOA Governance
• "Wild West" or “Rogue” Services
– Extremely difficult to gain control over
• "Duplicated” Services
– Superficially effective but limited real
savings
• "Shelfware” Services
– A waste of resource, won’t deliver
benefits
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 70
72. IBM on SOA Governance
• Unsecure Services
– Limits service use and business
opportunities
• Rigid Services
– Roadblock to agile, flexible business
processes
• Ineffective Service Management
– Services must be managed as
resources
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 71
73. SOA Mainframe Practices – Best and Worst
A few more Issues
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 72
74. Carsten Svensson, EEI: The Cost of SOA
• Cost of SOA =
(Cost of Data Complexity +
Cost of Service Complexity +
Cost of Process Complexity +
Enabling Technology Solution)
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 73
75. Performance
The daily CPU consumption pattern
will change 2500
If you only run an online workload
If you run online and a lot of batch 2000
If you run in a service enabled
environment, where services are
used by a self service web application 1500
If you rely on a batch window it might Online
collide with the self service window 1000 Self-service
after dinner, when the kids are in bed Online & Batch
500
0
1 3 5 7 9 11 13 15 17 19 21 23
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 74
76. The culture clash between Java and COBOL
• COBOL was born to optimize
administrative computing
– Java and XML was born for dynamic
interfaces
– It takes a lot of sweat to reconcile
them
• Data in COBOL working storage
allocated implicitly by the compiler
– Data in Java must be allocated
explicitly by the programmer
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 75
77. To Summarize
• There are many ways to integrate mainframe systems into service
oriented architectures
– A lot of best practices are to be found in the technical solutions
• You need a close cooperation between business
stakeholders, architects, developers and technologists to make it work
– The worst practices are found when application, design and architecture on one
hand does not take the requirements of the technologies and the infrastructure
seriously
• All the mistakes have already been made and are extensively
documented
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 76
78. SOA Mainframe Practices – Best and Worst
THANK YOU
CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 77
79. CSC Proprietary and Confidential 6/26/2011 8:47 AM PPT 2003_Toolkit_FMT.ppt 78
Editor's Notes
Service Oriented Architectures and service enablement have been in production for a number of years now. Experiences are broad and diverse. The speaker has participated in several projects for various companies and public authorities and will try to sum up some of the best practices and some of the worst practices usedWhy are we doing this?We have done integration projects in decadesThe new aspect is open, standardized, loosely coupled interfacesBetter separation of dutiesNot necessarily easierCase StudiesSeven different ways of doing itAnti-PatternsSome research in what can go wrong and howShow ofhands: Focusontechnicalaspectsormodelling and architecture?
An existing back end program or system can have a numberofentry points, either as callablemodulesor as screensthatneed to befacadedoruncoupledTechnicallyyou have to add a service interface in front oftheentry pointThismaps to a service and a numberof operations, and the real hard part ofthework is to model services and operations to match boththetechnical back end and the business processesthey have to support and to bereusable to save developmentcosts in thefutureIfyou have toocoarsegrained services and operations theymight not bereusableIfyou have too fine grained services and operations theymight not beable to performproperly
The world is fast and messyBusinesses and authorities need to adapt very quickly to surviveIT systems are generally heterogeneousNot only in platforms and technology, but also in space, time, culture, philosophy, paradigms, and scopeIt is a mess - and it will remain soUntil the current craftsmanship has been generally transformed to industrial processesService interfaces hope to remove some of the mess – but also introduce new kinds of mess
Software reuse was first proposed“Mass Produced Software Components” by Douglas McIlroy of Bell Laboratories at a NATO Software Engineering Conference in 1968.Four dimensionsAbstractionsA summary ofwhateachartifact is and does and howSelectionClassification and cataloging, Retrieval and ExpositionSpecializationHaving generic artifacts and being able to modify them by changing parametersIntegrationIs the focus of the case studies in this presentation
TheconceptofComponentLibrarieswithautomatedcustomizationpromisedreuse back in 1968Cut and pasteof COBOL codepromisedreuse. But alsomeantthaterrorscould live onforeverApplication Generators promisedreuse by automaticallyincluding standard codechunks. But theyworkbest in the domains theyweredesigned for, but not oftenwelloutsidethemObjectorientationpromisedreuse by inheritance and subclasses. But thesprawlofclasses and librariesmakesiteasier to build a new classratherthanfinding an existing to reuseFrameworkspromisedreuse. But their size and complexity makes understanding how to use them difficult:the functionality of components; the interactions between component; the mapping from the problem domain to the framework implementation and understanding the architectural assumptions in the framework design. Service interfaces promises reuse!!!Reuse is not a technology, but a culture. Why not reward for reuse and reusability? Otherwise a vendor will earn more by not reusing
You need an enterprise architecture, an enterprise data model and good business process descriptions to get the full business and economic advantage of such a projectIt should be possible to understand, visualize and enrich by everybody – or you can’t deliver!If you don’t have that you can still prepare the building bricks with a future architecture in mindReuse depends on a higher level of abstractionToo many levels of abstraction could mean that you can’t use it at all in the first place
MIPS comecheaper, but theyarecertainly not freeInvestment, amortization, otherlicenses, operations, housing, power, warming, cooling, changes, homecalls, problems, incidents, etc.Investment in real storageMuch more expensivethanothermemory hardwareJava Virtual Machinesneedvery large heapsizes for manyapplications, including SOA infrastructurelikeESB’s and BPMS’esMemoryovercommitperhapsonly 1,5 on new workloads?
Let us start at some definitions
Coarse-grained Provide significant business process capability, as opposed to low-level business functionsDiscoverable Can somehow be located and their interfaces understood. This is often misinterpreted as requiring dynamic (runtime) discoverySelf-contained Do not require context or state information of other services, nor do they maintain state from one request to anotherLoose coupling Modules have few, well-known dependencies, and interfaces to the module are defined to be as independent as possible from the implementation of the module
A Tactical Service EnablementVSE, CICS, 4GL, and a non-relational DBA Tactical Service EnablementEvolvingintoStrategyi-casegenerated COBOL and fat javaclients, CICS, DB2COBOL IMS in a large Scale SOA EnvironmentInternational, multilateral, heterogeneous and data replicatingIMS, DB2, MQ, and WebSphere Application ServerXML Services over MQCICS, COBOL, and DB2WebSphere ESB on zService Enablementof a Legacy Application Generator SystemEGL, COBOL, CICS, and DB2
The easiest solution was to use a WebSphere Application Server on WindowsDaily updating Web Service requests from partnersAccumulated on Windows serverFTP’ed to mainframe as part of nightly batch runWeekly master data extracts from mainframe for each partnerFTP’ed to Windows serverReceived by partner as Web Service requestsQuarterly and semi-annually control data extracted by mainframeFTP’ed to Windows serverReceived by partner as Web Service requests
Back end systemAn ”i-Case” tool generating COBOL in CICS at the back end and a fat Java client at the front end with required runtimes and a proprietary communication layerA generic callable interface to the application was already in placeThe business needOnline updates from business partners’ own systemsWeb Services a mandatory protocolTechnical challengesOperation and field names had to be mediated between existing WSDL-standard and COBOL restrictionsMeet-in-the-middle did not function in the WSED/WDz/RDz tooling at that timeMapping was done in a COBOL Wrapper program
CICS Web Support (now superseded by Web Services for CICS)No investments in hardware or runtime softwareA simple operations environmentWell known technology with a few additionsWorks best with proprietary IBM tooling: WSED/WDz/RDzAn existing COBOL adapter to the case tool generated code had to be modified to a dynamic wrapperOnly using a single generic adapterOne Web Service with many operations Each operation executes under a separate transaction code for access control, tuning and accountingMapping definitions hand craftedCOBOL copybooks generated from WSDL and Case ToolImported into Excel and mapped to data structure and application flowResult exported to DB2 table
Simple interface to IMS applicationsStrings in, strings outReusing large application ”motors”Legacy IMS COBOL front-ended by new logicOTMA (Open Transaction Manager Access)A transaction-based, connectionless client/server protocol into IMS running inside z/OSIMS ConnectRunning in separate address space, functioning as an OTMA ClientWebSphere Application Server with the AXIS frameworkGateway to the Enterprise Service Bus, exposing “tactical” servicesAlso coding new, outgoing Web Services requests from IMS COBOL applications
There are limits on the protocol of 400.000-500.000 bytesLegacy applications used screen oriented logicHandled by adding new logic on top of legacy applicationsPage up and page downScreen oriented messages misleading in the new environmentHad to change pseudo-conversational screen flows into one shot callsExisting web enablement coded in Visual Age for Java using the now obsolete Record FrameworkIf reused had to be changed, e.g. to SCA (Service Component Architecture)Headers were handled by custom code in COBOL programs due to lack of features in early versions
Back end systemAn IMS DB2 MQ systemThe business needInternational, multilateral exchange of data, going from data base replication to a service oriented architectureUpdates using SOAP over MQ/JMSInquiries using SOAP over HTTPLarge data using SOAP-initiated FTPTechnical challengesMany different data types with big variancesLarge binary data itemsConsistency checking between central and national data bases
A Web Services Gateway implemented as a simple servlet on WebSphere Application Server on zTransforming between between HTTP and MQMQ communication with IMS COBOL programs handling XML and Unicode, storing data in DB2Consistency checks became too complicated and too difficult to make well performingLocal data base solution superseded by direct service requests to the central systemMany COBOL/XML/Language Environment challenges met along the way
IMS COBOL XML Unicode program was originally coded only for functionality using XML PARSE, not for performanceThey used vast amounts of CPU, divided into two categoriesOne had two XML parses in order to get keys to subtables before data was extractedThe other had just one parse of XML to extract the payload dataThe CPU consumption was 2:1A code review did not show anything unsoundTransactions were profiled using Compuware STROBE to split CPU-usage into handling of MQ, XML, Unicode and DB2
Most of the consumption was in the LE/370 runtime when allocating storage (RAM)The working storage of the program was dimensioned for very large messages (1MB), which a business need review demonstrated unnecessary
A controlmeasurementusingthe IMS monitor Mainviewconfirmed an improvementof an orderofmagnitude
This is how far wecame by tuning thestorageallocation in the program and in LEFurther tuning wouldbeupgrading to thenewer COBOL compiler versions and/oroffloading to z**P processorsBut at that point a differentarchitecturewasdecidedwithdirectrequests from desktop clients to theremote partner
A modified Web Services for CICS implementationA Dummy SOAP Header Handler program Add SOAP headers to requestsRemove them from the responseReuse existing Biztalk toolingUse RDz tooling to generate WSBIND files for the existing callable interface
Back end systemAn IMS COBOL MQ systemThe business needIntegration between applications on z, Oracle/Unix and Wintel using Web Services and MQBased on an existing WebSphere Application Server on zExpected to use the SIBus featureThe integration requirements seemed to be easier to fulfil using an Enterprise Service BusThe need for real storage is much larger for JVM’s than for traditional workloadZ real storage is much more expensive than midrange memoryRecycling JVM’s costs hundreds of MIPS in quite extended periodsEven though most of it is offloaded to z**P processorsZ**P processors are not free eitherInvestment, amortisation, housing, power, cooling, warming, changes, incidents, problems, night calls, and various overhead
Back end systemCICS, DB2, originally coded in the 4GL CSPThe business needA 20 year old legacy application still represents the core of the business and a long-term investmentBack office and staff can use web and cloud applications, which need access to the core system and its data, both inquiries and updatesTechnical challengeThe application was coded in CSP and still has a 3270 user interfaceCSP has evolved into VAGen and then EGL, Enterprise Generation Language, which can be coded and generated using Rational Business Developer or RDzThe 3270 user interface can be supplemented by a parallel service interface, which can be exposed as Web Services for CICSThe callable back end modules can be used unchanged, but a service layer has to be coded as a façade
The first service with a single operation just updated a single field in the core data baseThe front end is a simple web form, which calls a web serviceVery easy to implement – took 20 minutes to code and generateThe second service with half a dozen operations ran into all kinds of technical problemsThe front end is a Salesforce cloud application, using a Cast Iron SOA appliance to create the web service requestThe Salesforce application supplies the screen flow to substitute the legacy 3270 screen flowThe service modelling was simple, based on the well known use of the 3270 screensThis was immediately accepted by our partners, who developed the Salesforce and Cast Iron applications
A static array like this:01 MYARRAY ARRAYRECORD[10];Is allocated in storage like a COBOL program wouldA dynamic array, distinguished by not having level numbers:myArrayarrayRecord[10];Is not allocated, but must be explicitly defined to avoid null pointersEGL generation for COBOL does not support multidimensional, dynamic arraysSince the arrays cannot be structured, you can effectively only have single level arrays in your service definitionsAll this is not considered an error, but a design, by IBM, even though it is deeply unintuitive for a legacy programmer
Performance and sizeMinimise the number of “round trips” to complete a service, i.e. combine operations to minimise calls and performance issuesAs a general quick rule of thumb combine service operations that execute in <5ms because of low overhead “ping”, split service operations that execute over >5ms where possibleOperations that take this long affect the concurrent usage of the service.Transactionality and StateTry to avoid designing services with transient states between operationsThis impacts on performance and also failure recovery.If the operation is to change data they it should be performed as part of one transaction (never more than one)Conversely do not put too much into one operation. Consider to split operations into individual services rather than handling a list to ensure parallel processing Business suitabilityStrive for a single operation to complete a whole business process stepSafe operationsTry to use Safe where possible for performance and scalabilityRead only or reentrantIdempotencyof operationsTry to use idempotent services wherever possible to minimize impact on businessIdempotent: (Mathematics) an element of a set which is unchanged in value when operated on by itself
ProcessIntegrityActive:The service invocation affects the state of the process e.g. update_stock_record by 500 units. If this service fails it DOES affect the state of the process within normal operational timeframeSafe: The service can be run safely without any impact on the business operation directly. It is basically a READ ONLY operation. Using read only operations reduces application errors and is useful for building scalability through caching. (i.e. you can cache many read only requests as found in typical web site URI requestsIdempotent: The service can be called any number of times and return a result without affecting the overall Process state and are safe to run (do not change any state) e.g. Get part list queryIf this service fails it DOES NOT affect the state of the process within normal operation timeframeData Integrity Transactionality and states:The service executes a specific business transaction. The service granularity is such that each service can be run independently of each other for maximum performance and decouple the data dependency between each service operation e.g. create_PO and validate_PO (see granuality)Data Synchronisation: The service executes a support function to maintain or deliver data to and from systemsLogic: The specific rule for the service is maintainedSecurity Integrity Security policies, processes and technologies
"Irrational exuberance" is a phrase used by the then-Federal Reserve Board Chairman, Alan Greenspan, in a speech given at the American Enterprise Institute during the Dot-com bubble of the 1990sThe phrase was interpreted as a warning that the market might be somewhat overvalued
AdoptionTechnology BandwagonA noted common trend where corporations decide to jump on the new technology bandwagon without careful consideration whether this technology have any contribution to the business.So, What’s New?Lack of understanding of the differences between SOA and previous computing paradigms drives skeptical to claim that SOA is just a name for same old techniques. The result is opposition to the adoption of SOA even if such adoption benefits the business.The Big BangThis antipattern is also known as “Bite more than you can chew.” It is observed when SOA is viewed as a panacea, leading to a push to change all the enterprise systems and architecture at once. Such a big bang adoption could result in failures that are then unjustly blamed on SOA.DesignBlob A large class with too many attributes and is the “heart” of the systemPoltergeists Unnecessary classes and too much abstractionStructureSpaghetti code Program code that has no structure (lots of goto statements)Stovepipe systemsApplications are unique and isolated
TechnologyWolf ticketA technology that claims openness, but has no conformance standard testContinuousobsolescenceThe attempt to use the latest releaseReuseCut-and-pasteSoftware errorsareduplicatedGolden hammerForcing everything to fit a selected tool
Identification & DesignWeb service = SOAWhen architects equate SOA with Web services they run the risk of replacing existing APIs with Web services without proper architecture. This will result in identifying many services that are not business aligned.The Silo ApproachIn this antipattern, services are identified based on isolated applications, thus same services are identified by different groups with different names. As a result, no common services or service sharing are realized.MisbehavingRegistriesDuplicate service registries and overlapping, unclear ownerships result in governance nightmare and runtime confusion, potential bad performance, and unplanned costs due to duplication.Realization Chatty ServicesThis antipattern describes a common mistake developers usually make when they realize a service by implementing a number of Web Services where each communicates a tiny piece of data. This will result in the implementation of a large number of services leading to degradation in performance and costly development.Point-to-point ServicesUsing XML or SOAP over HTTP between applications replacing middleware by point to point web services results in the emergence of point-to-point integration as the defacto integration pattern.Component-less ServicesBy jumping into development and implementation of Web services without having a clear association with owning components results in unstructured and undisciplined architecture (non strict layering). This leads to inflexibility behind the service interface and in preserving the legacy limitations of existing applications and packages.
The "Same old, Same old" PhenomenonWhile many of the good architecture principles (such as high cohesion and low coupling) still apply to an SOA, there are quite a few differences between SOA and its Object-oriented (OO) and Component-based architecture precursorsThe "Big Bang" ApproachChange everything, all at once, so that one day you'll come in, flip a switch, and everything will be one big, happy SOAService FiefdomsHere, instead of taking an enterprise view to the SOA transformation, each vertical silo within the company goes off on their own and recreates their applications as services within an SOA. In this case, there is the potential for a lot of duplication of effort due to the lack of an enterprise view. Furthermore, this fragmented approach to the SOA transformation often fails to create reusable organizational assets, which is one of the key benefits of undergoing the transformation that leads to higher organizational efficiency and improved cost effectiveness.
TechnophiliaInstead of the SOA initiative starting with the study of the processes that drive the business, the technology that will power the SOA becomes the apex of the effort. Instead of representing the business, the drivers of the SOA initiative get too wrapped up in technology specifics such XML, Web Services, determining which versions of the various technology standards to use, deciding how much BPEL will be needed, etc.Bloated ServicesIn this case, services within the SOA become "blobs" with one huge interface and lots of parameters. Each service does several things and so the degree of fragility of each service is much higher than it should be. This results in a system that is harder to maintain, enhance and test.Anorexic ServicesHere we have too many fine-grained services that also results in a system that is hard to use and maintain. It has the additional disadvantage of leading to a 'rude' system that hogs network bandwidth due to a phenomenon called "service chattiness," in which services are continuously communicating with each other.HyperBrokerageIn this case all service invocations go through a broker, even those that are internal to the system and do not need to be decoupled. As a result, performance and scalability suffer with the broker essentially becoming a system choke point. This is an example of taking loose coupling to its extreme
"Wild West" or “Rogue” ServicesExtremely difficult to gain control overServices proliferate wildly because no formal service-definition process is in placeNobody knows how many services are in place, where they are or what they doNeed to reconcile registered vs. deployed services (e.g. “rogue” services)No leverage and no reuse – defeats a major benefit of SOA"Duplicated” ServicesSuperficially effective but limited real savingsToo large and contain services often duplicated twice or moreDuplicated Web Services must be identifiedRewarding mechanisms for reusing and creating reusable services is vagueLittle reuse while maintenance costs multiply, much higher than needed"Shelfware” ServicesA waste of resource, won’t deliver benefitsFew applications actually use the Public servicesMost applications remain as they areApproved and deployed services have not been usedNeed to report on unused Web services to limit shelf-wareIT organization does not have information on usage of deployed servicesNo way to visualize what services are runningReuse is a promise that's never kept
Unsecure ServicesLimits service use and business opportunitiesServices are not used internally because they cannot be trustedSecurity threats from external accessCannot open up services to customers, partners, and suppliers due to lack of securityService usage is cumbersome due to multiple Authentication and Authorization systems needed to give partners accessRisk of security breaches No leverage and no reuse of servicesRigid ServicesRoadblock to agile, flexible business processesNo automatic selection of service providers based on rulesBusiness rules are hard-coded in applications and services—difficult and expensive to changeSOA cannot respond in real time to service problems—services are not reliableNeed to be able to dynamically determine service to use based on observed qualities of service (QoS)Need to be able to dynamically determine service to use based on message content or subscriptionPromise of reduced maintenance costs with SOA is not realizedNo leverage and no reuse of servicesIneffective Service ManagementServices must be managed as resourcesPoor performance of service endpoints; SLAs not respectedProduction disruption when changing service versions or adding / removing service endpointsServices must be managed as any other resourceWeb services must be monitored and issues mediated.Web service failures can affect many business applications and processes
COBOL was born to optimize administrative computingJava and XML was born for dynamic interfacesIttakes a lotofsweat to reconcilethemData in COBOL workingstorageallocatedimplicitly by thecompilerData in Java must beallocatedexplicitly by the programmerOtherwiseyougetnull pointer exceptions
There are many ways to integrate mainframe systems into service oriented architecturesA lot of best practices are to be found in the technical solutionsReuse your skills, tools and experienceThe standards and protocols will continue to change in the future anywayYou need a close cooperation between business stakeholders, architects, developers and technologists to make it workThe worst practices are found when application, design and architecture on one hand does not take the requirements of the technologies and the infrastructure seriouslyAll the mistakes have already been made and are extensively documentedI did not need to expose all our own mistakesThey were already made by someone elseLearn from the practical experience of others and from the research that has generalized it