In this session you will learn what Spring Integration and Spring Batch are all about, how they differ, their commonalities, and how you can use Spring Batch and Spring Integration together. We will provide a short overview of the Enterprise Integration Patterns (EIP) as described in the highly influential book of the same name. Based on these patterns, we will then see how Spring Integration enables the development of Message-driven applications. This allows you to not only modularize new or existing applications but also makes it easy to integrate with external systems. This session will also introduce Spring Batch. Spring Batch addresses the needs of any batch process, be it complex calculations in large financial institutions or simple data migration tasks as they exist in many software development projects. We will cover what Spring Batch is, how Spring approaches the concepts of batch and how Spring handles scaling batch processes to be able to handle any volume of data. You will also see how Spring Integration and Spring Batch maximize the reuse of the integration support provided by the core Spring Framework. In addition to providing a robust, proven foundation, this also flattens the learning curve considerably to all developers already familiar with Spring.

1. Introduction to
Spring Integration and Spring Batch
Gunnar Hillert, Spring Integration Team
Gary Russell, Spring Integration Team
Twitter: @ghillert, @gprussell
© 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission.

2. What we will cover...
• Spring Integration
• Spring Batch
• Using Spring Batch and Spring Integration together
2

3. Spring Integration
3

4. Integration Styles
JVM JVM
Core Messaging
EAI
• Business to Business Integration (B2B)
• Inter Application Integration (EAI)
• Intra Application Integration
B2B
External Business
Partner
4

5. Integration Styles
• File Transfer
• Shared Database
• Remoting
• Messaging
5

6. Common Patterns
Retrieve Parse Transform Transmit
6

7. Enterprise Integration Patterns
• By Gregor Hohpe & Bobby Woolf
• Published 2003
• Collection of well-known patterns
• http://www.eaipatterns.com/eaipatterns.html
• Icon library provided
7

8. “ Spring Integration provides an extension
of the Spring programming model
to support the well-known enterprise
integration patterns.
8

9. History
• Nov 13, 2007 - First commit: Nov 13, 2007
• Jan 23, 2008 - Spring Integration 1.0.0.M1
• Nov 26, 2008 - Spring Integration 1.0.0
• Nov 22, 2010 - Spring Integration 2.0.0
• Jan 6, 2012 - Spring Integration 2.1.0
• Sep 21, 2012 - Spring Integration 2.2.0.RC1
• 2013 - Spring Integration 3.0
9

10. What is Spring Integration?
• Light-weight messaging framework
• Provides an adapter-based platform
• Pipes and Filters at the core of Spring Integration’s architecture
– Endpoint (Filter)
– Channel (Pipe)
– Message
10

11. Advantages
• Provides building blocks to implement systems that are:
– are loosely Coupled (Logically or Physically)
– are Event Driven (EDA)
– have a staged event-driven architecture (SEDA)
• Sophisticated support for synchronous / asynchronous messaging
11

12. Configuration
• XML Namespace Support
• Annotation Support (e.g. @Transformer, @Router, @ServiceActivator)
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:int="http://www.springframework.org/schema/integration"
xsi:schemaLocation="..">
...
<int:channel id="orders"/>
<int:splitter input-channel="orders" expression="payload.items"
output-channel="drinks"/>
<int:channel id="drinks"/>
<int:router input-channel="drinks"
expression="payload.iced ? 'coldDrinks' : 'hotDrinks'"/>
...
</beans>
12

13. Spring Integration DSLs
• Scala
val messageFlow =
filter {payload: String => payload == "World"} -->
transform { payload: String => "Hello " + payload} -->
handle { payload: String => println(payload) }
messageFlow.send("World")
• Groovy
def builder = new IntegrationBuilder()
def flow = builder.messageFlow {
transform {payload->payload.toUpperCase()}
filter {payload-> payload =="HELLO"}
handle {payload->payload}
}
assert flow.sendAndReceive("hello") == "HELLO"
assert flow.sendAndReceive("world") == null
13

14. Deployment Options
• Embedded as a library
• Stand-alone deployment
• War deployment
14

15. What is in a Message?
• Unit of information
• Encapsulates data
• Passed between endpoints
• Consists of headers
– contains data relevant to the messaging system
• and a payload
– actual data for the receiver
– depending on use-cases: POJO instances or serialized data
15

16. What is in a Message?
package org.springframework.integration;
public interface Message<T> {
MessageHeaders getHeaders();
T getPayload();
}
16

17. Message Headers
• Message ID (automatically generated UUID)
• Timestamp
• Correlation Id
• Reply Channel
• Error Channel
• Expiration Date
• Priority
• ...
• Add your own headers using a Header Enricher
17

18. Function of a Message
• Command Message
C
• Event Message
E
• Document Message
D
18

19. What is a Channel?
• Channels connect producers and consumers (decoupling)
• MessageChannel Interface:
– PollableChannel (Polling Consumer)
– SubscribableChannel (Event Driven)
• Implementations:
– DirectChannel
– PublishSubscribeChannel
– QueueChannel
<int:channel id="input">
– PriorityChannel <int:queue capacity="10"/>
</int:channel>
– RendezvousChannel
– ExecutorChannel
19

20. What is an Endpoint?
• Polling or event-driven
• Inbound or outbound
• Unidirectional (Channel Adapter) or bidirectional (Gateway)
• Internal or external (application context)
<inbound-channel-adapter/>
<outbound-channel-adapter/>
<inbound-gateway/>
<outbound-gateway/>
<gateway/>
<service-activator/>
20

21. Router
• Message Router
• Content-based router
• Recipient list router (dynamic)
• Payload type router
• Header value router
• Exception type router
21

22. Transformer
• Delegating via ref/method
• Spring Expression Language
• Groovy, JRuby, Jython, JavaScript
• Object-to-JSON / JSON-to-Object
• Payload serializing/deserializing
• File-to-bytes, File-to-String
• JAXB, JibX, Castor, XMLBeans, Xstream
• XPath, XSLT
• Object XML Marshalling/Unmarshalling (Spring OXM)
• ...
22

23. Spring Integration Components
• Claim Check (In/Out) • Service Activator
• Content Enricher • Scripting support (JSR 223)
– Header Enricher – Ruby/JRuby, Javascript ...
– Payload Enricher • Groovy
• Control Bus • Message History
• Delayer • Message Store
• JMX Support – JDBC, Redis, MongoDB,
• Message Handler Chain Gemfire
• Messaging Bridge • Wire Tap
• Resequencer • ...
23

24. Adapters
• AMQP/RabbitMQ • MongoDB • Stored Procedures
• AWS* • POP3/IMAP/SMTP • TCP/UDP
• File/Resource • Print* • Twitter
• FTP/FTPS/SFTP • Redis • Web Services
• GemFire • RMI (SOAP or POX)
• HTTP (REST) • RSS/Atom • XMPP
• JDBC • SMB* • XPath
• JMS • Splunk* • XQuery*
• JMX • Spring Application
• JPA Events • ...
* Spring Integration Extensions Project
24

25. Tooling - Spring Tool Suite (STS)
• Namespace Support
• Visualization
• 4 Spring Integration specific STS Templates
– Simple Template (Core Components only)
– File Polling Template (File Adapter)
– War Template (Uses Twitter Adapter)
– Adapter Template (Create your own components)
25

26. Tooling - IntelliJ IDEA
• Spring Integration support since IDEA 10.5
• Namespace Support
• Bean visualization
• IntelliJ IDEA 12 (Preview) supports Spring Integration 2.1 & 2.2
26

27. Tooling - C24 Integration Objects (C24iO) Studio
• Supports a variety of standards:
– SWIFT
– ISO20022
– FIX
• Namespace Support:
<int-c24:transformer/>
<int-c24:validating-header-enricher/>
<int-c24:marshalling-transformer/>
...
• http://www.c24.biz/io-spring.html
27

28. Café Demo
28

29. Café Demo - Flow
29

30. Samples
• https://github.com/SpringSource/spring-integration-samples
• Contains 50 Samples and Applications
• Several Categories:
– Basic
– Intermediate
– Advanced
– Applications
30

31. Books
• Just Spring Integration
• Pro Spring Integration
• Spring Integration in Action
31

32. Source Code
• https://github.com/SpringSource/spring-integration
• https://github.com/SpringSource/spring-integration-samples
• https://github.com/SpringSource/spring-integration-extensions
• https://github.com/SpringSource/spring-integration-templates
• https://github.com/SpringSource/spring-integration-dsl-groovy
• https://github.com/SpringSource/spring-integration-dsl-scala
32

33. Contribute
• Post Question and Answers the Forums
– http://forum.springsource.org/forumdisplay.php?42-Integration
• Create Jiras
– https://jira.springsource.org/browse/INT
• Submit Pull Requests - Contributor Guidelines:
– github.com/SpringSource/spring-integration/wiki/Contributor-Guidelines
• New Spring Integration Extensions Repository
33

34. Spring Batch
34

35. Batch Jobs
Differ from online/real-time processing applications:
• Long-running
– Often outside office hours
• Non-interactive
– Often include logic for handling errors or restarts
• Process large volumes of data
– More than fits in memory or a single transaction
35

36. Batch and offline processing
• Close of business processing
– Order processing
– Business reporting
– Account reconciliation
• Import/export handling
– a.k.a. ETL jobs (Extract-Transform-Load)
– Instrument/position import
– Data warehouse synchronization
• Large-scale output jobs
– Loyalty scheme emails
– Bank statements
• 36

37. Job and Step
37

38. Chunk-Oriented Processing
• Input-output can be grouped together
• Input collects Items before outputting:Chunk-Oriented Processing
• Optional ItemProcessor
38

39. JobLauncher
39

40. Simple File Load Job
40

41. More Complex Use Cases
• It's very common to use an off-the-shelf reader
and writer
• More complex jobs often require custom readers
or writers
• ItemProcessor is often used if there's a need to
delegate to existing business logic
• Use a writer if it's more efficient to process a
complete chunk
41

42. Job and Step in Context
42

43. JobRepository and Batch Metadata
43

44. ExecutionContext
• We need to know where a failure occurred to restart a batch
process
• Job Repository metadata is used to determine the step at
which the failure occurred
• Application Code (in reader/writer) needs to maintain state
within a step (e.g. current chunk)
• Spring Batch can supply that data during restart to facilitate
repositioning
44

45. Common Batch Idioms
• Batch jobs typically process large amounts of homogeneous
input
• Makes iteration a common concern: Repeat
• Transient errors during processing may require a Retry of
an input item
• Some input may not be valid, may want to Skip it without
failing
• Some errors should fail the job execution, allowing one to fix
the problem and Restart the job instance where it left off
45

46. Spring Batch
• Spring Batch supports these common concerns
• Abstracts them in the framework
– Job business logic doesn't need to care about details
• Allows for simple configuration with pluggable strategies
46

47. Business Logic Delegation – Spring Application
47

48. Business Logic Delegation – Spring Integration
48

49. Spring Batch Admin
• Sub project of Spring Batch
• Provides Web UI and ReSTFul interface to manage batch
processes
• Manager, Resources, Sample WAR
– Deployed with batch job(s) as single app to be able to control &
monitor jobs
– Or monitors external jobs only via shared database
49

50. Scaling and Parallel Processing
• First Rule:
– Use the simplest technique to get the job done in the required
time
– Do not optimize/parallelize unnecessarily
• Options:
– Multi-threaded Step (single process)
– Parallel Steps (single process)
– Remote Chunking of Step (multi process)
– Partitioning a Step (single or multi process)
50

51. Using Spring Batch
and
Spring Integration
together
51

52. Launching batch jobs through messages
• Event-Driven execution of the JobLauncher
• Spring Integration retrieves the data (e.g. file system, FTP, ...)
• Easy to support separate input sources simultaneously
Inbound Channel Adapter Transformer
FTP
D
File C
JobLaunchRequest
JobLauncher
52

53. Providing feedback with informational messages
• Spring Batch provides support for listeners:
– StepListener
– ChunkListener
– JobExecutionListener
<batch:job id="importPayments">
...
<batch:listeners>
<batch:listener ref="notificationExecutionsListener"/>
</batch:listeners>
</batch:job>
<int:gateway id="notificationExecutionsListener"
service-interface="o.s.batch.core.JobExecutionListener"
default-request-channel="jobExecutions"/>
53

54. Externalizing batch process execution
• Use Spring Integration inside of Batch jobs, e.g.:
– ItemProcessor
– ItemWriter
• Offload complex processing
• Asynchronous processing support:
– AsyncItemProcessor
– AsyncItemWriter
• Externalize chunk processing using ChunkMessageChannelItemWriter
54

55. Learn More. Stay Connected.
At SpringOne 2GX:
• Building an Enterprise CRM with Grails and Spring Integration
• What's New in Spring Integration
• Spring Integration in the Wild
• Batch Processing and Integration on Cloud Foundry
• Building for Performance with Spring Integration & Spring Batch
• Spring Integration, Batch, and Data: Future Directions
• Managing and Monitoring Spring Integration Applications
• Java Batch JSR-352
55

56. Learn More. Stay Connected.
• Web: www.springintegration.org
• Twitter: @m_f_ @ghillert @z_oleg @gprussell
Questions?
Thank You!!
56