Beyond Horizontal Scalability: Concurrency and Messaging Using Spring
1. Beyond Horizontal Scalability:
Concurrency & Messaging Using Spring
Bruce Snyder, Senior Software Engineer, SpringSource/VMware
Friday, July 8, 2011
2. Problem Statement
• Software systems are growing larger than ever before
– Virtual Machines
– Physical Machines
– More CPUs
– Multi-core CPUs
– More requests
– More data
• With so many hardware changes, shouldn’t our software
practices change too?
Friday, July 8, 2011
4. Typical Code Construction
• Command-and-control scheme
– One method calls another
– Blocking calls
• Assumptions
– One task takes place at a time
– The order of operations in known
– The provider of a particular function is known
– All execution happens in a single JVM
4
Friday, July 8, 2011
5. Typical Component Interaction
• Single memory space
• Using the call stack
• Sequential execution
• Assumptions
– No separation between:
• Knowing what needs to happen next
• Knowing which method to invoke
– Interactions are always known at compile time
5
Friday, July 8, 2011
6. Typical Assumptions
• All work must take place in a single call stack
• Example:
– Receive request
– Verify data
– Save data
– Query data
– Generate PDF
– Send email
– Render response
6
Friday, July 8, 2011
7. • The higher the number of assumptions, the more tightly
coupled the system
7
Friday, July 8, 2011
11. Typical Assumptions
• All work must take place on a single machine
• Example:
– Receive request
– Verify data
– Save data
– Query data
– Generate PDF
– Send email
– Render response
11
Friday, July 8, 2011
12. Treating the Symptoms
• Horizontal scale is good to a point
– Treat the symptoms instead of fixing the cause of the illness
• Throughput of functions should not equal response time!
• This is not the case at forward-thinking companies
– Amazon.com
12
Friday, July 8, 2011
13. The Problem With Distributed Systems
• Most developers don’t understand them
– Call stack provides specific interaction style
– Most object-oriented systems focus on structure vs.
interaction
• Interaction becomes more important than structure
– Mediator pattern - encapsulates how objects interact
– Observer pattern - notifies dependent objects of state change
• The real point of service-orientated design
– Composition rules are rather simple
– Loosely coupled interaction becomes very important
13
Friday, July 8, 2011
14. Is There a Better Solution?
• Insert a level of indirection
– Remove direct interaction between components
– Extract the interaction into a separate element
– Enterprise Integration Patterns (EIP)
• http://enterpriseintegrationpatterns.com/
• Simplify the rules of component interaction
– Remove the coordination and continuation style of interaction
– Just send data
14
Friday, July 8, 2011
15. Concurrency
• Concurrency is a style of asynchronous execution
15
Friday, July 8, 2011
16. Concurrency With Spring
• Spring provides the TaskExecutor interface
– Abstracts the execution of a Runnable, i.e., asynchronous work
– SimpleAsyncTaskExecutor
• Creates a new thread for each task
– ThreadPoolTaskExecutor
• Uses a JDK 1.5 ThreadPoolExecutor to create a pool of threads
– org.springframework.scheduling.concurrent package
• Many scheduling related utility classes
– WorkManagerTaskExecutor
• Delegates to a JCA 1.5 CommonJ WorkManager
16
Friday, July 8, 2011
17. JDK ExecutorService
ExecutorService executorService = Executors.newSingleThreadExecutor();
executorService.execute(task);
executorService.shutdown();
executorService = Executors.newFixedThreadPool(10);
executorService.execute(task);
executorService.shutdown();
int corePoolSize = 5;
int maxPoolSize = 10;
long keepAliveTime = 5000;
executorService = new ThreadPoolExecutor(corePoolSize, maxPoolSize,
keepAliveTime, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
executorService.execute(task);
executorService.shutdown();
17
Friday, July 8, 2011
19. Spring TaskExecutor
public class SpringConcurrencyExample {
! @Autowired
! private Runnable task;
! @Autowired
! private SimpleAsyncTaskExecutor simpleAsyncTaskExecutor;
! @Autowired
! private ThreadPoolTaskExecutor threadPoolTaskExecutor;
!
! public void runExamples() {
! ! simpleAsyncTaskExecutor.execute(task);
! ! threadPoolTaskExecutor.execute(task);
! }
!
! public static void main(String[] args) {
! ! ApplicationContext context =
! ! ! new ClassPathXmlApplicationContext("/META-INF/spring/executor-context.xml",
SpringConcurrencyExample.class);
! ! SpringConcurrencyExample example =
context.getBean(SpringConcurrencyExample.class);
! ! example.runExamples();
! }
}
19
Friday, July 8, 2011
20. Messaging
• Messaging is a style of communication
• Often used for integration purposes
20
Friday, July 8, 2011
21. Messaging With Spring
• JMS Support
– JmsTemplate for sync send and receive
– DefaultMessageListenerContainer and
SimpleMessageListenerContainer for async receive
• javax.jms.MessageListener
• org.springframework.jms.listener.SessionAwareMessageListener
• org.springframework.jms.listener.adapter.MessageListenerAdapter
• AMQP Support
– RabbitTemplate for sync send and receive
– SimpleMessageListenerContainer for async receive
• org.springframework.amqp.core.MessageListener
• org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter
21
Friday, July 8, 2011
23. Spring JmsTemplate
Synchronous
// Use the default destination
jmsTemplate.convertAndSend("Hello World!");
// Use a different destination
jmsTemplate.convertAndSend(“TEST.BAR”, “Hello World!”);
// Use a different destination
String textMessage1 = (String) jmsTemplate.receiveAndConvert();
// Use a different destination
String textMessage2 = (String) jmsTemplate.receiveAndConvert(“TEST.BAR”);
23
Friday, July 8, 2011
26. Spring RabbitTemplate
Synchronous
// Use the default destination
rabbitTemplate.convertAndSend("Hello World!");
// Use a different destination
rabbitTemplate.send(“TEST.FOO”, “TEST.FOO”, message);
// Use a different destination
Message message1 = rabbitTemplate.receiveAndConvert();
// Use a different destination
String textMessage2 = (String) rabbitTemplate.receive(“TEST.FOO”);
26
Friday, July 8, 2011
28. Best of Both Worlds
• What if concurrency and messaging was provided
together?
28
Friday, July 8, 2011
29. Spring Integration
• A framework for integration
• Styles of integration
– Intra-application integration
– Inter-application integration
– External system integration
29
Friday, July 8, 2011
31. Spring Integration
• Provides both concurrency and messaging
– Message Endpoints
• Connections between services
– Channel Adapters
• Adapter between application and message broker
– Messaging Gateways
• Provides uni-directional or bi-directional messaging
– Service Activators
• Invokes a services based on an incoming message
– Routers
• Determines where to dispatch a message
– Splitters and Aggregators
• Breaks up a message and reassembles it after processing
31
Friday, July 8, 2011
32. Spring Integration
• Supports
– AMQP
– Email
– File system
– Gemfire
– JMS
– JMX
– MongoDB
– Redis
– Spring Batch
– Testing
– Web Services
32
Friday, July 8, 2011
43. Your Coffee Shop Does Not Use 2PC
• Order is accepted
• Cup is labeled and placed in the queue
• Money is exchanged
• Coffee drink is processed
• Multiple baristas = competing consumers
• Drinks are processed out of order = correlation id (label)
• Cannot pay for drink = discard (write-off)
• Erroneous drink = retry
• Drink machine fails = compensating action (refund, etc.)
43
Friday, July 8, 2011
44. Conversation Pattern
• Interaction between two parties
– Short synchronous interaction
– Longer asynchronous interaction
• Other examples
– Amazon.com
44
Friday, July 8, 2011
