Building With Open Mq V2

Using OpenMQ
Linda Schneider
Technical Lead
Sun Microsystems,
Inc.

What will be covered ?
• An introduction to OpenMQ.
• A Customer example.
• Basic customer requirements.
• Building a piece of the infrastructure.

• Warnings:
> No in depth coverage
> Assumes basic JMS knowledge

Copyright Sun Microsystems Inc. All Rights Reserved. [2]

What is OpenMQ ?


Overview
• Allows heterogenous applications to reliability and
asynchronously pass data between each other.
• Open Source Java Message Service (JMS)
implementation (+ additions)
• Default Messaging Provider for Glassfish
• Useful on its own for standalone JMS applications


Overview (cont.)
• Enterprise level quality (>8 years in development)
• Open Source since JavaOne 2006
• Available as a supported product: Sun Java System
Message Queue (SJSMQ)


What this looks like ....

A set of loosely coupled applications talking through
OpenMQ apis


Ok – maybe a little about JMS
Some basic concepts of JMS:
• Producers and Consumers
• Destinations
• Consumer Types
• Reliability


What are Producers and Consumers?
• Producers
> Create messages which are given to the “provider” (e.g.
OpenMQ) to deliver.
> Once the message has been handed off to the provider it
goes off and does other processing.
• Consumers
> Are sent messages from the “provider” to process
> Notify the provider when they are done with the message.


So, what are destinations ?
• Named buckets to hold groups of messages.
• Messages are sent to “destinations”
> Topics: each messages goes to all consumers
> Queues: a message goes to only one consumer


What are the types of Consumers ?
• Receivers are delivered messages from a Queue
• Subscribers are delivered messages from a Topic
> Normal subscribers only receive messages sent while they
are running
> Messages for durable subscribers are maintained when
the subscriber is not running


And what does reliability mean ?
• Messages can be set to be persistent or non-
persistent
> Persistent messages are stored
> Non-persistent messages are not and could be lost if a
server crashed
• Messages can be transacted or non-transacted
> Transactions guarantees that a set of messages will be
processed or not processed (not partially processed) if a
server fails
> XA transactions allow other resources (e.g. databases) to
be processed in a group with messages. [Requires an
Application Server]

Tell Me More
• Developer and User discussion forums
• Stable builds with product releases
• Early access, promoted builds available
> New features, and fixes

• Dual license support (GPL v2 and CDDL)
• Open source version of Java MQ is available from
http://mq.dev.java.net


Using OpenMQ


An example
To mimic problems faced in designing applications, an
example:
• Represents a complex system with loosely
connected applications
• Utilizes various types of messaging
• Is easy to understand
• Is at least minimally interesting


Our Example: Santa Claus, Inc
Why, you ask ?

• Even if you don't believe in Santa Claus, you must
still understand that delivering all those presents
would be a daunting task
• And while its not Christmas which comes but once a
year, but requires year round planning and
preparation.
• Just because Santa Claus lives at the North Pole,
doesn't imply he can't use technology

Overall System Requirements
Santa Claus, Inc. software applications need do the
following:
• Handle gift selection and delivery
• Manage resources e.g.
> gifts
> reindeer
> Elves
• Track general status information
> how many days before Christmas
> etc.

What are we doing:
• Focusing on handling christmas gift processing
• Steps to design it include:
> Determining the high level operation
> Coming up with the name and type of destinations
> Determining models used for the messaging
> Determining load characteristics
> Looking at code for some components


Defining the High Level
Operation of the System


What do we need to do ?


What do we need to do ? (cont.)


How can Santa Do it ?


What are the
destinations ?


The Child Queue


The Naughty/Nice Queues


The Wrap Queue


The Stuff to Pack Queue


Select-a-gift Queues


Topic LogChild


A quick overview to
design patterns


Some basic design patterns:
Pattern Description
Message is sent to another application
Request/Reply who sends back a response
Messages go through several iterations,
the message is persisted at key points
where processing it again would be
Step Operations expensive
Broadcast One message goes to many consumers
Multiple consumers send messages to a
Conduit single destination
Batch Messages are processed in a chunk
Messages must be processed within a
short period of time (e.g. under an hour)
Time Critical/Sensitive and can not be lost


More things to think about:
• Use persistent messages if it can not afford to be lost
• Use non-persistent messages for:
> Non-critical step messages (when it can be repeated
> Request/Reply
> Anytime a message can be lost on a server crash
• Use durables for Topics when it may need to be
retrieved later
• Use normal or XA transactions when multiple things
must process together:
> XA if it includes other resources like databases

Processing Queue Child
• Conduit: many producers to one queue
• Persistent: would be time consuming to lose message


Processing Naughty and Nice
• Step Pattern: one step of it
• Naughty Queue: Non-
Persistent
> its OK if a child who is bad
misses their coal
• Nice Queue: Persistent.
> They must get their present.


Processing Nice
• Step Pattern: more steps of it
• Multiple resources so XA


Processing Nice (Select a gift)
• Request/Reply Pattern
• Non-persistent
• Action repeated on failure


The Wrap Queue
• Step operation
• Persistent: end of an expensive set of steps that they
don't want to repeat


The Log Child Topic
• Broadcast Pattern
• Persistent because santa wants his database
accurate


In this example:
• The batch pattern was not used
> Santa does use It for processing HR updates for the
elves
• The time sensitive/critical data pattern was not used:
> Santa does use it during present delivery on christmas
eve to track where he is
• Because he has no time sensitive/critical data,
reliability is important however data availability isn't
for gift processing


Performance Requirements
• 22 billion kids
• 364 days for preparation (since christmas is taken)
> 31,526,000 seconds
• 70 children/second must be processed
• Assume 60% are “nice”
• Assume 40% downtime to cover outages and normal
processing (so goal is approx 100 kids/second)


Performance Requirements (cont)
• Naughty Kids use
> 1 Persistent queue (child)
> 1 Non-persistent queue (naughty)
> 1 Persistent Topic (log child)
• Nice Kids Use:
> Persistent Queue (child)
> Persistent Queue (nice)
> 2 Non-Persistent Queues (Inventory request and reply
queues)
> Non-persistent queue (Wrap)
> 1 Persistent Topic (log child)

The cold hard requirements
• Messages:
> Child: 100 msgs/second (persistent)
> Naughty: 40 msgs/second (non-persistent)
> Log Child: 100msgs/second (persistent)
> Nice: 60 msgs/second (persistent in XA transaction)
> Inventory request/reply: 60 msgs/second *2 (non-
persistent)
> Wrap: 60 msgs/second (persistent)
• TOTALS:
> Persistent: 380 msgs/second
> Non-persistent: 160 msgs/second

Some Sample Code


Sending the “Child” message
public void doPost(HttpServletRequest request,
HttpServletResponse response) throws ServletException,
IOException {
// retrieve initial context (ic)
QueueConnectionFactory qcf = (QueueConnectionFactory)
ic.lookup(quot;MyConnectionFactoryquot;);
Queue destQueue = (Queue)ic.lookup(quot;Childquot;);
QueueConnection connection = qcf.createQueueConnection();
try {
QueueSession session = connection.createQueueSession(
False, Session.AUTO_ACKNOWLEDGE);
QueueSender sender = session.createSender(destQueue);
MapMessage msg = session.createMapMessage();
// Set each item
msg.setString(“firstname”,
request.getParameter(“firstname”));
// … retrieve other properties …;
sender.send(msg);
} finally {
connection.close();
}
}


Processing the “nice” queue
public void onMessage(Message inMessage) {
TextMessage msg = null;
try {
//Message is of type text and has a unique child id
msg = (TextMessage) inMessage;
String id = msg.getText();
String[] list = db.getList(id); // makes SQL call
String item = null;
if (list == null) { // no list, send request
String item = getListItem(); //next slide
} else {
item = list[0];
}
//update inventory
db.updateInventory(item, id);//makes SQL call
// put on packing list
pack(item, id);
} catch (Exception e) { // things went wrong roll back
mdc.setRollbackOnly();
}
}


Processing the “nice” queue (step 2)
public String getListItem(String childid)
throws Exception {
QueueConnectionFactory factory =
jndiContext.lookup(“MyQueueFactory”);
QueueConnection qc = factory.createQueueConnection();
qc.start();
QueueSession session = qc.createSession(true,
Session.AUTO_ACKNOWLEDGED);
Queue q = session.createQueue(“RandomPresent”);
Queue reply = session.createTemporaryQueue();
// get sender and receiver
QueueSender sender = session.createSender(q);
QueueReceiver receiver = session.createReceiver(q);
//send message and wait
TextMessage m = session.createTextMessage(childid);
m.setJMSReplyTo(reply);
//send the message
sender.send(m);
TextMessage back = (TextMessage)
receiver.receive(60*1000); // wait a minute
if (back == null) { didn't get anything
throw new Exception(“Nothing”);
return back.getText();
}

Processing the “nice” queue (step 3)
public String pack(String item, String child_id)
throws JMSException
{
QueueConnectionFactory factory =
jndiContext.lookup(“MyQueueFactory”);
QueueConnection qc = factory.createQueueConnection();
QueueSession session = qc.createSession(true,
Session.AUTO_ACKNOWLEDGED););
Queue q = session.createQueue(“Pack”);
// get sender
QueueSender sender = session.createSender(q);
//send message
MapMessage m = session.createMapMessage(childid);
m.setString(“child_id”, child_id);
m.setString(“present”, item);
sender.send(m);
}


You'll need to fill in the
rest


More Information


OpenMQ -- More Information
•Visit the product webpage
>http://sun.com/software/products/message_queue
•Join the Open Message Queue project
>https://mq.dev.java.net
•Browse the product documentation
>http://docs.sun.com/app/docs/coll/1307.3
•Take the free technical training
>http://www.sun.com/training/catalog/courses/WMT-SMQ-1491.xml


Related Information
•Java Composite Application Platform Suite
>http://sun.com/software/javaenterprisesystem/javacaps/
•Java System Identity Manager
>http://sun.com/software/products/identity
•Project GlassFish
>https://glassfish.dev.java.net/
•The Aquarium, A community forum
>http://blogs.sun.com/theaquarium/


Thank You!

Using OpenMQ
Linda Schneider
linda.schneider@sun.com

Building With Open Mq V2

Recommandé

Recommandé

Contenu connexe

Similaire à Building With Open Mq V2

Similaire à Building With Open Mq V2 (20)

Plus de Eduardo Pelegri-Llopart

Plus de Eduardo Pelegri-Llopart (20)

Dernier

Dernier (20)

Building With Open Mq V2