Note that the graphic does NOT depict to scale the relationship of Message headers and Properties to the message body (which can be up to 100MB)
Messages written to DASD datasets - queue contents are safe (persistent).
Movement of messages logged on separate DASD.
Messages that have been written (MQPUT) by an application will never be lost or discarded
messages destined for a local queue are written synchronously to the queue.
Messages destined to remote queue are written to a local intermediate queue (transmission queue).
Messages sent by a connected queue manager that cannot be delivered to the target queue are written to the dead letter queue.
No direct connections between programs Message queuing is a technique for indirect program-to-program communication. It can be used within any application where programs communicate with each other. Communication occurs by one program putting messages on a queue (owned by a queue manager) and another program getting the messages from the queue. Programs can get messages that were put on a queue by other programs. The other programs can be connected to the same queue manager as the receiving program, or to another queue manager. This other queue manager might be on another system, a different computer system, or even within a different business or enterprise.
There are no physical connections between programs that communicate using message queues. A program sends messages to a queue owned by a queue manager, and another program retrieves messages from the queue (see Figure 1). Figure 1. Message queuing compared with traditional communication
As with electronic mail, the individual messages that are part of a transaction travel through a network on a store-and-forward basis. If a link between nodes fails, the message is kept until the link is restored, or the operator or program redirects the message.
The mechanism by which a message moves from queue to queue is hidden from the programs. Therefore the programs are simpler.
Time-independent communication Programs requesting others to do work do not have to wait for the reply to a request. They can do other work, and process the reply either when it arrives or at a later time. When writing a messaging application, you need not know (or be concerned) when a program sends a message, or when the target is able to receive the message. The message is not lost; it is retained by the queue manager until the target is ready to process it. The message stays on the queue until it is removed by a program.
Small programs Message queuing allows you to exploit the advantages of using small, self-contained programs. Instead of a single, large program performing all the parts of a job sequentially, you can spread the job over several smaller, independent programs. The requesting program sends messages to each of the separate programs, asking them to perform their function; when each program is complete, the results are sent back as one or more messages. Event-driven processing Programs can be controlled according to the state of queues. For example, you can arrange for a program to start as soon as a message arrives on a queue, or you can specify that the program does not start until there are, for example, 10 messages above a certain priority on the queue, or 10 messages of any priority on the queue.
Message priority A program can assign a priority to a message when it puts the message on a queue. This determines the position in the queue at which the new message is added. Programs can get messages from a queue either in the order in which the messages appear in the queue, or by getting a specific message. (A program might want to get a specific message if it is looking for the reply to a request that it sent earlier.)
Security Authorization checks are carried out on each resource, using the tables that are set up and maintained by the WebSphere® MQ administrator. Use Security Server (formerly known as RACF®) or other external security managers on WebSphere MQ for z/OS®.
On WebSphere MQ on UNIX systems, Windows systems, and i5/OS®, a security manager called the Object Authority Manager (OAM) is provided as an installable service. By default, the OAM is active.
Data integrity Data integrity is provided by units of work. The synchronization of the start and end of units of work is fully supported as an option on each MQGET or MQPUT, allowing the results of the unit of work to be committed or rolled back. Sync point support operates either internally or externally to WebSphere MQ depending on the form of sync point coordination selected for the application.
Recovery support For recovery to be possible, all persistent WebSphere MQ updates are logged. In the event that recovery is necessary, all persistent messages are restored, all in-flight transactions are rolled back, and any sync point commit and backouts are handled in the normal way of the sync point manager in control. For more information on persistent messages, see Message persistence.
Program A and B communicate locally via Q1 – no channels involved.
Program A and C communicate remotely via Q2 – channels involved, using an XmitQ. The XmitQ allows Program A to operate as though Q2 were local (because the XmitQ is in fact local). So neither program need have any awareness of the queue manager topology or the underlying network details.
Real life example:
Mortgage Loan Request
-Verification of Employment
- Credit Report
- Bank Balance Inquiry
Final Mortgage Approval
- All results are in (one logical unit)
- E compiles the results and reports them
On Linux/UNIX, sample programs are in QMA MQ_INSTALLATION_PATH/samp/bin directory
On Linux/UNIX, sample programs are in QMA MQ_INSTALLATION_PATH/samp/bin directory
An application uses the MQCONN call to connect to a queue manager.
The application then uses the MQOPEN call to open a queue so that it can put messages on it.
A queue manager has a definition for each of its queues, specifying information such as the maximum number of messages allowed on the queue.
If the messages are destined for a queue on a remote system, the local queue manager holds them in a message store until it is ready to forward them to the remote queue manager. This can be transparent to the application.
Each queue manager contains communications software called the moving service component; through this, the queue manager can communicate with other queue managers.
The transport service is independent of the queue manager and can be any one of the following (depending on the platform):
Systems Network Architecture Advanced Program-to Program Communication (SNA APPC)
Transmission Control Protocol/Internet Protocol (TCP/IP)
Network Basic Input/Output System (NetBIOS)
Sequenced Packet Exchange (SPX)
Panah Message Flow nya salah???
When a message arrives on a transmission queue that satisfies the triggering criteria for that queue, a message is sent to the initiation queue, triggering the channel initiator to start the appropriate sender channel.
-> channels can be started automatically, based upon messages arriving on the appropriate transmission queue
The channel listener detects incoming network requests and starts the associated channel (Responder MCA)
CLUSTER contains three queue managers, QM1, QM2, and QM3.
QM1 and QM2 host full repositories of information about the queue managers and queues in the cluster.
QM2 and QM3 host some cluster queues, that is, queues that are accessible to any other queue manager in the cluster.
Each queue manager has a cluster-receiver channel called TO.qmgr on which it can receive messages.
Each queue manager also has a cluster-sender channel on which it can send information to one of the repository queue managers.
QM1 and QM3 send to the repository at QM2 and QM2 sends to the repository at QM1.
As with distributed queuing, you use the MQPUT call to put a message to a queue at any queue manager.
You use the MQGET call to retrieve messages from a local queue.
The sequence of processing is as follows:
The security exits are called after the initial data negotiation between both ends of the channel. These must end successfully for the startup phase to complete and to allow messages to be transferred.
The message exit is called by the sending MCA, and then the send exit is called for each part of the message that is transmitted to the receiving MCA.
The receiving MCA calls the receive exit when it receives each part of the message, and then calls the message exit when the whole message has been received.
There are times when you need alternative channels, perhaps for security purposes, or to trade off delivery speed against sheer bulk of message traffic
Any queue manager can send a message to any other queue manager in the same cluster without explicit channel definitions, remote-queue definitions, or transmission queues for each destination. Every queue manager in a cluster has a single transmission queue from which it can transmit messages to any other queue manager in the cluster. Each queue manager in a cluster needs to define only:
One cluster-receiver channel on which to receive messages
One cluster-sender channel with which it introduces itself and learns about the cluster
http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.con.doc/q015720_.htm
Transmission queues are transparent to the application. They are used internally by the queue manager
Sebaiknya memang aplikasi tidak mengambil data dari XMIT queue.
Jadi cara handle yang benar jika message tidak bisa dikirimkan ke remote queue dalam waktu tertentu dan kita akan lakukan sesuatu terhadap message tersebut (dalam hal ini dijadikan file text)
adalah dengan cara
.Pada saat membuat channel dispesifikasikan berapa kali MQ (MCA) akan retry mengirimkan message dan berapa lama akan menunggu untuk dikirim. Jika MCA melakukan retry sampai maksimum atau intervalnya sudah tercapai maka message akan dikirim ke dead-letter-queue atau dimasukan ke queue tertentu dispesifikasikan di header message.Ini attribute konfigurasi channel (hanya ada di receiver channel) tersebut: * Message retry count (MRRTY) * Message retry interval (MRTMR)
.Saran saya pada saat PUT message, dispesifikasikan message descriptor MQRO_EXCEPTION_WITH_FULL_DATA dan MQRO_DISCARD_MSG agar jika gagal dikirim dikirim ke queue tertentu. Perlu dispesifikasikan juga nama reply-to queue dan reply-to queue manager Berikut contoh program Java saat put, untuk C# tidak beda jauhMQMessage qMessage = new MQMessage();qMessage.format = MQC.MQFMT_STRING;qMessage.messageType = MQC.MQMT_REQUEST;qMessage.characterSet = 850;qMessage.replyToQueueName = "QUEUE_TEST";qMessage.replyToQueueManagerName = "QM_TEST";qMessage.report = MQC.MQRO_EXCEPTION_WITH_FULL_DATA + MQC.MQRO_DISCARD_MSG;
.Lalu buat program untuk GET message dari reply-to queue tersebut.
A WebSphere MQ client application and a server queue manager communicate with each other by using an MQI channel.
An MQI channel starts when the client application issues an MQCONN or MQCONNX call to connect to the queue manager
ends when the client application issues an MQDISC call to disconnect from the queue manager.
The input parameters of an MQI call flow in one direction on an MQI channel and the output parameters flow in the opposite direction.
WMQ V7 Clients http://www-01.ibm.com/support/docview.wss?uid=swg24019253
Notes:
1. Synchronous messaging only.
2. Only for Queues owned by the Server it is immediately connected to.
3. Requires an external transaction manager.
4. WebSphere MQ for z/OS servers require an additional Client Attachment license for MQ clients to connect into it. This also applies when connecting the new WebSphere MQ V6.0 MQ Explorer tooling into WebSphere MQ for zOS V6.0 deployments since it uses client channels – although there is no longer a need to install a client on machines running this release of MQ Explorer.
Create a queue manager, called QM1 for example: crtmqm QM1
Start the queue manager: strmqm QM1
Start MQSC commands: runmqsc QM1
This feature is not available on z/OS
WebSphere MQ MQI clients can be configured to look up a repository to obtain connection definitions using a pre-connect exit library.
A client application can connect to a queue manager using a client channel definition table (CCDT). Generally, the CCDT file is located on a central network file server, and have clients referencing it.
Since it is difficult to manage and administer various client applications referencing the CCDT file, a flexible approach is to store the client definitions in a global repository like an LDAP directory, a WebSphere Registry and Repository or any other repository.
Storing the client connection definitions in a repository makes managing client connection definitions easier, and applications can access the correct and most current client connection definitions.
During the MQCONN/X call execution, the MQI client loads an application specified preconnect exit library, and invokes an exit function to retrieve connection definitions. These are then used to establish connection to a queue manager. The details of exit library and function to invoke are specified in the mqclient.ini configuration file.
Emphasize that Client Channel Definition Tables are used in all but the simplest of cases. They are created by an administrator when client channels are defined, and allow connect options to be specified administratively as opposed to options that can be overridden by the developer (using MQCONNX) or the end user (using the MQSERVER environment variable). In addition, there are channel definition options available that can ONLY be set using a CCDT.
A CCDT is somewhat analogous to a Connection Factory with JMS or XMS. In fact, a JMS or XMS Connection Factory can reference a CCDT, the two working together.
Some of the reasons for choosing to use a Queue Manager Group are:
To connect a client to any one of a set of queue managers that is running, to improve availability.
To reconnect a client to the same queue manager it connected to successfully last time, but connect to a different queue manager if the connection fails.
To automatically reconnect a client connection to another queue manager if the connection fails, without writing any client code.
To automatically reconnect a client connection to a different instance of a multi-instance queue manager if a standby instance takes over, without writing any client code.
To balance your client connections across a number of queue managers, with more clients connecting to some queue managers than others.
To spread the reconnection of many client connections over multiple queue managers and over time, in case the high volume of connections causes a failure.
To be able to move your queue managers without changing any client application code.
To write client application programs that do not need to know queue manager names.
In this example the user has defined two client channels.
The client searches through the client channels in alphabetical channel name order. It looks for a channel definition with a QMNAME field which matches what the application specified on the MQCONN call. We therefore find channel ‘chl2’. If we did not find any channel definitions which match the application would receive a 2058 (Queue Manager name error) reason code.
The transmission protocol and associated connection are extracted from the channel definition and an attempt is made to start the channel to the machine identified (venus). In order for the connection to be successful clearly there must be started listener at the remote machine and the queue manager itself must be started.
If the connection can not be established then a 2059 (Queue Manager not available) reason code is returned to the application. If you believe the Queue Manager is running then look in the client error log for an error message explaining the reason for the failure.
The error log is in <mq install path>\errors\AMQERR01.LOG
In this example there are three channels, that all connect to the same queue manager using different connections (ethernet, tokenring and dialup). This provides a level of redundancy.
The client has to pick one, but which one?
The client attempts to start channel 'chl2' (since the search is in alphabetical channel name order); its QMNAME attribute matches the name in the MQCONN. However the communication link is currently broken.
Channel 'chl3' is now started instead because QMNAME still matches what was specified on the MQCONN call.
So the client is connected to queue manager “venus" but via ethernet.
In this example the client tries to connect to a queue manager first using "chl1" but the communication link is down.
Secondly it tries "chl2" but the queue manager is not currently running.
Finally the client tries to connect using channel "chl5". The communications link is running and the queue manager is running.
However, the name of the queue manager "pluto" does not match the one specified on the MQCONN call “planet”. What we need is a way to tell MQ that we, the application, don’t really care what the actual Queue Manager name is.
We can do that by specifying "*planet“ rather than just “planet”. The * specifies that the client does not care if the actual name of the Queue Manager does not match the name given.
When using a client channel definition table (CCDT) to configure the client connectivity used by your client applications, you can provide a number of destination queue managers to choose from in order to provide redundancy and alternate destinations when one fails.
You can define these destinations with a weighting so that the spread of client connections between the group of queue managers is as you require.
You can then use the same CCDT with all your clients – no need to produce different copies of the CCDT to spread out your client connections across all the back-end servers.
The default value of CLNTWGHT is 0 – which retains the V6 behaviour of primary then secondary choices chosen by alphabetical order.
By default client channels have AFFINITY(PREFERED) set. This means that any particular client application will attempt to connect to the same queue manager each time. This is the same behaviour that was available in V6 with the mechanism that the primary connection was attempted first, then if it was not available, the secondary connection was attempted, and so on. If it is desired that connections from the same machine are to be workload balanced as above, AFFINITY(NONE) can be chosen.
ShareConv Multiplex channel setting
Shareconv default is 10 but can be set server side
0 - Multiplexing and all v7 dependent functionality disabled (i.e async consumer)
1 - Multiplexing disabled but all v7 functionality enabled
>1 - Multiplexing and v7 functionality enabled
BE SURE TO STRESS THAT THE FOLLOWING SLIDES THAT COVER PERFORMANCE REQUIRE AT LEAST SHARECNV(1)!!!!!
For JMS clients, the client side controls negotiating position using JMS Connection Factory Property "XMSC_WMQ_SHARE_CONV_ALLOWED"
WMQ_SHARE_CONV_ALLOWED_YES will accept any positive value (i.e. > 0)
WMQ_SHARE_CONV_ALLOWED_NO will only accept a ShareConv value of 1.
Setting ShareConv to 0 can only be supported using the v6 leg.
IP Sprayers
Connections to v6 Queue Managers may take two attempts
Tolerant to second attempt not being to the same Queue Manager
Read Ahead represents a recognition of the fact that a large proportion of the cost of an MQGET from a client is the line turnaround of the network connection. When using Read Ahead the MQ client code makes a request for more than one message from the server. The server will send as many non-persistent messages matching the criteria (such as MsgId) as it can up to the limit set by the client. The largest speed benefit will be seen where there are a number of similar non-persistent messages to be delivered and where the network is slow.
Read Ahead is useful for applications which want to get large numbers of non-persistent messages, outside of syncpoint where they are not changing the selection criteria on a regular basis. For example, getting responses from a command server or a query such as a list of airline flights.
If an application requests read ahead but the messages are not suitable, for example, they are all persistent then only one message will be sent to the client at any one time. Read ahead is effectively turned off until a sequence of non-persistent messages are on the queue again.
The message buffer is purely an 'in memory' queue of messages. If the application ends or the machine crashes these messages will be lost.
Because this mechanism is designed to remove the network delay it currently only has a benefit on client applications. However, it is recommended that applications that might benefit from it, use it for local bindings as well since in the future there is the possibility that the server could perform some optimisations when this option is used.
A benefit here is that when a client application issues an MQGET(WAIT) there is not a thread on the server which is sitting in an MQGET(WAIT). In fact, there need be no thread directly related to an individual client connection any more.
Asynchronous Put (also known as 'Fire and Forget') is a recognition of the fact that a large proportion of the cost of an MQPUT from a client is the line turnaround of the network connection. When using Asyncronous Put the application sends the message to the server but does not wait for a response. Instead it returns immediately to the application. The application is then free to issue further MQI calls as required. The largest speed benefit will be seen where the application issues a number of MQPUT calls and where the network is slow. Asynchronous put can be requested via the MQPMO_ASYNC_RESPONSE put option or administratively via the DEFPRESP queue attribute.
Once the application has competed it's put sequence it will issue MQCMIT or MQDISC etc which will flush out any MQPUT calls which have not yet completed.
Because the client does not wait for a response from the MQPUT call it will not be told at MQPUT time whether there was a problem putting the message. For example, the queue could be full. There are three things the application can do :
Ignore the situation
In many cases of say a non-persistent message the application does not care too much whether the message makes it or not. If no response is received then another request can be issued within a few seconds or whatever.
Issue an MQCMIT
If the messages put are persistent messages in syncpoint then if any of them fail they will cause a subsequent MQCMIT call to also fail.
Issue a new verb MQSTAT
This new verb allows the application at any time to flush all messages to the server and respond with how many of the messages succeeded or failed.
Because this mechanism is designed to remove the network delay it currently only has a benefit on client applications. However, it is recommended that applications use it for local bindings as well since in the future there is the possibility that the server could perform some optimizations when this option is used.
The Secure Sockets Layer (SSL) provides an industry standard protocol for transmitting data in a secure manner over an insecure network. The SSL protocol is widely deployed in both Internet and Intranet applications. SSL defines methods for authentication, data encryption, and message integrity for a reliable transport protocol, usually TCP/IP.
SSL can be enabled on client channels by specifying a CipherSpec on the client and server connection channel definitions.
SSL cannot be used if using the MQSERVER environment variable.
If using the MQCNO structure to pass in the client channel on an MQCONNX call, a CipherSpec can be set in the MQCD structure.
If using Active Directory on Windows you can use the setmqcsp control command to publish the client-connection channel definitions in Active Directory. One or more of these definitions can specify the name of a CipherSpec.
In contrast the other “sibling” is point to point messaging.
In the same way it is not the message content that makes an application point to point or Pub / sub, it is the infrastructure that handles the message.
Again the infrastructure is what is pub/sub
Over 10,000 magazines in the US,
Publishers use the magazine title and advertising to indicate the contents.
We readers select which magazine to pick up based on that information.
A topic tree is an internal representation of the topic hierarchy – is references using the Topic string.
The tree has a root node at the very top. This is described using an MQ-defined base topic object (topic objects will be discussed on the next slide).
The shape of the topic tree is implied from the complete set of topic strings in use - either defined (as topic objects), published to, subscribed to.
There is not necessarily a one-to-one mapping between topic objects and nodes in the tree. This will become clear as we move to the next slide and discuss exactly what we mean by topic objects.
Use DISPLAY TPSTATUS to display the status of one or more topic nodes in a topic tree.
The value of TPSTATUS determines which topic nodes are returned on the call to DISPLAY TPSTATUS. The TPSTATUS attribute requires a topic string value. This value may be one of the following:
A specific topic string value, e.g. DIS TPS(‘Sports/Football’) – returns just the ‘Sports/Football’ node
A topic string containing a ‘+’ wildcard, e.g. DIS TPS(‘Sports/Football/+’) – returns all immediate children of the ‘Sports/Football’ node
A topic string containing a ‘#’ wildcard, e.g. DIS TPS(‘Sports/Football/#’) – returns all descendants of ‘Sports/Football’ in the topic tree, plus the ‘Sports/Football’ node itself
A topic string containing multiple wildcards, e.g. DIS TPS(‘Sports/+/teams/#’) – returns any immediate child of ‘Sports’ which has a child of ‘teams’, plus the all descendants of that node
Note that the use of wildcards should follow the rules as laid out in INSERT REFERENCE TO BOOK CONTAINING INFO ON HOW WILDCARD CHARS WORK AS IN 99966. An asterisk is not a supported wildcard for the TPSTATUS attribute.
To return a list of all root-level topics, use DIS TPS(‘+’)
To return a list of all topics in the topic tree (depending on your configuration, this may return a large volume of data), use DIS TPS(‘#’)
The list of topics that is returned may be further modified by the use of a filter on TOPICSTR, e.g. DIS TPS(‘Sports/Football/+’) WHERE(TOPICSTR LK ‘Sports/Football/L*’) will return all immediate child nodes of the node ‘Sports/Football/’, which begin with the letter ‘L’.
Review objectives for Lab 3.
The purpose of this lab is to show an example of
Key Points:
There is nothing else quite like MQ in the Market today. It is the market leader in Message Oriented Middleware. It is The Universal Communicator
MQ’s key strength is its breadth:
With a whole range of programming languages – including Java, C/C++, C#, .NET, COBOL,…
A wide range of Interfaces from MQI to JMS, XMS and .Net. Other interfaces that can be used are to HTTP & FTP
This support means that applications can be connected without modification
Also virtually any IT commercial platform – including native z/OS is supported.
This means your customers can integrate anything they have and use the skills they already have…
So lets now look a bit more closely at HTP ….
Within a unit of work, changes to resources are atomic. That is, either all of them take place and are committed, or none of them take place. There are no in-between states.
In the event of failure during a unit of work, or if the application determines that it cannot complete the unit of work for any reason, changes to resources that have already been made are backed out, or rolled back.
The point at which changes to the resources within a unit of work are committed or backed out is known as a point of synchronization, or more simply a syncpoint. At a syncpoint, the data within the resources is in a consistent state from the point of view of the business and its applications.
The visual shows changes to queues and a database, but other resources such as files and working storage might also be affected.
The current depth reflects these messages but any attempt to use MQGET to retrieve them fails with MQRC_NO_MSG_AVAILABLE
Supported by env:
- UNIX and Linux systems
- Windows systems
First
TriggerMsgPriority
Every
TriggerMsgPriority
Depth
TriggerDepth
TriggerMsgPriority
MQTM, MQTMC and MQTMC2 contain the fields displayed above.
MQTM defines the version and ApplType fields as MQLONG, while the MQTMC defines those felds as characters.
MQTMC2 is the same definition as MQTMC, though the version value is 2 and there is one additional field, QMgrName
The COBOL copy files (containing the named constants and structure definitions) located in QMQM/QCBLLESRC
Command on slide is from Infocenter
This is from MQ Explorer Windows:
Rem These commands give group 'mqopr' read only access on WebSphere MQ for Windows.
setmqaut -m QM.TEST -t qmgr -g mqopr +connect +inq +dsp
setmqaut -m QM.TEST -n "**" -t q -g mqopr +dsp +browse
setmqaut -m QM.TEST -n "**" -t topic -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t channel -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t process -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t namelist -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t authinfo -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t clntconn -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t listener -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t service -g mqopr +dsp
setmqaut -m QM.TEST -n "**" -t comminfo -g mqopr +dsp
Rem The following commands provide administrative access for MQ Explorer.
setmqaut -m QM.TEST -n SYSTEM.MQEXPLORER.REPLY.MODEL -t q -g mqopr +dsp +inq +get
setmqaut -m QM.TEST -n SYSTEM.ADMIN.COMMAND.QUEUE -t q -g mqopr +dsp +inq +put
FFST = First Failure Support Technology
Each file size of AMQERR0X.LOG is 256KB, can be change in ini file, e.g. “QMErrorLog: ErrorLogSize=1048576” for 1MB
http://www-01.ibm.com/support/knowledgecenter/SSFKSJ_7.1.0/com.ibm.mq.doc/ia12420_.htm?lang=en