IBM MQ on cloud and containers

IBM MQ on Cloud and containers
Robert Parker James McGuire
Software Engineer Software Engineer
IBM MQ, Cloud Development MQ on IBM Cloud
parrobe@uk.ibm.com mcguire7@uk.ibm.com
12 June 2018
AWS

Notices and disclaimers
© 2018 International Business Machines Corporation. No part of this
document may be reproduced or transmitted in any form without
written permission from IBM.
U.S. Government Users Restricted Rights — use, duplication or
disclosure restricted by GSA ADP Schedule Contract with IBM.
Information in these presentations (including information relating to
products that have not yet been announced by IBM) has been reviewed
for accuracy as of the date of initial publication and could include
unintentional technical or typographical errors. IBM shall have no
responsibility to update this information. This document is distributed
“as is” without any warranty, either express or implied. In no event,
shall IBM be liable for any damage arising from the use of this
information, including but not limited to, loss of data, business
interruption, loss of profit or loss of opportunity. IBM products and
services are warranted per the terms and conditions of the agreements
under which they are provided.
IBM products are manufactured from new parts or new and used parts.
In some cases, a product may not be new and may have been previously
installed. Regardless, our warranty terms apply.”
Any statements regarding IBM's future direction, intent or product
plans are subject to change or withdrawal without notice.
Performance data contained herein was generally obtained in a
controlled, isolated environments. Customer examples are presented as
illustrations of how those
customers have used IBM products and the results they may have
achieved. Actual performance, cost, savings or other results in other
operating environments may vary.
References in this document to IBM products, programs, or services does
not imply that IBM intends to make such products, programs or services
available in all countries in which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared
by independent session speakers, and do not necessarily reflect the
views of IBM. All materials and discussions are provided for informational
purposes only, and are neither intended to, nor shall constitute legal or
other guidance or advice to any individual participant or their specific
situation.
It is the customer’s responsibility to insure its own compliance with legal
requirements and to obtain advice of competent legal counsel as to
the identification and interpretation of any relevant laws and regulatory
requirements that may affect the customer’s business and any actions
the customer may need to take to comply with such laws. IBM does not
provide legal advice or represent or warrant that its services or products
will ensure that the customer follows any law.
© 2018 IBM Corporation

Notices and disclaimers
continued
Information concerning non-IBM products was obtained from the
suppliers of those products, their published announcements or other
publicly available sources. IBM has not tested those products about this
publication and cannot confirm the accuracy of performance, compatibility
or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of
those products. IBM does not warrant the quality of any third-party
products, or the ability of any such third-party products to
interoperate with IBM’s products. IBM expressly disclaims all
warranties, expressed or implied, including but not limited to, the
implied warranties of merchantability and fitness for a purpose.
The provision of the information contained herein is not intended to, and
does not, grant any right or license under any IBM patents, copyrights,
trademarks or other intellectual property right.
IBM, the IBM logo, ibm.com and [names of other referenced IBM
products and services used in the presentation] are trademarks of
International Business Machines Corporation, registered in many
jurisdictions worldwide. Other product and service names might
be trademarks of IBM or other companies. A current list of IBM
trademarks is available on the Web at "Copyright and trademark
information" at: www.ibm.com/legal/copytrade.shtml.
.

Contents
4© 2018 IBM Corporation
• Moving to the Cloud
• Why MQ
• Adopting best practices for MQ (in the cloud)
• MQ is there for you, wherever you want to be!
• Private cloud
• Public cloud
• Hybrid
• Summary

Top business drivers for cloud
Source: OpenStack user survey, April 2016
Top 3

In other words…
Be more effective!
...increase rate of innovation
...achieve more
...save money

Characteristics of a cloud environment
Self-service
Empowers users to provision resources without
requiring human intervention, most likely using a
web-based portal or an API.
Elastic scaling
Enables scaling up and down on demand, driving
the need for high levels of automation.
Shared resources
Offers economies of scale through the use of shared
infrastructure and software, securely separating the
resources at a logical level.
Metered usage
Allows pay-as-you-go billing through monitoring,
measurement and reporting of usage.
Private cloud
IBM Cloud Private
Public Cloud

Exactly once delivery
MQ is designed from the ground up to ensure that
when messages of worth are being processed and
exactly once, transactional, messages are required,
that really means exactly once
MQ’s key value capabilities and
why they matter…
The world depends on reliable, secure
messaging
IBM MQ is the most reliable and trusted
messaging solution
Your bank transfers complete without losing your
money
Your credit card information is kept safe
You move your customer data between systems
without exposing it to hackers
Your developers focus on the business needs,
not transaction recovery or security
Scalable
Scales to meet the
needs of your
business, handling
billions of messages
per day
Simple
Your applications are
focused on the
business, not fixing
connectivity problems
or tracking their
progress

Secure
MQ is secure by design, from fine grain
authentication and authorization of messaging
resources, to encryption of data from one
application to another, without the application even
knowing or the administrators gaining access
The world depends on reliable, secure
messaging
IBM MQ is the most reliable and trusted
messaging solution
Your bank transfers complete without losing your
money
Your credit card information is kept safe
You move your customer data between systems
without exposing it to hackers
Your developers focus on the business needs,
not transaction recovery or security
Reliable
Only MQ has 25 years
of proven reliability and
robustness when used
by the most
demanding customers
in the most demanding
systems, without you
even knowing it’s there
Connectivity
Simple queues and
topics provide simple
application messaging
but MQ provides true
routing capabilities to
join applications and
systems together,
locally and globally
MQ’s key value capabilities and
why they matter…

Run IBM MQ in any
location or cloud
exactly as you
need it
On-premise, software
and the MQ Appliance
Run it yourself in any
cloud, public or private
Let IBM host it for you
with its new managed
MQ service in IBM
Cloud
Private cloud
AWSAWS
AWSAzure
AWS
…
Linux
Windows
Solaris
HPE
AIX
IBMi
IBM Z
Appliance
NEW
IBM Cloud Private

Adopting best practices for MQ
(including in the cloud)

MQ’s capabilities have always been
a perfect fit for cloud scenarios;
– Integrating remote or heterogenous
systems
– Dynamic client connectivity
– Dynamic scaling
– Workload balancing
– High security
– High scale and robustness
– Repeatable, remote administration and
monitoring
…
Do you currently take best
advantage of those strengths?

How many of these do you have?
 Hand crafted, shared queue managers
 Applications hard coding connection details
 Applications bound to a single IP address
 Edge security at most
 Internal architecture complexity exposed to
the applications
 A lengthy change control process
 Manual installation, deployment and
configuration
Rethink your use of MQ
Don’t confuse old MQ practices for MQ itself
“MQ is too hard to use”
“Our MQ system is too complicated to change”
“MQ isn’t cloud, it’s too old!”
Dare to re-invent
your approach!

Managing MQ
Management of systems is evolving towards
cattle not pets. What does that mean for
MQ?
Consistent configuration and operations across
multiple queue managers
Automated deployment
Configuration as code
Self service
Collection and analysis of diagnostic data
Simple integration into standard devops and
automation tooling

Marks & Spencer – UK retailer
© 2018, Marks and Spencer, reproduced with permission

Error logs output JSON for easy parsing
Managing diagnostic data
The need to centrally collect and analyse diagnostic
data is increasing, using tools such as Splunk,
Elasticsearch and Grafana
MQ generates a wide range of information and has
demonstrated how this can be collected using off the
shelf tooling
Subscribing to metrics in MQ V9 makes that even
easier
MQ 9.0.x has seen enhancements to the error log
data it generates to aid such solutions
Universal timestamps and severity levels
Separated inserts
JSON output
Multiple logs
Syslog output
Forward MQ error logs
to ElasticSearch or
Splunk
Publish MQ statistics
to Prometheus and
Grafana
{
"ibm_messageId":"AMQ6287I",
"ibm_arithInsert1":0,
"ibm_arithInsert2":0,
"ibm_commentInsert1":"Linux 4.13.0-36-generic (MQ Linux (x86-64 platform) 64-bit)",
"ibm_commentInsert2":"/opt/mqm (Installation1)",
"ibm_commentInsert3":"9.0.5.0 (p905-L180228.1)",
"ibm_datetime":"2018-03-04T13:18:27.506Z",
"ibm_serverName":"QM905",
"type":"mq_log",
"host":"david-VirtualBox",
"loglevel":"INFO",
"module":"amqxeida.c:6238",
"ibm_sequence":"1520169507_506462655",
"ibm_processId":2119,
"ibm_threadId":1,
"ibm_version":"9.0.5.0",
"ibm_processName":"strmqm",
"ibm_userName":"david",
”ibm_installationName":"Installation1",
"ibm_installationDir":"/opt/mqm",
"message":"AMQ6287I: IBM MQ V9.0.5.0 (p905-L180228.1).”
}
NEW

General principles
Create or obtain
an image
• Virtual machine
• EC2 AMI
• Docker container
Configure the
instance
Use tools to
deploy as needed
• Typically, execute
a runmqsc script,
e.g;
• Embedded in the
image
• Pulled from
source control
• Command-line or
deployment tools for
your platform
• Terraform, Chef,
Puppet, Ansible, …
• AWS CloudFormation,
OpenStack Heat,
Kubernetes, Helm, …

Virtual machine failover
• AWS Auto-scaling group
• VMWare HA cluster
• Pacemaker heart-beating
Container failover
• Orchestration technology such as Kubernetes,
Docker Swarm, managing a cluster of worker
instances
• Healthcheck endpoints in the container
• Container is moved to a different worker in the
cluster
Instance availability
High availability for IBM MQ is a combination of instance availability and
storage availability
Instance failover can be handled in a number of different ways,
depending on the type of platform in which you are running

Network attached storage
• Stored outside the instance so can be
connected from multiple locations
• File storage: NFSv4, Amazon EFS, CephFS,
GlusterFS, IBM Cloud File storage
• Block storage: Amazon EBS, OpenStack Cinder,
Ceph RBD, IBM Cloud Block storage
MQ replicated data queue manager
• Data is replicated between local disk on multiple
active instances
• Synchronous and asynchronous replication
options
• Difficult to achieve in container environments
due to operating system kernel dependencies
Storage availability
The MQ queue manager generally relies on reliable persistent storage.
The persistent data must be available wherever the instance might run
NEW

Resilience
Cloud deployments emphasize resilience against failures
Cloud servers are likely to get restarted or moved
• Use the MQ client’s auto-reconnect feature, e.g;
factory.setClientReconnectOptions(WMQConstants.WMQ_CLIENT_RECONNECT)
• The IP address of the queue manager may change if it fails
over to a new host
• Scalability may mean that multiple equivalent queue managers
are available
• ConnectionNameList, or CCDT
Scalability
Clients should be deployed and scaled separately from servers
Increasing horizontal scaling is easy, for example;
• Load balanced set of identical queue managers
• MQ cluster
Decreasing horizontal scaling depends on your application;
• Need to remove a queue manager in a controlled manner
making sure that all of the messages are safely processed
Message ordering
• Scaling out rather than scaling up, brings concurrent
processing
• Messages could be received out of sequence
• MQ provides features to allow groups of messages to be
handled in small ordered batches, or you can manage this
yourself in your application
© 2018 IBM Corporation https://developer.ibm.com/messaging/2018/05/02/availability-scalability-ibm-mq-containers

1. Automatically encrypted by the sending client
so that it can only be decrypted by the intended
recipient
2. Or encrypted by the queue manager on receipt,
for cases where the application deployment
cannot be updated
Benefits
• No application code changes required
• Goes beyond TLS channel security, which only
protects data in transit between processes
• Message data can only be read by the intended
receiving application code
• Not on the queue by the system administrator
• Not on the disk by your infrastructure or cloud
provider
• Proven, trusted approach to fulfilling
compliance requirements such as GDPR,
PCI, HIPAA etc
End-to-end message level security
MQ Advanced Message Security (AMS) provides the capability to encrypt messages in transit and at rest
between sender and receiver.
Application B
Application A
Channels
IBM MQ
Advanced
Queue
Manager
Queue
Manager

Find out more!
It has always been important for MQ to run where it is
needed and integrated into the tools of your choice
• We have been demonstrating running MQ in
various public and private clouds
• Using a variety of tooling for provisioning,
configuration and monitoring
• And we’ve been sharing that information for
everyone to use
developer.ibm.com/messaging/mq-on-cloud

Why MQ for hybrid cloud
connectivity?

Servic
e
Georgia, USA
Web App
Use IBM MQ to connect enterprise applications within a cloud location

© 2018 IBM Corporation… to reliably connect the Cloud to your on-premises data center
Stock System
On-prem
Cloud
Application

IBM Cloud
AWS
App B
App A
… to connect different cloud providers

MQ in on-premises private clouds
Private cloud
IBM Cloud Private

Software install of
IBM MQ just
requires a
supported operating
system and file
system
Run in your preferred IaaS private cloud or virtualization platform
VMWare
OpenStack
…
Xen
CloudStackCloudFoundry
Bluebox

MQ in Containers MQ has been supporting
Docker containers since
2015 with images on
Docker Hub and
Docker Store and
sample setups on
Github
More recently it has
been demonstrating
how to get the most
from containers using
Kubernetes
Including RedHat
OpenShift + Pivotal
Container Service
And now MQ Advanced
is available as a fully
supported product with
IBM Cloud Private, a
Kubernetes-based
solution from IBM

IBM Cloud Private Solution Overview
Think 2018 / 8312 / March 21, 2018 / © 2018 IBM Corporation 3434
Runs on existing IaaS: System Z
IBM
Spectrum
Kubernetes-based
Container Platform
Industry leading container
orchestration platform
Cloud Foundry
For prescribed
application
development &
deployment
Terraform (CAM)
Infrastructure as Code for
multi-cloud provisioning to
public and on-prem private
clouds
IBM Middleware & Open Source – e.g. Data, Analytics and Developer Services
Cloud-enabled middleware, application runtimes, messaging, databases & analytics to optimize current
investments and rapidly innovate
Core Operational Services
To simplify Operations Management, Security, DevOps, and hybrid integration
Dell, Cisco, NetApp, Lenovo, …
IBM Cloud / © 2018 IBM Corporation

Current licensing for MQ
Use existing MQ license.
Perpetual PVUs
Monthly PVUs or VPCs
Think 2018 / © 2018 IBM Corporation
Statement of Direction
IBM intends to offer clients the
ability to purchase an entitlement
that is based on the container size
in Virtual Processor Cores and the
number of hours that MQ was
deployed for in each container

MQ in public clouds
AWSAWS
AWSAzure
AWS

Run in your
preferred public
cloud IaaS
Same requirements
for supported
operating system
and file system
IBM Cloud (IaaS)
Amazon EC2
…
Google Compute Engine
Microsoft Azure
https://developer.ibm.com/messaging/mq-cloud-blog/

Getting up and running
with MQ quickly is
important, but running it
in the best possible way
is critical
The MQ AWS
QuickStart demonstrates
AWS architectural best
practice
Deployed from scratch
in around 30 minutes
IBM MQ on AWS
Quick Start
https://aws.amazon.com/quickstart/architecture/ibm-mq/

Containers in the
cloud
Same requirements
for supported
operating system
and file system
IBM Kubernetes Service
Amazon Elastic
Container Service
…
Google Kubernetes Engine
Microsoft Azure
Container Service
https://developer.ibm.com/messaging/mq-cloud-blog/

Trivial to get started
Default developer
configuration
Continue to use your
favourite administration
tooling
MQ on the IBM
Kubernetes service
https://developer.ibm.com/messaging/2017/09/04/kubernetes-service-mq-docker-bluemix/
# Create a free Kubernetes cluster
bx cs cluster-create --name mq-test
# Launch MQ using public image from the IBM Cloud docker registry
kubectl run my-mq --image=registry.eu-gb.bluemix.net/ibm-mq
--env="LICENSE=accept" --env="MQ_QMGR_NAME=QM1"

Run IBM MQ in any
location or cloud
exactly as you
need it
Let IBM host it for you with its new managed
MQ service in IBM Cloud
NEW

Hardware, virtualization, servers,
network, storage
Queues, topics, channels,
clustering, applications
MQ installation, basic configuration,
security, maintenance
Managed &
operated
by IBM
Configured &
monitored
by the
customer
Provision queue managers
directly into IBM Cloud
IBM owns the infrastructure
and the responsibility to keep
the systems running
The customer owns the
application-level configuration
and the monitoring
Hosted on

MQ on Cloud
service
IBM / VendorCustomer
Data centre
Networking
Servers
Storage
Hypervisor
Virtual machine
OS patching
MQ patching
QM failover
Queues, Topics
Clustering
QM availability/restart
Q / Msg monitoring
Application
System monitoring
Data centre
Networking
Servers
Storage
Hypervisor
Virtual machine
OS patching
MQ patching
QM failover
Queues, Topics
Clustering
Q / Msg monitoring
Application
System monitoring
Data centre
Networking
Servers
Storage
Hypervisor
Virtual machine
OS patching
MQ patching
QM failover
Queues, Topics
Clustering
Q / Msg monitoring
Application
System monitoring
Data centre
Networking
Servers
Storage
Hypervisor
Virtual machine
OS patching
MQ patching
QM failover
Queues, Topics
Clustering
Q / Msg monitoring
Application
System monitoring
Data centre
Networking
Servers
Storage
Hypervisor
Virtual machine
OS patching
MQ patching
QM failover
Queues, Topics
Clustering
Q / Msg monitoring
Application
System monitoring
BYOL Cloud
IaaS install
MQ ApplianceMQ in IBM
Cloud Private
On-premises
software MQ
Division of responsibility based on form factor

Adopting managed cloud services
Benefits of cloud services Common implications of cloud services
The benefits we get from using cloud services sometimes require different expectations for
consuming those services compared to running in a bespoke on-premises deployment
Self service
provisioning in
minutes
Installation /
maintenance
overhead handled
by the provider
Automatically
keep up to date
with security
patches
“Infinite” compute
capacity at the touch
of a button
Lower price due to
economies of scale
Multi-tenancy /
shared
infrastructure
Standardized
configurations
Limited points of
variability
Design for
failure
Updates applied
regularly
Shared responsibility
for the overall
solution
“One” size fits all

© 2018 IBM Corporation Quickly provision messaging capability in the Cloud
Free Trial so you
can get started at
no cost
Paid options in
sizes to suit
typical workloads

The MQ on IBM Cloud service
IBM MQ that you know and trust
… but in a managed Cloud service!
Try the service for free at:
console.bluemix.net/catalog/services/mq

In summary

Run IBM MQ in any
location or cloud
exactly as you
need it
Let IBM host it for you
with its new managed
MQ service in IBM
Cloud
AWSAWS
AWSAzure
AWS
…
Linux
Windows
Solaris
HPE
AIX
IBMi
IBM Z
Appliance
© 2018 IBM Corporation Private cloud
IBM Cloud Private

IBM MQ on cloud and containers

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à IBM MQ on cloud and containers

Similaire à IBM MQ on cloud and containers (20)

Plus de Robert Parker

Plus de Robert Parker (20)

Dernier

Dernier (20)

IBM MQ on cloud and containers