Software Defined WebSphere Messaging Infrastructure with Puppet

1. © 2015 IBM Corporation
#ibminterconnect #AEC-1649
Software Defined WebSphere
Messaging Infrastructure with
Puppet
AEC-1649
Rakesh Sharma
Walmart Stores Inc.
Rahul Gupta
IBM
Global Technology Services
@rahulguptaibm

2. Agenda
1
1 2 3
4 5 6
Introduction to
Walmart
IT Automation and
Software Defined
Environment
Puppet Basics
and Architecture
Automation of WMQ
and IIB with Puppet
Workflow and
Demo
Key Takeaways

3. 2
1
Introduction to Walmart

4. Walmart facts and figures
 Annual revenue $473B
(Year ending Jan 2014)
 2.2 million associates around the world
 1.3 million associates in US alone
 11,270 retail units world wide
 158 distribution centers
 Walmart logistics has a fleet of 6,500
tractors, 55,000 trailers and more than
7,000 drivers
 A regional distribution center can have
up to 12 miles of conveyor belts.

5. Walmart locations
11,270 retail stores world wide, in 27 countries
90% of Americans
live within 15
minutes of a
Walmart
11,270
5044
71 banners in 27 countries

6. 5
2
IT Automation and Software Defined Environment

7. Power, Complexity, Expectations and Opportunities
6
 MORE COMPUTING POWER
 MORE COMPLEXITY
 GREATER EXPECTATIONS
 GREATER OPPORTUNITIES
http://www.slideshare.net/PuppetLabs/top-10-business-benefits-of-it-automation

8. 7
http://en.wikipedia.org/wiki/Uncle_Ben#.22With_great_power_comes_great_re
sponsibility.22
“With great
power comes
great
responsibility”
Ben Parker
With Great Power Comes Great Responsibility

9. 8
With Greater Computing Power, We
have greater responsibilities and
opportunities for IT Automation

10. 9
Manual Configurations
Custom Scripts
Golden Images
Legacy Tools
DON’T WORK
It costs reliability, speed, productivity & insight

11. 10
AUTOMATE ITMake rapid, repeatable changes and enforce the consistency
of system and devices

12. 2014 State of Devops Report
11http://puppetlabs.com/sites/default/files/2014-state-of-devops-report.pdf
“Last year, we were delighted to discover that we could actually
quantitatively define IT performance. We discovered that high
performing IT organizations are more agile and reliable:
They deploy code 30 times more frequently than their
lower-performing peers, with 50 percent fewer failures.
This year, we’ve learned that IT performance has real impact on
the business: Companies with high IT performance are twice
as likely to exceed their profitability, market share
and productivity goals.”

13. Why Automation
12
#1. Avoiding Downtime
http://dealbook.nytimes.com/2012/08/02/knight-capital-says-trading-mishap-cost-
it-440-million/?_r=0
“Knight Capital Says Trading
Glitch Cost It $440 Million”
The company said the problems happened because of
new trading software that had been installed. The event
was the latest to draw attention to the potentially
destabilizing effect of the computerized trading that has
increasingly dominated the nation’s stock markets

14. Why Automation
13
#2. Visibility, Auditability and Predictability

15. Why Automation
14
#3. Consistency
file { ‘/tmp/foo
ensure => present
content => ‘correct’
}
Enforces changes
and remediate
configuration drift

16. Why Automation
15
#4. Quicker Recovery
High Performing IT Organizations have double
the change success rate and restore services
12 times faster than peers.
Fewer failures and faster recovery mean less risk
to business when changes are deployed.
http://puppetlabs.com/sites/default/files/2014-state-of-devops-report.pdf

17. Why Automation
16
#5. Fast Response to Software Vulnerabilities
How quickly can you identify which of your
servers are running vulnerable
versions, get the software update through
testing and change control ?
Confidently report that all systems are
patched?
Can you tell how long any given system
has been vulnerable?

18. What is a Software Defined Environments
17
Software
Defined
Environment
Abstracted and
virtualized IT infrastructure resources managed by software
IT infrastructure
that extends multiple environments to go beyond the data center
Applications
automatically define infrastructure requirements and configuration

19. Software Defined Environment Approach
18
4. Report
package { ssh: ensure => installed }
service {sshd: ensure => running,}
1. Define
Re-usable infrastructure
as a code
2. Simulate
Before deploying
changes
3. Enforce
Current
state
Desired
state
Automatically
and reliably
Insight into
changes
SDE

20. 19
3
Puppet Basics and Architecture

21. What is Puppet
• Puppet is a software for automating system administration
tasks.
• Allows the sysadmin to administrate infrastructure through
code
• The sysadmin work is written as code in Puppet’s custom
language which is shareable just like any other code
20
• Idempotent
• Cross Platform
• Model & Graph
Based

22. Puppet Architecture
21
PUPPET
AGENT
PUPPET
AGENT
PUPPET
AGENT
Web Server
Hardware
Application
Server
Virtual
Machine
PUPPET MASTER SERVER
PUPPET ENTERPRISE
Database
Server
Cloud
Reporting
GUI &
workflow
Content Admin and
Security
Puppet
Forge

23. What do Agent and Masters Do ?
22
https://docs.puppetlabs.com/learning/agent_master_basic.html
Puppet compiles and
applies a manifest
Puppet’s agent
mode is pull-
based. Usually,
agents are
configured to
periodically fetch a
catalog and apply
it, and the master
controls what goes
into that catalog.

24. Lifecycle of a Puppet Run
23
Node
1. Facts 2. Catalog
3. Report
4. Report
1. Facts
The node sends data about its state to the
puppet master server
2. Catalog
Puppet uses the facts to compile a catalog that
specifies how the node should be configured
3. Report
Configuration changes are reported back to the
Puppet mater
4. Report
Puppet open API can also send data to 3rd
party tools

25. Puppet components
24
Puppet Dashboard MCollective Puppet DB
The Marionette
Collective AKA
MCollective is a
framework to build
server orchestration or
parallel job execution
systems.
MCollective use
modern tools
like Publish Subscribe
Middleware and
modern philosophies
like real time discovery
of network resources
using meta data and
not hostnames
Puppet Dashboard is
an open source web
console for Puppet,
which can analyze
reports, browse
inventory data, and
assign classes to
nodes.
Puppet DB is the fast,
scalable, and reliable data
warehouse for Puppet. It
caches data generated by
Puppet, and gives you
advanced features at
awesome speed with a
powerful API.

26. 25
4
Puppet with WebSphere MQ and IBM Integration Bus

27. Everyday Challenges for Middleware Administrator
 Create new middleware environment for new projects or for
disaster recovery
 Diligently follow naming conventions for creating objects
 Follow the implementation guide, without adding any personality
of their own to the existing work
 Ensure the work is documented appropriately so the other
sysadmins can understand the work done by them.
 Ensure the consistency of middleware infrastructure while
scaling for business requirements
26

28. 27
Puppet Use Cases for WMQ & IIB
Kernel settings required for mqm user in Linux
MQ installation, with version, location as a parameter
Queue Manager creation
Either pass queue manager name as class parameter
Or Create a randomly generated queue manager name from
“facts” like hostname.
Start queue manager with all default values
Default listener with port number as a parameter or default
values.
Dead-letter and default XMIT Queue creation.
IIB Broker, execution group, configurable service.

29. 28
5
Puppet workflow
Git Repository
Hosting
Puppet Master
Server
Puppet
Agent
Node
Puppet
Agent
Node
Puppet
Agent
Node
QMGR1/BRKR1
QMGR2/BRKR2
QMGR3/BRKR3
Module
check-in
MQ/IIB
modules
MQ/IIB
modules
MQ/IIB
modules
Middleware
System administrator

30. Use case for auditability and visibility
29

31. 30
5
Demo

32. Puppet and WebSphere MQ Demo
31
• A new agent
node is added
• Install
WebSphere
WMQ on this
node
• Create a Queue
Manager
• Tune Queue
Manager logs
• Create Listener
• Create
Channels
• Create Default
Queues

33. Queue manager created with Puppet
32

34. Queue manager log files created with Puppet
33

35. 34
6
Key Takeaways

36. Technical/Business Benefits
 Improved efficiency. Once verified, deploy to as many nodes as
you require to
 Better than gold images/custom scripting
 Low level documentation
 No configuration drift provides confidence to system
administrator about any system in environment
 Reduces operational cost & provides faster decision making
 Engineers do high level layout of environment(architecture
work), low level details taken care by puppet
35

37. Notices and Disclaimers
Copyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or
transmitted in any form without written permission from IBM.
U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with
IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has been
reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM
shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY,
EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF
THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT
OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the
agreements under which they are provided.
Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without
notice.
Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are
presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual
performance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products,
programs or services available in all countries in which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not
necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither
intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.
It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal
counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s
business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or
represent or warrant that its services or products will ensure that the customer is in compliance with any law.

38. Notices and Disclaimers (con’t)
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products in connection with this
publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to
interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED,
INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any
IBM patents, copyrights, trademarks or other intellectual property right.
• IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS®, Enterprise Document
Management System™, Global Business Services ®, Global Technology Services ®, Information on Demand,
ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™,
PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®,
pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, SoDA, SPSS, StoredIQ, Tivoli®, Trusteer®,
urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of
International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and
service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on
the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

39. Thank You
Your Feedback is
Important!
Access the InterConnect 2015
Conference CONNECT Attendee
Portal to complete your session
surveys from your smartphone,
laptop or conference kiosk.