SlideShare une entreprise Scribd logo
1  sur  16
Splunk for JMX
Damien Dallimore
Developer Evangelist
Copyright©2013,SplunkInc.
Splunk for JMX
2
• ConnecttoanylocalorremoteJVM'sJMXserver,Hotspot/JRockit/IBMJ9/OpenJDK/AzulZing
• QueryanyMBeanrunningonthatserver
• ExtractanyMBeanattributes(simple,compositeortabular)
• InvokeMBeanoperations
• Writeattributesandoperationresultsoutinadefaultkey/valueformat,orpluginyourowncustomformat,forSPLUNK
indexingandsearching
• TransporteventsoverSTDOUT(default),TCP,Syslog,SplunkRESTendpointordirecttofile.
• DeclareclustersofJVM'sforlargerscaleJVMdeployments
• Runson*NixandWindows
• OutoftheboxdashboardsforcommonJVMMBeans
• FreelyavailablefromSplunkBase,allsourcecodeisonGitHub
Copyright©2013,SplunkInc.
Connectivity Options
3
• Remote JMX interface
– rmi (JSR160StandardImplementationandMX4J'sJSR160Implementation)
– iiop(JSR160StandardImplementationandMX4J'sJSR160Implementation)
• Direct Process attachment
– ConnectdirectlytoalocallyrunningJVMprocess
• MX4J HTTP connectors (requires MX4J in the target JVM also)
– soap,soap+sssl
– hessian,hessian+ssl
– burlap,burlap+ssl
Copyright©2013,SplunkInc.
Setup and Configuration
4
• The main goalof the app was to make it as simple and intuitive as possible to connect to
your JVMs and start Splunking JMX data
• Enableyour targetJVM’s remoteJMX interface, test connectivitywith JConsole
• InstallSplunk for JMX
• ExtractSplunkforJMXtarballtoSPLUNK_HOME/etc/apps
• RestartSplunk
• Atthesetupscreen,choosea scriptedinputforyourplatform(Nix/Windows)
• Setup your JMX configurationfile
• Thedefaultconfig.xmlfileispreconfiguredforcommonJVMMBeans
• BrowseyourJVM(usingJConsole)forotherMBeansthatyouwishtopollandconfigurethese
• Youcanhaveasmanyconfigfilesasyourequire,andyoumightsettheseuptofireoffatdifferent
scheduledfrequencies
Copyright©2013,SplunkInc.
Configuration Examples - Simple
5
Copyright©2013,SplunkInc.
Configuration Examples - Wildcards
6
MBean Object name format “domain:key=value,key2=value2”
Around 25KBytes per dump on Hotspot JVMs
Don’t know the domain or properties name or have many Mbeans
with the same attributes ?
* and ? wildcards are supported in the Mbean name
Copyright©2013,SplunkInc.
Configuration Examples - Clusters
7
• Define clusters of JVM’s that share the
same MBean definitions
• Note , in these examples, for brevity I am
using “dumpAllAttributes” , but in
production you’d want to pick and
choose specific MBean attributes you
are interested in, and perhaps split
definitions over multiple files run at
varying frequencies
Copyright©2013,SplunkInc.
Configuration Examples - Operations
8
• Invoke JMX operations that return a value or simply
perform some action on the target JVM
• Operation definitions can take parameters
Use Case 1 : your developers might code a JMX
operation that returns a CSV or JSON formatted
snapshot of some metrics for Splunking
Use Case 2 : dynamically trigger HPROF dumps.
The “com.sun.management:type=HotSpotDiagnostic”
Mbean exposes a “dumpHeap” operation
Copyright©2013,SplunkInc.
Configuration Examples - Connecting
9
• IP Address with credentials
• Hostname
• Static Process ID
• Process ID lookup from file
• Process ID lookup from command output
• Raw JMX Service URL
• MX4J HTTP Connector
Copyright©2013,SplunkInc.
Custom Formatters/Transports
10
• The Splunk for JMX configuration is user extensible
• You can code and configure your own Formatters and Transports
Formatters
• Takes the raw MBean polled output and formats it for Splunking
• A Java implementation of the "com.dtdsoftware.splunk.formatter.Formatter" interface
• If the optional formatter declaration is omitted, then the default formatter will be used
Transports
• Takes the formatted output and transports it to a destination
• A Java implementation of the "com.dtdsoftware.splunk.transport.Transport" interface
• If the optional transport declaration is omitted, then the default transport(STD out) will be used
Copyright©2013,SplunkInc.
Formatter Examples
11
Copyright©2013,SplunkInc.
Transport Examples
12
Copyright©2013,SplunkInc.
Deployment Architectures 1
13
• Simplest scenario
• Monolithic Splunk installation
• Splunk for JMX polling 1 or more remote/local JVMs via
the remote JMX interface
• There is support for many target JVM’s in the
configuration schema but to really scale out, you need
a more advanced Splunk architecture
Copyright©2013,SplunkInc.
Deployment Architectures 2
14
Splunk UF running locally
with target JVM
Splunk Indexer
Cluster
Splunk Search Head
Pool
Load Balancer
• Run Splunk UF locally with target
JVM.Can connect use remote JMX
interface or direct process attachment.
• Each tier scales out horizontally.
• Can overcome firewall issues that are
sometimes inherent with Java RMI
• Deploy Splunk for JMX components and
configurations with Splunk Deployment
Server, Puppet or Chef.
Demo
Copyright©2013,SplunkInc.
Contact me
16
Email : ddallimore@splunk.com
Twitter : @damiendallimore
Skype : damien.dallimore
Github : damiendallimore
Splunkbase : damiend
Slideshare : http://www.slideshare.net/damiendallimore
Blogs : http://blogs.splunk.com/dev
Web : http://dev.splunk.com

Contenu connexe

Tendances

Introduction to Web Application Clustering
Introduction to Web Application ClusteringIntroduction to Web Application Clustering
Introduction to Web Application Clustering
Piyush Katariya
 
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesik
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A GrzesikApache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesik
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesik
mfrancis
 

Tendances (20)

OSMC 2021 | Robotmk: You don’t run IT – you deliver services!
OSMC 2021 | Robotmk: You don’t run IT – you deliver services!OSMC 2021 | Robotmk: You don’t run IT – you deliver services!
OSMC 2021 | Robotmk: You don’t run IT – you deliver services!
 
Managing ejabberd Platforms with Docker - ejabberd Workshop #1
Managing ejabberd Platforms with Docker - ejabberd Workshop #1Managing ejabberd Platforms with Docker - ejabberd Workshop #1
Managing ejabberd Platforms with Docker - ejabberd Workshop #1
 
Introduction to Web Application Clustering
Introduction to Web Application ClusteringIntroduction to Web Application Clustering
Introduction to Web Application Clustering
 
Simple tweaks to get the most out of your jvm
Simple tweaks to get the most out of your jvmSimple tweaks to get the most out of your jvm
Simple tweaks to get the most out of your jvm
 
Liberty management
Liberty managementLiberty management
Liberty management
 
AMIS SIG - Introducing Apache Kafka - Scalable, reliable Event Bus & Message ...
AMIS SIG - Introducing Apache Kafka - Scalable, reliable Event Bus & Message ...AMIS SIG - Introducing Apache Kafka - Scalable, reliable Event Bus & Message ...
AMIS SIG - Introducing Apache Kafka - Scalable, reliable Event Bus & Message ...
 
No Surprises Geo Replication - Pulsar Virtual Summit Europe 2021
No Surprises Geo Replication - Pulsar Virtual Summit Europe 2021No Surprises Geo Replication - Pulsar Virtual Summit Europe 2021
No Surprises Geo Replication - Pulsar Virtual Summit Europe 2021
 
Performance Testing using Real Browsers with JMeter & Webdriver
Performance Testing using Real Browsers with JMeter & WebdriverPerformance Testing using Real Browsers with JMeter & Webdriver
Performance Testing using Real Browsers with JMeter & Webdriver
 
Batching and Java EE (jdk.io)
Batching and Java EE (jdk.io)Batching and Java EE (jdk.io)
Batching and Java EE (jdk.io)
 
Yang in OpenDaylight
Yang in OpenDaylightYang in OpenDaylight
Yang in OpenDaylight
 
Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk Dragonflow Austin Summit Talk
Dragonflow Austin Summit Talk
 
High density deployments using weblogic multitenancy
High density deployments using weblogic multitenancyHigh density deployments using weblogic multitenancy
High density deployments using weblogic multitenancy
 
High performance network programming on the jvm oscon 2012
High performance network programming on the jvm   oscon 2012 High performance network programming on the jvm   oscon 2012
High performance network programming on the jvm oscon 2012
 
KubeFuse - A File-System for Kubernetes
KubeFuse - A File-System for KubernetesKubeFuse - A File-System for Kubernetes
KubeFuse - A File-System for Kubernetes
 
Docker Swarm and Traefik 2.0
Docker Swarm and Traefik 2.0Docker Swarm and Traefik 2.0
Docker Swarm and Traefik 2.0
 
Common Sense Performance Indicators in the Cloud
Common Sense Performance Indicators in the CloudCommon Sense Performance Indicators in the Cloud
Common Sense Performance Indicators in the Cloud
 
Jakarta EE 8 on JDK17
Jakarta EE 8 on JDK17Jakarta EE 8 on JDK17
Jakarta EE 8 on JDK17
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache Kafka
 
The best of Apache Kafka Architecture
The best of Apache Kafka ArchitectureThe best of Apache Kafka Architecture
The best of Apache Kafka Architecture
 
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesik
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A GrzesikApache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesik
Apache Karaf - Building OSGi applications on Apache Karaf - T Frank & A Grzesik
 

Similaire à Splunk for JMX

Tornado Web Server Internals
Tornado Web Server InternalsTornado Web Server Internals
Tornado Web Server Internals
Praveen Gollakota
 
AMF Flash and .NET
AMF Flash and .NETAMF Flash and .NET
AMF Flash and .NET
Yaniv Uriel
 
WebLogic JMX for DevOps
WebLogic JMX for DevOpsWebLogic JMX for DevOps
WebLogic JMX for DevOps
Frank Munz
 
We4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshell
We4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshellWe4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshell
We4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshell
We4IT Group
 
Mom those things v1
Mom those things v1 Mom those things v1
Mom those things v1
von gosling
 

Similaire à Splunk for JMX (20)

Linked Process
Linked ProcessLinked Process
Linked Process
 
Five cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark fasterFive cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark faster
 
A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.
A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.
A Java Implementer's Guide to Boosting Apache Spark Performance by Tim Ellison.
 
Tornado Web Server Internals
Tornado Web Server InternalsTornado Web Server Internals
Tornado Web Server Internals
 
FOSDEM 2017 - Open J9 The Next Free Java VM
FOSDEM 2017 - Open J9 The Next Free Java VMFOSDEM 2017 - Open J9 The Next Free Java VM
FOSDEM 2017 - Open J9 The Next Free Java VM
 
Getting Started with JDK Mission Control
Getting Started with JDK Mission ControlGetting Started with JDK Mission Control
Getting Started with JDK Mission Control
 
You Can't Correlate what you don't have - ArcSight Protect 2011
You Can't Correlate what you don't have - ArcSight Protect 2011You Can't Correlate what you don't have - ArcSight Protect 2011
You Can't Correlate what you don't have - ArcSight Protect 2011
 
jcmd #javacasual
jcmd #javacasualjcmd #javacasual
jcmd #javacasual
 
Like loggly using open source
Like loggly using open sourceLike loggly using open source
Like loggly using open source
 
Effective out-of-container Integration Testing
Effective out-of-container Integration TestingEffective out-of-container Integration Testing
Effective out-of-container Integration Testing
 
AMF Flash and .NET
AMF Flash and .NETAMF Flash and .NET
AMF Flash and .NET
 
WebLogic JMX for DevOps
WebLogic JMX for DevOpsWebLogic JMX for DevOps
WebLogic JMX for DevOps
 
Ingesting hdfs intosolrusingsparktrimmed
Ingesting hdfs intosolrusingsparktrimmedIngesting hdfs intosolrusingsparktrimmed
Ingesting hdfs intosolrusingsparktrimmed
 
We4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshell
We4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshellWe4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshell
We4IT LCTY 2013 - x-pages-men - ibm domino xpages - performance in a nutshell
 
Mom those things v1
Mom those things v1 Mom those things v1
Mom those things v1
 
CCNP Data Center Centralized Management Automation
CCNP Data Center Centralized Management AutomationCCNP Data Center Centralized Management Automation
CCNP Data Center Centralized Management Automation
 
Oracle WebLogic Diagnostics & Perfomance tuning
Oracle WebLogic Diagnostics & Perfomance tuningOracle WebLogic Diagnostics & Perfomance tuning
Oracle WebLogic Diagnostics & Perfomance tuning
 
Being HAPI! Reverse Proxying on Purpose
Being HAPI! Reverse Proxying on PurposeBeing HAPI! Reverse Proxying on Purpose
Being HAPI! Reverse Proxying on Purpose
 
OpenSouthCode 2018 - Integrating your applications easily with Apache Camel
OpenSouthCode 2018 - Integrating your applications easily with Apache CamelOpenSouthCode 2018 - Integrating your applications easily with Apache Camel
OpenSouthCode 2018 - Integrating your applications easily with Apache Camel
 
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
BSides LV 2016 - Beyond the tip of the iceberg - fuzzing binary protocols for...
 

Plus de Damien Dallimore

QCon London 2015 - Wrangling Data at the IOT Rodeo
QCon London 2015 - Wrangling Data at the IOT RodeoQCon London 2015 - Wrangling Data at the IOT Rodeo
QCon London 2015 - Wrangling Data at the IOT Rodeo
Damien Dallimore
 
Splunk as a_big_data_platform_for_developers_spring_one2gx
Splunk as a_big_data_platform_for_developers_spring_one2gxSplunk as a_big_data_platform_for_developers_spring_one2gx
Splunk as a_big_data_platform_for_developers_spring_one2gx
Damien Dallimore
 

Plus de Damien Dallimore (9)

QCon London 2015 - Wrangling Data at the IOT Rodeo
QCon London 2015 - Wrangling Data at the IOT RodeoQCon London 2015 - Wrangling Data at the IOT Rodeo
QCon London 2015 - Wrangling Data at the IOT Rodeo
 
SpringOne2GX 2014 Splunk Presentation
SpringOne2GX 2014 Splunk PresentationSpringOne2GX 2014 Splunk Presentation
SpringOne2GX 2014 Splunk Presentation
 
A Brief History Of Data
A Brief History Of DataA Brief History Of Data
A Brief History Of Data
 
Integrating Splunk into your Spring Applications
Integrating Splunk into your Spring ApplicationsIntegrating Splunk into your Spring Applications
Integrating Splunk into your Spring Applications
 
Spring Integration Splunk
Spring Integration SplunkSpring Integration Splunk
Spring Integration Splunk
 
Splunk Developer Platform
Splunk Developer PlatformSplunk Developer Platform
Splunk Developer Platform
 
Splunk as a_big_data_platform_for_developers_spring_one2gx
Splunk as a_big_data_platform_for_developers_spring_one2gxSplunk as a_big_data_platform_for_developers_spring_one2gx
Splunk as a_big_data_platform_for_developers_spring_one2gx
 
Using the Splunk Java SDK
Using the Splunk Java SDKUsing the Splunk Java SDK
Using the Splunk Java SDK
 
Splunking the JVM (Java Virtual Machine)
Splunking the JVM (Java Virtual Machine)Splunking the JVM (Java Virtual Machine)
Splunking the JVM (Java Virtual Machine)
 

Dernier

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Dernier (20)

Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 

Splunk for JMX

  • 1. Splunk for JMX Damien Dallimore Developer Evangelist
  • 2. Copyright©2013,SplunkInc. Splunk for JMX 2 • ConnecttoanylocalorremoteJVM'sJMXserver,Hotspot/JRockit/IBMJ9/OpenJDK/AzulZing • QueryanyMBeanrunningonthatserver • ExtractanyMBeanattributes(simple,compositeortabular) • InvokeMBeanoperations • Writeattributesandoperationresultsoutinadefaultkey/valueformat,orpluginyourowncustomformat,forSPLUNK indexingandsearching • TransporteventsoverSTDOUT(default),TCP,Syslog,SplunkRESTendpointordirecttofile. • DeclareclustersofJVM'sforlargerscaleJVMdeployments • Runson*NixandWindows • OutoftheboxdashboardsforcommonJVMMBeans • FreelyavailablefromSplunkBase,allsourcecodeisonGitHub
  • 3. Copyright©2013,SplunkInc. Connectivity Options 3 • Remote JMX interface – rmi (JSR160StandardImplementationandMX4J'sJSR160Implementation) – iiop(JSR160StandardImplementationandMX4J'sJSR160Implementation) • Direct Process attachment – ConnectdirectlytoalocallyrunningJVMprocess • MX4J HTTP connectors (requires MX4J in the target JVM also) – soap,soap+sssl – hessian,hessian+ssl – burlap,burlap+ssl
  • 4. Copyright©2013,SplunkInc. Setup and Configuration 4 • The main goalof the app was to make it as simple and intuitive as possible to connect to your JVMs and start Splunking JMX data • Enableyour targetJVM’s remoteJMX interface, test connectivitywith JConsole • InstallSplunk for JMX • ExtractSplunkforJMXtarballtoSPLUNK_HOME/etc/apps • RestartSplunk • Atthesetupscreen,choosea scriptedinputforyourplatform(Nix/Windows) • Setup your JMX configurationfile • Thedefaultconfig.xmlfileispreconfiguredforcommonJVMMBeans • BrowseyourJVM(usingJConsole)forotherMBeansthatyouwishtopollandconfigurethese • Youcanhaveasmanyconfigfilesasyourequire,andyoumightsettheseuptofireoffatdifferent scheduledfrequencies
  • 6. Copyright©2013,SplunkInc. Configuration Examples - Wildcards 6 MBean Object name format “domain:key=value,key2=value2” Around 25KBytes per dump on Hotspot JVMs Don’t know the domain or properties name or have many Mbeans with the same attributes ? * and ? wildcards are supported in the Mbean name
  • 7. Copyright©2013,SplunkInc. Configuration Examples - Clusters 7 • Define clusters of JVM’s that share the same MBean definitions • Note , in these examples, for brevity I am using “dumpAllAttributes” , but in production you’d want to pick and choose specific MBean attributes you are interested in, and perhaps split definitions over multiple files run at varying frequencies
  • 8. Copyright©2013,SplunkInc. Configuration Examples - Operations 8 • Invoke JMX operations that return a value or simply perform some action on the target JVM • Operation definitions can take parameters Use Case 1 : your developers might code a JMX operation that returns a CSV or JSON formatted snapshot of some metrics for Splunking Use Case 2 : dynamically trigger HPROF dumps. The “com.sun.management:type=HotSpotDiagnostic” Mbean exposes a “dumpHeap” operation
  • 9. Copyright©2013,SplunkInc. Configuration Examples - Connecting 9 • IP Address with credentials • Hostname • Static Process ID • Process ID lookup from file • Process ID lookup from command output • Raw JMX Service URL • MX4J HTTP Connector
  • 10. Copyright©2013,SplunkInc. Custom Formatters/Transports 10 • The Splunk for JMX configuration is user extensible • You can code and configure your own Formatters and Transports Formatters • Takes the raw MBean polled output and formats it for Splunking • A Java implementation of the "com.dtdsoftware.splunk.formatter.Formatter" interface • If the optional formatter declaration is omitted, then the default formatter will be used Transports • Takes the formatted output and transports it to a destination • A Java implementation of the "com.dtdsoftware.splunk.transport.Transport" interface • If the optional transport declaration is omitted, then the default transport(STD out) will be used
  • 13. Copyright©2013,SplunkInc. Deployment Architectures 1 13 • Simplest scenario • Monolithic Splunk installation • Splunk for JMX polling 1 or more remote/local JVMs via the remote JMX interface • There is support for many target JVM’s in the configuration schema but to really scale out, you need a more advanced Splunk architecture
  • 14. Copyright©2013,SplunkInc. Deployment Architectures 2 14 Splunk UF running locally with target JVM Splunk Indexer Cluster Splunk Search Head Pool Load Balancer • Run Splunk UF locally with target JVM.Can connect use remote JMX interface or direct process attachment. • Each tier scales out horizontally. • Can overcome firewall issues that are sometimes inherent with Java RMI • Deploy Splunk for JMX components and configurations with Splunk Deployment Server, Puppet or Chef.
  • 15. Demo
  • 16. Copyright©2013,SplunkInc. Contact me 16 Email : ddallimore@splunk.com Twitter : @damiendallimore Skype : damien.dallimore Github : damiendallimore Splunkbase : damiend Slideshare : http://www.slideshare.net/damiendallimore Blogs : http://blogs.splunk.com/dev Web : http://dev.splunk.com