Integrating Splunk into your Spring Applications
•Télécharger en tant que PPTX, PDF•
7 j'aime•11,331 vues
How much visibility do you really have into your Spring applications? How effectively are you capturing,harnessing and correlating the logs, metrics, & messages from your Spring applications that can be used to deliver this visibility ? What tools and techniques are you providing your Spring developers with to better create and utilize this mass of machine data ? In this session I'll answer these questions and show how Splunk can be used to not only provide historical and realtime visibility into your Spring applications , but also as a platform that developers can use to become more "devops effective" & easily create custom big data integrations and standalone solutions.I'll discuss and demonstrate many of Splunk's Java apps,frameworks and SDK and also cover the Spring Integration Adaptors for Splunk.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (12)
Similaire à Integrating Splunk into your Spring Applications
Similaire à Integrating Splunk into your Spring Applications (20)
Plus de Damien Dallimore
Plus de Damien Dallimore (10)
Dernier
Dernier (20)
Integrating Splunk into your Spring Applications
- 1. © 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission. Integrating Splunk into your Spring Applications By Damien Dallimore
- 2. About Me
- 3. Developer Evangelist at Splunk 3 TalkMake
- 4. Came from the Splunk Community 4
- 5. Coder 5
- 6. From Aotearoa (New Zealand) 6
- 7. Agenda
- 8. Agenda Splunk Overview Splunk Developer Platform Integrating Splunk and your Spring App Questions (time allowing, else see me after)
- 11. Splunk is a Platform for Machine Data 11 Developer Platform Data collection and indexing Report and analyze Custom dashboards Monitor and alert Ad hoc search HA Indexes and Storage Commodity Servers
- 12. What Does Machine Data Look Like? 12 Twitter Care IVR Middleware Error Order Processing
- 13. Machine Data Contains Critical Insights 13 Customer ID Order ID Customer’s Tweet Time Waiting On Hold Twitter ID Product ID Company’s Twitter ID Twitter Care IVR Middleware Error Order Processing Customer IDOrder ID Customer ID
- 14. Machine Data Contains Critical Insights 14 Order ID Customer’s Tweet Time Waiting On Hold Product ID Company’s Twitter ID Twitter Care IVR Middleware Error Order Processing Order ID Customer ID Twitter ID Customer ID Customer ID
- 16. Developer Platform JavaScript Java Python PHP C# Ruby REST API DevOps Integrate Build
- 17. Splunk REST API • An API method for all features of the platform • Send data in • Search and Export data out • JSON , CSV, XML
- 18. Integrating Splunk and your Spring App
- 19. How can Splunk help out the Spring Developer ? • During Dev/Test – Use Splunk to deliver deeper insights , hook in more thorough test case assertions – Aggregate data from your apps in development • Integrate the Data you have collected with Splunk – Use Spring as the EAI backbone to build integrated data solutions , correlate data form numerous sources • Build standalone Big Data apps – Let Splunk do the hard yards on the data and searching side
- 20. What are some of the hooks? • REST API • Splunk SDK for Java • Spring Integration Adaptors • Logging • JMS Messaging • JMX MBeans • REST • BCI(byte code injection) tracing
- 21. Splunk SDK for Java
- 22. Splunk SDK for Java • Open sourced under the Apache v2.0 license • git clone https://github.com/splunk/splunk-sdk-java.git • JRE 6+ • Maven/Gradle Repository • Code Examples : – Connect – Hit your first endpoint – Send data in – Search for data , Simple and Realtime – Scala and Groovy can play too
- 23. SDK Class Design 23 Service HTTPService Resource ResourceCollection Entity EntityCollection JobJobCollection BaseService Args JobResultsArgs
- 25. When Spring and Splunk collide • Developers like tools & frameworks that increase productivity • An SDK makes it easier to use a REST API • A declarative Enterprise Integration framework makes it easier to build solutions that need to integrate, transform, filter and route data from heterogeneous channels and data sources • We now have the Spring Integration Splunk Adaptors to make it easier for Java Developers to integrate Splunk into their solutions utilizing a semantic they are most familiar with in the Spring framework.
- 26. Spring Integration And Splunk Inbound Adapter – Used to execute Splunk searches and produce messages containing results – Search modes: BLOCKING, NORMAL, REALTIME, EXPORT, SAVEDSEARCH – Date/Time Ranges Outbound Adapter – Write system or application events to Splunk – Write to a named index, submit a REST request, write to a data input bound to a server TCP port Message payload for Splunk I/O adapters is SplunkEvent
- 27. Spring Integration Twitter adaptor polls for tweets Spring Integration Splunk outbound adaptor sends events to Splunk via HTTP REST Realtime or historical search from SplunkWeb Raw events from Twitter are transformed into best practice logging format Splunk Java SDK Tweets
- 28. Logging
- 29. SplunkJavaLogging • A logging framework to allow developers to as seamlessly as possible integrate Splunk best practice logging semantics into their code • Transport log events to Splunk directly from your code • Custom handler/appender implementations(REST and Raw TCP) for common Java logging frameworks . • LogBack • Log4j (Log4j2 coming also) • java.util.logging • Utility classes for formatting log events • Configurable in memory buffer to handle network outages 29
- 30. Developers just log as they are used to 30 2012-08-07 15:54:06:644+1200 name="Failed Login" event_id="someID" app="myapp" user="jane" somefieldname="foobar" Better A-HA
- 31. Semantic Logging Log anything that can add value when aggregated, charted or further analyzed Example Bogus Pseudo-Code: void submitPurchase(purchaseId) { log.info("action=submitPurchaseStart, purchaseId=%d", purchaseId) //these calls throw an exception on error submitToCreditCard(...) generateInvoice(...) generateFullfillmentOrder(...) log.info("action=submitPurchaseCompleted, purchaseId=%d", purchaseId) } • Create Human Readable Events • Clearly Timestamp Events • Use Key-Value Pairs (JSON Logging) • Separate Multi-Value Events • Log Unique Identifiers
- 33. Java stacktraces are a nuisance 33
- 34. SplunkJavaLogging is your friend 34
- 35. Java stacktraces in Splunk unravelled 35
- 36. JMS Messaging
- 37. JMS Messaging Splunk Input 37 • JMS is an interface that abstracts your underlying MOM provider implementation • Send messages to parallel queues or topics in Spring that Splunk can tap into • Index messages from : • MQ Series / Websphere MQ • Tibco EMS • ActiveMQ • HornetQ • RabbitMQ • SonicMQ • JBoss Messaging • Weblogic JMS • Native JMS • StormMQ • Note : Non-JMS inputs also available (Stomp , ZeroMQ)
- 38. JMS input fully integrated into Splunk 38
- 39. Add a new queue/topic input 39
- 40. Configure the properties to connect 40
- 41. Get instant operational visibility 41
- 42. JMX
- 43. Expose Mbeans in your Spring App • 3 tiers can be exposed – JVM (java.lang domain) – Framework / Container (Spring , Tomcat etc…) – Application (whatever you have coded) • Attributes , Operations and Notifications • Use Splunk for JMX to monitor the internals of your running Spring apps
- 44. Splunk for JMX • Multiple connectivity options – rmi/iiop ,direct process attachment, mx4j http connectors • Works with all JVM variants • Scales out to monitor large scale JVM infrastructures
- 45. REST
- 46. REST is easy with Spring • Create an endpoint with Spring Integration • Splunk can poll the REST API endpoint • Multiple authentication mechanisms • Custom response handling / pre-processing • Send responses in XML , JSON • Splunk can natively index the responses and then simply search over the auto extracted fields
- 47. REST : The Data Potential • Twitter • Foursquare • LinkedIn • Facebook • Fitbit • Amazon • Yahoo • Reddit • YouTube • Flickr • Wikipedia • GNIP • Box • Okta • Datasift • Google APIs • Weather Services • Seismic monitoring • Publicly available socio-economic data • Traffic data • Stock monitoring • Security service providers • Proprietary systems and platforms • Other “data related” software products • The REST “dataverse” is vast , but I think you get the point. 47 There is a world of data out there available via REST that can be brought into Splunk, correlated and enriched against your existing data, or used for entirely new uses cases that you might conceive of once you see what is available and where your data might take you.
- 48. BCI(Byte Code Injection) Tracing
- 49. Splunk Java Agent 49 An instrumentation agent for tracing code level metrics via bytecode injection, JMX attributes/operations/notification and decoded HPROF records and streaming these events directly into Splunk https://github.com/damiendallimore/SplunkJavaAgent • class loading • method execution • method timings (cumulative, min, avg, max, std deviation) • method call tracing(count of calls, group by app/app node(for clustered systems)/thread/class/package) • method parameter and return value capture (in progress) • application/thread stalls , thread dumps and stacktraces • errors/exceptions/throwables • JVM heap analysis, object/array allocation count/size,class dumps, leak detection, stack traces, frames • JMX attributes/operations/notifications from the JVM or Application layer MBean Domains
- 50. Design goals 50 • Just pull out the raw metrics , then let Splunk perform the crunching • Format events in best practice semantic , well defined key value pairs , tagged events help correlation across distributed environment • Low impact to the instrumented application • No code changes required • Flexible configuration • Extensible • Generic open source agent , I may have used some Splunk terms in the naming conventions, but it is still completely generic , anyone want to collaborate ! • Not a full blown APM solution , just pulling raw data. • Incorporate into your Spring apps during Dev/Test to get deeper insights
- 51. Setup should be as simple as possible 51 This is all you pass to the JVM at startup : -javaagent:splunkagent.jar Everything required by the agent is built into the one single jar file We also have a new Eclipse plugin that incorporates this functionality if you don’t want to setup the JVM command line argument manually.
- 52. Configuration should allow for flexibility 52
- 53. Raw events streamed into Splunk 53
- 54. Use Splunk to derive insights 54
- 55. A couple of other integrations you may like
- 56. Let’s integrate some mobile data
- 57. Android SDK project 57 • Cousin to the Splunk SDK for Java, has all the same functionality and code examples are the same • Utility classes to make common tasks easier • logging to Splunk • searching Splunk • pulling system/device/sensor metrics from Android and logging to Splunk • Send Android data to Splunk and then use the Spring Integration Splunk Inbound adaptor to integrate this into your Spring applications. • Community preview currently published to Github
- 58. Making Hadoop analytics easier
- 59. HUNK (Splunk Analytics for Hadoop) • A new product offering from Splunk , currently in Beta preview • Allows you to use the power and simplicity of Splunk to search over data locked away in HDFS • Sits on top of HDFS as if it was a native Splunk Index • Virtual Indexes • So you can use the Spring Integration Splunk Inbound Adaptor to search over data in HDFS, or correlate your HDFS data with other data you have indexed and integrate it into your Spring Applications. 59
- 60. Splunk sits on top of HDFS Hadoop Storage Immediately start exploring, anal yzing and visualizing raw data in Hadoop 1 2Point Splunk at Hadoop Cluster Explore Analyz e Visualiz e Dashboard s Share
- 62. Where to Go for More Info 62 Twitter @splunkdev Blog http://blogs.splunk.com/dev Demos http://demos.splunk.com Email devinfo@splunk.com Portal http://dev.splunk.com Github https://github.com/splunk
- 63. Easy to Get Started Download and install in minutes 3. Start Splunking1. Download 2. Eat your Machine Data
- 64. splunk.com/goto/conf 4th Annual Event 3+ days | 100+ sessions | 30+ customer sessions 1,500+ IT Pros | 20+ partners 2 days of Splunk University Specialist Tracks: CIO, Big Data, Executive Sept 30 – Oct 3 Las Vegas
- 65. Links Github Gists for SDK code examples : https://gist.github.com/damiendallimore SDK docs at dev.splunk.com : http://dev.splunk.com/view/SP-CAAAECN Splunk SDK for Java Github repository : https://github.com/splunk/splunk-sdk-java Maven/Gradle/Ivy Repository : http://splunk.artifactoryonline.com/splunk/ext-releases- local Splunk Spring Integration repository on Github : https://github.com/SpringSource/spring- integration-extensions/tree/master/spring-integration-splunk Splunk Spring Integration demo on Github : https://github.com/damiendallimore/spring- integration-splunk-webex-demo Splunk Eclipse plugin : http://dev.splunk.com/view/splunk-plugin-eclipse/SP-CAAAEQP Splunk Java Logging on Github : https://github.com/splunk/splunk-library-javalogging 65
- 66. Links cont…. Splunk Java Agent on Github : https://github.com/damiendallimore/SplunkJavaAgent Splunk Android SDK on Github : https://github.com/damiendallimore/splunk-sdk- android Splunk REST API reference : http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTcontents Free Splunk download : http://www.splunk.com/get?r=header Best practice logging overview : http://dev.splunk.com/view/logging-best- practices/SP-CAAADP6 Splunk SDK for Java videos : http://dev.splunk.com/view/get-started/SP- CAAAECH HUNK Beta video : http://www.splunk.com/view/SP-CAAAH2F
- 67. Contact me 67 Email : ddallimore@splunk.com Twitter : @damiendallimore Skype : damien.dallimore Github : damiendallimore Splunkbase : damiend Slideshare : http://www.slideshare.net/damiendallimore
- 68. Thanks !
Notes de l'éditeur
- Developer Evangelist at SplunkBuild various apps,add-ons & tools across the Splunk developer landscape , talk about it , get other developers knowledgeable and interested in our various developer hooksSplunk architect/admin and community collaboratorI was once a customer and was spending a lot of time on Splunkbase (still do !)CoderMostly in the Java ecosystem, but have several languages in my batbelt, generally interested in coding anything that’s useful to someone, use the right language for the job.Born and raised in Aotearoa (New Zealand)
- Tee shirtDividiedloyaltysKudos to elison for a great spectacle
- Massive amounts of “data” are being generated everydayWe are often blind to where that data is and what that data is telling us , it lives buried away in a dark caveWe can navigate our way through the murk and make discoveries if we have the right equipment and know how (big data tools and techniques )Not a teeshirt company , not a twitter analysis company , not a twitter competition platform
- Where we started founders , where we areIntegrated platform for data collection, searching & correlation and visualizationReal time visibility of events, Real time alertingNo need to write Map Reduce jobsOpen and extensibleNumerous ways to get your data into Splunk (TCP,UDP,REST,File monitoring, custom inputs etc..)“Schema on the fly”, no need to define/enforce structure up frontHighly available distributed architecture allows Splunk to scale out to TB’s per day , Index replicationStream in data directly from your application codeProvides comprehensive controls for data security, retention and integritySplunkbase community , heaps of free apps and add-ons, community supportInternet of things , elevators , SCADA , cars, medical devices , sensors, wifi , camera , mics , Mobile , rfid, network packets , SIEM quadrant , vending machines, fitbitetc..Splunk’sflagship product is Splunk Enterprise. Splunk Enterprise is a fully featured, powerful platform for collecting, searching, monitoring and analyzing machine data.Splunk collects machine data securely and reliably from wherever it’s generated. It stores and indexes the data in real time in a centralized location and protects it with role-based access controls. You can even leverage other data stores. Splunk lets you search, monitor, report and analyze your real-time and historical data. Now you have the ability to quickly visualize and share your data, no matter how unstructured, large or diverse it may be. Troubleshoot problems and investigate security incidents in minutes (not hours or days). Monitor your end-to-end infrastructure to avoid service degradation or outages. Gain real-time visibility and critical insights into customer experience, transactions and behavior. Use Splunk and make your data accessible, usable and valuable across the enterprise.
- Unlike traditional structured data or multi-dimensional data– for example data stored in a traditional relational database for batch reporting – machine data is non-standard, highly diverse, dynamic and high volume. You will notice that machine data events are also typically time-stamped – it is time-series data. Take the example of purchasing a product on your tablet or smartphone: the purchase transaction fails, you call the call center and then tweet about your experience. All these events are captured - as they occur - in the machine data generated by the different systems supporting these different interactions. Each of the underlying systems can generate millions of machine data events daily. Here we see small excerpts from just some of them. Volume , velocity , variety , veracity ,valueStitchSingle pain of glass
- When we look more closely at the data we see that it contains critical information – customer id, order id, time waiting on hold, twitter id … what was tweeted. What’s important is first of all the ability to actually see across all these disparate data sources, but then to correlate related events across disparate sources, to deliver meaningful insight.
- Semantic logging:Create Human Readable EventsClearly Timestamp EventsUse Key-Value Pairs (JSON Logging)Separate Multi-Value EventsLog Unique Identifiers
- What have developers been building using Splunk Enterprise? Examples include the following:Run searches and retrieve Splunk data from existing Customer Service/Call Center applications (Comcast use case) Integrate Splunk data into existing BI tools and dashboard (Tableau, MS Excel)Build mobile applications with KPI dashboards and alerts powered by Splunk (Otto Group use case)Log directly to Splunk from remote devices (Bosch use cases)Build customer-facing dashboards powered by user-specific data in Splunk (Socialize, Hurricane Labs use cases)Programmatically extract data from Splunk for long-term data warehousing
- Code examples from Eclipse
- Service is a wrapper that facilitates access to all Splunk REST endpointsCollections use a common mechanism to create and remove entitiesEntities use a common mechanism to retrieve and update property values, and access entity metadataArgs provide a POJO getter/setter paradigm fro REST API endpoint arguments
- David TuranskiJarrod leiDemo : Spring outbound twitter + splunk search + create dashboards + splunk booth exampleSpring inbound search , console output
- Writing to file it nice , buffers things , but sometimes there are constraints that forbid this , disk space etc..Bureaucratic constraints to depoing collectors / forwarders.
- Demo : Splunk java logging code example + Splunk search
- Demo : Spring integration code + active MQ with Splunk search
- Splunk Java Agent demo from Eclipse with Splunk searches
- Demo :Show shome data off my Nexus going into splunkcowboy.comSimple search exampleHaversine
- Splunk Enterprise is simple to deploy, scales from a single server deployment to global large-scale operations and delivers fast payback. Download Splunk Enterprise for free, install it in 5 minutes on your laptop or on any commodity server, point it at any machine data and start using it. Splunk software is often deployed for the first time while under fire. A serious service outage or security incident in progress is stressful, but with Splunk Enterprise, you can complete your investigation in a few minutes versus hours or days.
- Developer track , hackathon , and vegas !.conf2013 war our 3rd annual conferenceHeld in Las Vegas at The Cosmopolitan Hotel in September.Goal here is to make our customers smarter, because smarter customers find new ways to use Splunk and tell their colleagues to use Splunk. Specific conference goals:Help customers answer: Where will your data take you?Empower customers with knowledgeFoster deep, supportive relationships within the Splunk communityGarner rich feedback and input to create a better SplunkReinforce Splunk CommunityEquip Customers and Partners with skills for successCreate channel for sharing best practices—expanding use casesLive, in-person venue for trainingFoundation for everything Splunk--future Users’ Conferences, regional user groups, fueling Splunkbase and Splunk Answers…