Learn the latest and greatest on the Splunk Java SDK. Demos, Code, Code and more Code.

1. Using the Splunk Java SDK
Presented by Damien Dallimore
Developer Evangelist at Splunk
Copyright © 2012 Splunk Inc.

2. About me
• Developer Evangelist at Splunk since July 2012
• http://dev.splunk.com
• http://splunk-base.splunk.com
• Slides available for my “Splunking the JVM” session
• Splunk Community Member
• Splunk4JMX
• SplunkJavaLogging
• SplunkBase Answers
• Splunk Architect and Administrator
• Coder, hacker, architect of Enterprise Java solutions around the globe in
many different industries(aviation, core banking, card payments etc…)
• Yes, I do have an accent , so please restrain all your sheep, Lord of the
Rings and Kim Dotcom heckles until beer o’clock 
2

3. Agenda
• Overview of the Splunk Platform
• REST API & SDKs
• Java SDK overview
• Code, Code, Code !
• Thinking outside the Square
• Alternate JVM Languages
• Making it easier for developers to log to Splunk
• Splunk is not just for Production
• Questions (feel free to yell out at any time also)

4. Splunk & Developers
Accelerate development &
Machine Data SplunkUI Custom/Existing
testing with proactive
(Splunk Apps) Applications
monitoring
SDKs
Search, chart and graph
Save and schedule searches as alerts
Integrate data from Splunk into
Export search results your existing IT environment for
Manage inputs and indexes
Add & remove users and roles
operational visibility
REST APIs
Quickly deliver real-time
Splunkd business insights from Big Data
outside of IT
4

5. REST API & SDKs

6. What you can do with the SDKs & API
• Integrate with third-party reporting tools and portals
• Log directly to Splunk
• Integrate Splunk search results into your application
• Extract data for archiving, compliance
• Build a custom UI of your choice
6

7. Splunk REST API
• Exposes an API method for every feature in the product
• Whatever you can do in the UI – you can do through the API.
• Run searches
• Manage Splunk configurations
• API is RESTful
• Endpoints are served by splunkd
• Requests are GET, POST, and DELETE HTTP methods
• Responses are Atom XML Feeds
• JSON coming in 5.0
• Versioning coming in 5.0
• Search results can be output in CSV/JSON/XML
7

8. Language SDKs
• The SDKs make it easier for you to use the raw REST API ,
abstracting away much of the lower level plumbing, so you can
instead just focus on developer productivity
• Handling HTTP access
• Authenticating
• Managing namespaces
• Simplifying access to REST endpoints
• Building the correct URL for an endpoint
• Displaying simplified output for searches
• Input of data to a Splunk index
• Python, Java, Javascript in beta - Supported
• PHP available now!
• Still study the core REST API though , if you’re anything like me
you like to know what is going on under the hood
8

9. Java SDK overview

10. Java SDK Design Principles
• Provide comprehensive coverage of the REST API
• Have a 1:1 mapping of endpoint to class in the SDK
• Provide implementation that felt intuitive to a Java developer
• Lowest common denominator for build – ANT (Any maven people out there?)
• Project support for Eclipse and IntelliJ – to ease getting started
10

11. Get the Java SDK setup
• Open sourced under the Apache v2.0 license
• Clone from Github : git clone https://github.com/splunk/splunk-sdk-java.git
• Current release status is “beta”
• Project level support for Eclipse and Intellij IDE’s
• I use Eclipse with the eGit plugin
• Pre-requisites
• JRE 6+
• Ant (builds, javadoc generation)
• Splunk installed
• Run the unit tests and examples
• Setup a “.splunkrc” file in your user’s home directory
• Run an Ant build
• Run examples with the command line wrappers
• Run the Junit tests from Ant or within your IDE
11

12. Key Java SDK Concepts
• Namespaces
• owner : splunk username
• app : app context
• sharing : user | app | global | system
• Defaults to current user and default app
• Service class
• Instantiate an object to connect and login
• Entry point for REST API calls
• Client/Server state
• Need to maintain state explicitly
• update() : to push changes to splunkd
• refresh() : to get changes from splunkd
12

13. Java SDK Class Model
HTTPService Resource
Service ResourceCollection Entity
EntityCollection Application Index Input
InputCollection SavedSearchCollection
• Collections use a common mechanism to create and remove entities
• Entities use a common mechanism to retrieve and update property values, and access entity metadata
• Service is a wrapper that facilitates access to all Splunk REST endpoints
13

14. public String codeTime(){
return “Lets Rock n Roll”;
}

15. Connecting / Authenticating
15

16. Simple Entity Retrieval
16

17. Logging Events via HTTP REST
Uses receivers/simple endpoint Uses receivers/stream endpoint
17

18. Logging Events via Raw TCP
If you don’t already have a TCP port listening, simply create one via the REST API
Setup Log to Splunk Teardown
18

19. Searching Overview
• Search query
• a set of commands and functions you use to retrieve events from an index or a real-time stream , "search *
| head 10".
• Saved search
• a search query that has been saved to be used again and can be set up to run on a regular schedule
• Search job
• an instance of a completed or still-running search operation.Using a search ID you can access the results of
the search when they become available. Job results are saved for a period of time on the server and can be
retrieved
• Search Modes
• Normal : asynchronous , poll job for status and results
• Blocking : synchronous , a job handle is returned when search is completed
• Oneshot : synchronous , no job handle is returned, results are streamed
• Export : synchronous, not a search per say, doesn’t create a job, results are streamed oldest to newest
Heaps more juicy examples here : http://dev.splunk.com/view/SP-CAAAEHQ
19

20. Blocking Searches
A Job is created No Job is created
20

21. Non-Blocking Search
21

22. Non-Blocking Search (with Paging)
• “maxresultrows” in Splunk config default 50K
• Not recommended to change this
• If result set > 50K , then page through results
22

23. Realtime Search
23

24. Saved Search
24

25. Processing CSV/JSON/XML results
25

26. Client/Server State
26

27. Namespaces
27

28. Thinking outside the square

29. Alternate JVM Languages
Scala Groovy Clojure
Javascript(Rhino) JRuby PHP(Quercus)
Ceylon Kotlin Jython
We don’t need SDK’s for these languages , we can
just use the Java SDK !
29

30. Scala “SDK”
30

31. Groovy “SDK”
31

32. SplunkJavaLogging
• A logging framework to allow developers to as seamlessly as
possible integrate Splunk best practice logging semantics into
their code.
• Custom handler/appender implementations(REST and Raw
TCP) for the 3 most prevalent Java logging frameworks in
play. Splunk events directly from your code.
• LogBack
• Log4j
• java.util.logging
• Implementation of the SPLUNK CIM(Common Information
Model)
32

33. Developers just log as they are used to
Better
A-HA
2012-08-07 15:54:06:644+1200 name="Failed Login" event_id="someID" app="myapp" user="jane" somefieldname="foobar"
33

34. Logging Framework takes care of the Splunk
transport , REST or Raw TCP
34

35. <barf>Typical Java Stacktraces in logs</barf>
35

36. SplunkJavaLogging is your friend
36

37. Java Stacktraces in Splunk
37

38. Use Splunk in dev/test => better quality
delivered to prod
Testing Tools • It’s not good enough to assert that your
Splunk
software is production ready because the
Java SDK load test “doesn’t make it fall over”
Execute Tests • Splunk the app’s machine data throughout
REST
Perform test assertions the lifecycle of the test
• Via a Splunk SDK, enrich your test harness
Splunk REST% TCP
/% pass/fail assertions with outputs of Splunk
Your Java SDK
SplunkD
searches
App Other Universal% • Catch malignant code tumors that may have
Metrics Forwarder flown under the radar that your harness
alone couldn’t possibly know about.
• Grinder + Java SDK + Assertions
38

39. Contact Details
Always more than happy to be contacted for questions,
feedback, collaborations, ideas that will change the world etc…
Email : ddallimore@splunk.com
SplunkBase: damiend
Github: damiendallimore
Twitter : @damiendallimore
Blog : http://blogs.splunk.com/dev
Splunk Dev Platform Team : devinfo@splunk.com
39

40. Links
Gists for all code examples : https://gist.github.com/damiendallimore
Java SDK Homepage : http://dev.splunk.com/view/java-sdk/SP-CAAAECN
Java SDK Github repository : https://github.com/splunk/splunk-sdk-java
SplunkJavaLogging : https://github.com/damiendallimore/SplunkJavaLogging
Splunk Best Practice Logging : http://dev.splunk.com/view/logging-best-
practices/SP-CAAADP6
Splunk REST API :
http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTcontents
40

41. //Thanks for coming !
System.exit(5150);