5. Install Splunk
Splunk Home
• WIN: Program FilesSplunk
• Other: /opt/splunk (Applications/splunk)
Start Splunk
• WIN: Program FilesSplunkbinsplunk.exe start (services start)
• *NIX: /opt/splunk/bin/splunk start
www.splunk.com/download
• 32 or 64 Bit?
• Indexer or Universal Forwarder?
6. Splunk Licenses
Free Download Limits Indexing to 500MB/day
• Enterprise Trial License expires after 60 days
• Reverts to Free License
Features Disabled in Free License
• Multiple user accounts and role-based access controls
• Distributed search
• Forwarding to non-Splunk Instances
• Deployment management
• Scheduled saved searches and alerting
• Summary indexing
Other License Types
• Enterprise, Forwarder, Trial
7. Default installation on: http://localhost:8000
7
Splunk Web Basics
Browser Support
• Firefox 10.x and latest
• Internet Explorer 7, 8, 9 and 10
• Safari (latest)
• Chrome (latest)
Index data
• Add data
• Getting Started App
• Install an App (Splunk for Windows, *NIX)
8. 8
Splunk Web Basics continued…
Splunk Home
• Provides Interactive portal to the Apps & data.
• Includes a search bar and three panels:
1 – Apps 2 – Data 3 - Help
Splunk Apps
• Splunk Home Find more apps
• Provide different contexts for your data out of
sets of views, dashboards, and configurations
• Default Search App
• You can create your own!
9. Optional: add some test data
Download the sample file, follow this link and save the file to your
desktop, then unzip: http://www.splunkbook.com (Using Splunk Book)
To add the file to Splunk:
– From the Welcome screen, click Add Data.
– Click From files and directories on the bottom half of the screen.
– Select Skip preview.
– Click the radio button next to Upload and index a file.
– Click Save.
9
10. Best Practice Suggestion:
Create an individual Index based on
sourcetype.
• Easier to re-index data if you make a
mistake.
• Easier to remove data.
• Easier to define permissions and data
retention.
10
12. Search app – Summary viewcurrent view
global stats
app navigation time range
picker
Selecting Data
Summary:
• Host
• Source
• Sourcetype
start
search
search box
13. Searching
13
Search > *
Select Time Range
• Historical, custom, or real-time
Select Mode
• Smart, Fast, Verbose
Using the timeline
• Click events and zoom in and out
• Click and drag over events for a specific range
14. 14
Everything is searchable
Everything is searchable
• * wildcards supported
• Search terms are case insensitive
• Booleans AND, OR, NOT
– Booleans must be uppercase
– Implied AND between terms
– Use () for complex searches
• Quote phrases
fail*
fail* nfs
error OR 404
error OR failed OR (sourcetype=access_*(500 OR 503))
"login failure"
16. Search Assistant
16
Contextual Help
- advanced type-ahead
History
- search
- commands
Search Reference
- short/long description
- examples
suggests search terms
updates as you type
shows examples and help
toggle off / on
17. Searches can be managed as
asynchronous processes
Jobs can be
• Scheduled
• Moved to background tasks
• Paused, stopped, resumed, finalized
• Managed
• Archived
• Cancelled
Job Management
Modify Job Settings
pause
finalize
delete
17
18. Search Commands
18
Search > error | head 1
Search results are “piped” to the command
Commands for:
• Manipulating fields
• Formatting
• Handling results
• Reporting
21. Fields
21
Default fields
• host, source, sourcetype, linecount, etc.
• View on left panel in search results or all in field picker
Where do fields come from?
• Pre-defined by sourcetypes
• Automatically extracted key-value pairs
• User defined
22. Sources, Sourcetypes, Hosts
• Host
- hostname, IP address,
or name of the network
host from which the
events originated
• Source
- the name of the file,
stream, or other input
• Sourcetype
- a specific data type or
data format
2
2
23. 23
Tagging and Event Typing
Eventtypes for more human-readable reports
• to categorize and make sense of mountains of data
• punctuation helps find events with similar patterns
Search > eventtype=failed_login instead of
Search > “failed login” OR “FAILED LOGIN” OR “Authentication failure” OR “Failed to
………………authenticate user”
Tags are labels
• apply ad-hoc knowledge
• create logical divisions or groups
• tag hosts, sources, fields, even eventtypes
Search > tag=web_servers instead of
Search > host=“apache1.splunk.com” OR host=“apache2.splunk.com” OR
…………….host=“apache3.splunk.com”
29. Alerting Continued…
29
Searches run on a schedule and fire an alert
• Example: Run a search for “Failed password” every 15 min
over the last 15 min and alert if the number of events is
greater than 10
Searches are running in real-time and fire an alert
• Example: Run a search for “Failed password user=john.doe” in
a 1 minute window and alert if an event is found
32. Reporting
32
results of any search
Define your Search and set your time range,
accelerate you search and more Choose the type of chart (line, area, column, etc) and
other formatting options
Build reports from
33. Reporting Examples
33
• Use wizard or reporting commands (timechart, top, etc)
• Build real-time reports with real-time searches
• Save reports for use on dashboards
36. Manager Settings
36
For All of that Cool Stuff
You Just Created (and more!)
• Permissions
• Saved Searches/Reports
• Custom Views
• Distributed Splunk
• Deployment Server
• License Usage….
38. Splunk Has Four Primary Functions
38
• Searching and Reporting (Search Head)
• Indexing and Search Services (Indexer)
• Local and Distributed Management (Deployment Server)
• Data Collection and Forwarding (Forwarder)
A Splunk install can be one or all roles…
39. Getting Data Into Splunk
39
Agent and Agent-less Approach for Flexibility
perf
shell
code
Mounted File Systems
hostnamemount
syslog
TCP/UDP
WMI
Event Logs Performance
Active
Directory
syslog compatible hosts
and network devices
Unix, Linux and Windows hosts
Windows hosts Custom apps and scripted API connections
Local File Monitoring
log files, config files
dumps and trace files
Windows Inputs
Event Logs
performance counters
registry monitoring
Active Directory monitoring
virtual
host
Windows hosts
Scripted Inputs
shell scripts custom
parsers batch loading
Agent-less Data Input Splunk Forwarder
40. Understanding the Universal Forwarder
40
Forward data without negatively impacting production performance.
Scripts
Universal Forwarder Deployment
Logs ConfigurationsMessages Metrics
Central Deployment Management
Monitor files, changes and the system registry; capture metrics and status.
Universal Forwarder Regular (Heavy) Forwarder
Monitor All
Supported
Inputs
✔ ✔
Routing,
Filtering,
Cloning
✔ ✔
Splunk Web ✔
Python
Libraries
✔
Event Based
Routing
✔
Scripted
Inputs
✔
43. High Availability, On Commodity Servers and Storage
43
As Splunk collects data, it keeps
multiple identical copies
If indexer fails, incoming data
continues to get indexed
Indexed data continues to be
searchable
Easy setup and administration
Data integrity and resilience
without a SAN
Index Replication
Splunk Universal
Forwarder Pool
Constant
Uptime
45. Integrate External Data
45
LDAP, AD Watch
Lists
CRM/ER
P
CMDB
Correlate IP addresses with locations, accounts with regions
Extend search with lookups to external data sources.
46. Integrate Users and Roles
46
Problem Investigation Problem Investigation Problem Investigation
Save
Searches
Share
Searches
LDAP, AD
Users and Groups
Splunk Flexible Roles
Manage
Users
Manage
Indexes
Capabilities& Filters
NOT
tag=PCI
App=ERP
…
Map LDAP & AD groups to flexible Splunk roles. Define any search as a filter.
Integrate authentication with LDAP and Active Directory.
50. Support Through the Splunk Community
50
Browse and share Apps
from Splunk, Partners and
the Community
apps.splunk.com
Splunkbase
Community-driven
knowledge exchange
and Q&A
answers.splunk.com
5 tracks, more than 40
sessions, the smartest
Splunk users together
conf.splunk.com
.conf2014
51. Where to Go for Help
51
• Documentation
– http://www.splunk.com/base/Documentation
• Technical Support
– http://www.splunk.com/support
• Videos
– http://www.splunk.com/videos
• Education
– http://www.splunk.com/goto/education
• Community
– http://answers.splunk.com
– http://apps.splunk.com
• Splunk Book
– http://splunkbook.com