Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Making Pretty Charts in Splunk
Similaire à Making Pretty Charts in Splunk (20)
Dernier
Dernier (20)
Making Pretty Charts in Splunk
- 2. Johnvey Hwang Splunk Inc.
- 4. Q: Why are you making a chart? COMM153: Business Graphics, Fall 2010, Midterm exam
- 5. GraphJam.com
- 6. Step 1: Prep
- 7. What data is available?
- 8. Who is going to read it?
- 10. Decision Step 2: Visualize
- 13. Tables are sexy.
- 14. Q2 Revenue $17.5B So are numbers.
- 15. Column chart
- 16. Line chart
- 18. Area chart (100% stacked)
- 19. Is ‘ Georgian ’ half or a third of ‘ Hawaiian ’? Is this linear, geometric, or exponential? Pie charts suck
- 20. Pie chart remedy
- 21. Trending is better Pulse = 90 vs.
- 22. Time range: real-time Requires immediate response 30 second – 1 hour window
- 23. Time range: historical Multiple day – month window Time to respond is in weeks or months
- 24. Step 3: Build it
- 25. Report builder
- 26. Advanced charting view See search box and chart on one page
- 27. Search language: overview index=main error datacenter=SFO | timechart avg(bytes) by host event search clause transforming clause Over 100 search commands…
- 28. The dataset _time _raw host field1 18283495832 2010-08-10 08:52:01 ERROR: something went wrong on server Prod_apache_1 ERROR 18383827123 2010-08-10 08:52:01 INFO: redirect to a better page Prod_apache_2 INFO
- 29. Search language: timechart … | timechart span=1h count “ count the number of events per hour” … | timechart avg(delay) by host “ calculate the average delay and track each host separately” … | timechart avg(delay) min(delay) max(delay) “ calculate the average, minimum, and maximum delays per auto-bucket”
- 30. Search language: timechart index=_internal | timechart count by group
- 31. Search language: top … | top limit=50 users “ list the top 50 users” … | top major_version, minor_version “ list the top (10) combinations of major and minor versions” … | top source by host “ list the top sources grouped by most frequent host”
- 32. Search language: top index=_internal source="*access.log" | top uri_path | fields uri_path count
- 33. Search language: chart … | chart avg(delay) by sender “ list the average delay for every sender” … | chart max(bytes) over clientip by uri useother=f “ list the maximum bytes of the top ‘uri’ for every ‘clientip’”
- 34. Search language: chart index=_internal source="*access.log" | chart avg(bytes) by clientip
- 35. Search language: ctable … | ctable clientip http_status “ list every combination of ‘clientip’ and ‘http_status’ and their freqencies” … | ctable clientip http_status maxcols=10 “… restrict to a max of 10 http_status columns’”
- 36. Search language: ctable index=_internal source="*access.log" | ctable clientip status maxcols=10 maxrows=10
- 37. Search language: more stats : average, min, max, stdev, distinct count, mode, variance,… streamstats : calculate running statistics up to current event rangemap : bucket results into ranges like ‘low’, ‘medium’, ‘high’ kmeans : partition results into k-means clusters trendline : calculate moving averages accum : creates a new field of running total of any field
- 38. Step 4: Automate
- 41. Step 5: Polish
- 42. What can you change? View CSS Chart properties
- 44. Custom CSS 1. Add custom CSS files to the app: $SPLUNK_HOME/etc/apps/<APP_NAME>/appserver/static/<FOO>.css 2. Restart splunkweb (only on first create) 3. Add to view: <view stylesheet=“FOO.css”> 4. Save to: $SPLUNK_HOME/etc/apps/<APP_NAME>/default/data/ui/views/<NAME>.css
- 46. Charting properties Hundreds of different properties available Ex: Common property to change tick label visibility: charting.primaryAxis.majorLabelVisibility = hide Ex: Area chart type-specific series stacking mode: charting.AreaChart.stackMode = stacked100 Ex: Common legend label width: charting.legend.labelStyle.maximumWidth = 500
- 48. Simple XML form <dashboard> <label>My dashboard</label> <row> <chart> <searchName>My saved report</searchName> <option name=" charting.seriesColors "> [0xFF0000,0xFFFF00,0x00FF00] </option> <option name=" charting.legend.placement "> top </option> </chart> </row> </dashboard>
- 49. Advanced XML form ... <module name="HiddenChartFormatter"> <param name=" charting.seriesColors "> [0xFF0000,0xFFFF00,0x00FF00] </param> <param name=" charting.legend.placement "> top </param> <module name="FlashChart" /> </module> ...
- 50. More help Charting reference documentation http://www.splunk.com/base/Documentation/latest/Developer/AdvancedCharting http://www.splunk.com/base/Documentation/latest/Developer/ChartReference Splunk community Q&A site http://answers.splunk.com Edward Tufte - “Father of data visualization” http://www.edwardtufte.com Blogs for inspiration http://infosthetics.com http:// smashingmagazine.com Demo material http://blogs.splunk.com/author/johnvey/
Notes de l'éditeur
- Welcome!
- Intro
- There are 5 main steps you generally go through before you can have a meaningful and pleasing chart. Today we’re going to cover all 5 steps in Splunk. But before that, can you answer the obvious question…
- Do you actually have a reason for making a chart? What is the desired outcome? -- want some business intelligence? -- need to monitor something? -- or just for the hell of it? If you don’t know…
- Might I suggest GraphJam.com. -- can make lots of different charts -- can share them with friends -- can use pretty colors At least have a purpose in mind.
- Step 1: do you prep work. -- due-diligence -- mis-en-place The right questions can make your charting easier.
- Question #1: what data do you have at your disposal? -- Splunk can index anything -- any text, stream, file, packet, script Who owns the data? Are you going to have to ask permission?
- Question #2: Who is going to read it? -- Consider your audience when picking data to display. -- Technical staff generally want detailed information. -- Your boss may not. What is the reader going to do with it? -- make business decisions -- incidence response -- feel good -- look at pretty pictures -- be able to assign blame There was a story about a potential customer who was so excited about Splunk, threw everything at it. Then tried to discover who was to blame for a current production problem. Found out it was his friend. Radio silence. Splunk was that good.
- Think about what what format you want. More importantly, what will the caption be? You may end up a a slave to a less than desirable format if you just wing it. This chart: General Stanley McChrystal declared that &quot;When we understand that slide, we'll have won the war&quot; at a briefing in Kabul last summer Does not reflect well upon you.
- One way to visualize is to think about the big picture first.
- This is an example VP-level dashboard built entirely in Splunk * 60-day period * Shows medium-term trending; influences business decisions * dashboard.html template * custom view-specific CSS * HiddenSavedSearch module * HiddenChartFormatter module * SingleValue module
- This is a real-time dashboard intended for a NOC situation -- everything is on a 30-second window -- shows info that people may need to react to quickly I’ll cover both of those examples later in the talk.
- Note that both previous examples used tables! -- “visualization” doesn’t really just mean charts -- tables can be far more efficient at displaying certain kinds of data than charts
- Even simpler are single numeric values -- Apple’s Q2 revenue for example -- if context is well known, there’s no need to complicate matters with “chartjunk”
- OK, so on to charts. These are all native chart types that Splunk can render. I’ll be brief; I want to highlight how you match chart types to data. The column chart: -- great for inspecting discrete data -- can easily compare single value series on common axis -- identify trends
- The line chart: -- great for comparing multiple series -- compare on similar Y-axis
- Split line chart: -- variant of the line series -- great for when trending of individual series is wildly different
- The area chart -- use to track multiple series in relationship to each other -- in 100% form, very quickly see proportional changes over time
- The pie chart: do not use. -- cannot compare 2 slices together easily -- no common point of reference -- cannot determine distribution
- A column chart lets you actually see the data on comparable terms -- ‘Georgian’ is more than half the value of ‘Hawaiian’ -- &quot;The only worse design than a pie chart is several of them.“ – Edward Tufte You can see the trend among data (in this case it’s exponential). Why is trending important?
- Example: -- in basic medical triage you must always record vitals over time -- walking into ER and only saying your pulse is 90 is useless -- you need to know if it’s rising or falling, or stable So what are you trending over? Well Splunk likes time…
- We’re going to focus on time-based ranges and the main modes of trending: Real-time: -- still one of the coolest things about splunk -- even if you know nothing about the data, it’s still cool to see stuff come in real-time What is it good for? -- things that require immediate response -- when you only need to see at most an hour of content; 30 seconds is also useful Examples: -- network operations -- security operations -- just in time operations
- The flip side is historical; what people typically expect What is it good for? -- making business decisions based on data -- 7, 30 day moving trends Examples: -- infrastructure planning -- bandwidth usage -- peak/off-peak tracking
- Finally! We cover the tools in Splunk that can make effective charts.
- How many people are familiar with the report builder? The standard report builder is accessible from the search interface -- you can start charting searches that are still in flight -- easy dropdown-based chart building -- handles simple cases
- How many people are familiar with the advanced charting view? Advanced charting is where most of us like to chart -- direct access to the search language -- tabular view below -- has common set of charting controls
- The search language is inspired by the UNIX command line -- typically the first command is assumed to be the ‘search’ command -- any number of commands can be chained together -- there are over 100 search commands that come with Splunk How many people are familiar with the search language? Novice? Intermediate? Advanced? There were a bunch of sessions on the search language (check your preso material if you didn’t attend) I’ll go through some of the workhorse commands used to generate charts…
- Splunk search results are nothing more than a big table of data -- the event text is copied into the ‘_raw’ field -- it’s just another field in the result set -- if you understand this, you will grasp the search language easily -- the UI depends on underscore fields (which are not displayed) Knowing that this is just a table, you can use search language to transform the results any which way you want
- This is the heavy hitter for any IT ops commands -- shows you over time what something is doing -- can take any of the stats commands and generate multiple series -- can control the granularity of the bucketing
- Here is an example of the timechart command that looks at Splunk internal components over time -- uses automatic defaults to determine sensible time buckets
- The ‘top’ command does what it sounds like -- displays the top values of any field in your results -- can do top n combinations -- can specify how many ordered items to return
- Display is great when paired with a bar chart, or column -- please don’t use pie charts (will cover that in a bit)
- This is the generic version of timechart -- behind the scenes, this powers ‘timechart’ -- in essence, plots some function of field A by field B -- like ‘timechart’, you can actually invoke the eval() command
- This example shows the average bytes transferred over a time period split by client IP addresses -- this is over every piece of data the Splunk knows about -- you can restrict by time window by just setting the time range
- The contingency table is not a graphical command, but is equally as powerful -- contingency is used very often in statistical analysis: determine if variable X really has an effect on measured property Y -- essentially is a counter: will tally combinations of X/Y
- Note that the Y is ‘clientip’ and X is ‘http_status’; the numbers in the middle of the table simply show the number of occurences -- you can then apply the heatmap decorator to visually differentiate hotspots
- There are lots of other commands than can manipulate the data any which way you want.
- Let Splunk do your dirty work -- automation can alleviate much of the manual labor -- no reason to always check splunk when it can email you
- Scheduling a saved search is the best practice -- defer work to off-peak hours -- allow multiple users to share results -- have Splunk alert you when certain conditions are met -- receive results via email, RSS plug: customizing and using scripted alerts Plug: monitoring with splunk
- Assembling multiple saved searches on a dashboard is great for overviews -- use simple dashboard creator -- takes existing saved searches and lets you arrange onto a dashboard -- don’t cram a ton of searches onto a dashboard: use multiple dashboards!
- The main areas of presentation-layer customizations fall into: -- view-level CSS: full access to override any CSS rules -- chart-level properties: each chart can have its own formatting
- This is the before picture: -- default CSS -- default charting properties -- uses the standard panels -- uses standard singleValue modules -- uses default charting properties: note the axis labels, legend, colors, etc
- You can have as many different CSS files on a per-app basis -- each view can reference any CSS file that is available in its app -- not going to cover CSS customization here; plenty of online resources -- recommend Firebug+Firefox or Chrome+Web Inspector
- After: -- this is simple CSS and charting properties applied -- no other structural changes involved Source is available online
- There are many different properties you can adjust on a per-chart basis -- these are set in XML configuration -- each property name is hierarchical -- dots are used to denote hierarchy -- http://www.splunk.com/base/Documentation/latest/Developer/AdvancedCharting -- http://www.splunk.com/base/Documentation/latest/Developer/ChartReference
- * Add link to advanced XML
- Promote usability sessions! We’re hiring