The document discusses machine data, which refers to data generated by technology like sensors, servers, databases, and mobile devices. It notes that machine data comes in huge volumes and has high velocity, variety, and variability. The document explores some challenges with machine data, like how a lot of it is in raw, unstructured formats that are difficult for humans to understand. It suggests that data languages and tools are improving to help humans analyze machine data. The document encourages reaching out to vendors to demand real-time streaming of machine data in JSON format to help more people leverage insights from it.

1. @michaelwilde, Co-CTO, Splunk
Exploring
Machine Data

2. Hi... I work at Splunk.

3. We stare at data all day.

5. WTF is Machine
Data?!

6. is it logs?

7. is it netflow?

8. is it TWEETS?

9. Aaaahhh, well... kind of.

10. a simple way to describe
the exhaust from technology
*or a big giant pain in the butt.

11. Machine data is the BIGgest
DATA
Machine-generated data is one of the
GPS,
fastest growing, most complex RFID,
and most valuable segments of big data Hypervisor,
Web Servers,
Email, Messaging
Clickstreams, Mobile,
Telephony, IVR, Databases,
Sensors, Telematics, Storage,
Servers, Security Devices, Desktops
Volume | Velocity | Variety | Variability

13. no, not us
we’re just
nice guys
who want
show you
cool stuff

14. building a service?
you are a producer and
consumer of data
using an app?

15. Seth Rabinowitz James Rodmell
CEO CTO
Location-­‐Based
Messaging
and
Intelligence
For
Your
App
and
Your
Customers

16. DATE/TIME
Data! Good!
DEVICE ID
2011-11-06 11:57:31,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.75496,-73.963853,60
2011-11-06 12:17:32,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.755001,-73.963886,70
2011-11-06 12:37:33,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.754982,-73.963849,75
LAT/LONG
2011-11-06 12:57:34,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.754984,-73.963883,85
2011-11-06 13:17:35,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.754941,-73.9639,90
2011-11-06 13:37:36,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.754948,-73.963874,90
2011-11-06 13:57:37,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.754931,-73.963892,95
BATTERY STRENGTH
2011-11-06 14:17:38,50,00027d27-ae02-627d-a79a-fa0004d3a347,40.755232,-73.963522,100
2011-11-06 14:37:33,65,00027d27-ae02-627d-a79a-fa0004d3a347,40.754979,-73.9639,100

17. show them something
cool already!

18. Oh, real quick. Did you check in
or tweet #splunk #interop
...please

19. All this data can be pretty cool
and empowering

20. except one little
PROBLEM Text

21. alot of it looks like this

22. 0,1
13/Apr/2011 08:52:53,Info,Teardown,ASA-session-6-302014,TCP,
192.168.2.16,192.168.1.6,(empty),(empty),1100,43025,43025_tcp,
(empty),0,1
13/Apr/2011 08:52:55,Info,Teardown,ASA-session-6-302014,TCP,
192.168.2.75,192.168.1.6,(empty),(empty),1048,135,epmap,(empty),
0,1
13/Apr/2011 08:52:55,Info,Teardown,ASA-session-6-302014,TCP,
192.168.2.75,192.168.1.6,(empty),(empty),1049,43025,43025_tcp,
(empty),0,1
13/Apr/2011 08:52:55,Info,Teardown,ASA-session-6-302014,TCP,
192.168.2.75,192.168.1.6,(empty),(empty),1051,135,epmap,(empty),
0,1
13/Apr/2011 08:52:55,Info,Teardown,ASA-session-6-302014,TCP,
192.168.2.75,192.168.1.6,(empty),(empty),1052,43025,43025_tcp,
(empty),0,1
13/Apr/2011 08:52:55,Info,Teardown,ASA-session-6-302014,TCP,
192.168.2.64,192.168.1.6,(empty),(empty),1694,135,epmap,(empty),

23. and we’re expected to talk to
it like this

24. select (select max(answer.answer) from answer where answer.member_id in (
select member_id from team_members where project_id in ( select project_id
from project where Business_stream='Upstream' and stage='Appraise' and
project_id in (select project_id from projectextra where subteam<>1 ) ) ) and
answer.page_id=page.page_id) as thinl, (select max(avgscore) from task_project
where task_project.project_id not in (select project_id from projectextra
where subteam=1 ) and task_project.project_id in (select project_id from
project where stage='Appraise' and Business_stream = 'Upstream') and
task_project.page_id=page.page_id) as bmax, (select max(answer) from answer
where answer.page_id=page.page_id) as datamax, (select avg(avgscore) from
task_project where project_id=1 and task_project.page_id=page.page_id) as
projavg, (select avg(avgscore) from task_project where project_id not in
(select project_id from projectextra where subteam=1) and
task_project.page_id=page.page_id) as companyavg, (select avg(avgscore) from
task_project where project_id not in (select project_id from projectextra
where subteam=1) and project_id in (select project_id from project where
Business_stream = 'Upstream') and task_project.page_id=page.page_id) as
businessavg, page.* from page,riverorder where page.category_name='Business
Boundaries' and stage_name='Appraise' and
riverorder.category_name=page.category_name order by
riverorder.riverorder,page.order_id select (select max(answer.answer) from
answer where answer.member_id in ( select member_id from team_members where
project_id in ( select project_id from project where

25. It could be better.
yes? better is good!

26. {[-­‐]
checkin
:
{[-­‐]
badges
:
[],
created
:
1331454784,
geolat
:
"30.2640941786",
geolong
:
"-­‐97.7414819408",
mayor
:
{[-­‐]
type
:
"nochange"
},
primarycategory
:
{[-­‐]
fullpathname
:
"Food:American
Restaurants",
iconurl
:
"https://foursquare.com/img/categories/food/default.png",
id
:
"4bf58dd8d48988d14e941735", Text
nodename
:
"American
Restaurants"
},
timezone
:
"America/Chicago",
user
:
{[-­‐]
gender
:
"male"
},
venue
:
{[-­‐]
id
:
"4d752b1bba682d43e7563876",
name
:
"CNN
Grill
@
SXSW
(Max's
Wine
Dive)"
}
}
} readable, ya think?

27. failed password | timechart count
by client_ip
The languages to talk to data are
getting better for us humans

28. Guys.. come on! Go
back to the data please.

30. Need data?
a simple way to describe a
massive problem
A friend in Boulder can help

31. The Social Media API
Jud Valeski
Co-Founder, CEO

32. Sometimes machine data is
helpful to those OTHER than IT

33. Someone with
a different
perspective
sees your
exhaust as a
source of fuel

34. please, please, please
CALL THE VP OF
ENGINEERING
at all of your vendors.

35. DEMAND REALTIME DATA
IN A STREAM OVER THE WEB
IN JSON FORMAT

36. Hey audience!
We still have a few
minutes.
What questions
might you have
been saving until
this exact moment?

37. Thanks.
@michaelwilde
Michael Wilde
Splunk Ninja
Co-CTO, Splunk
Who else sends you on your way with a cute dog photo?