All Things Open 2014 - Day 1 Wednesday, October 22nd, 2014 Leslie Hawthorn Director of Developer Relations for Elasticsearch Big Data What Does Big Data Really Mean for Your Business?

1. What Does Big Data Really Mean
for your Business?
Leslie Hawthorn
Director of Developer Relations
leslie.hawthorn@elasticsearch.com
@lhawthorn

3. Even our small data has “Big Data”

4. Every day we create more…
User Generated Machine Generated
• Tweets
• Blogs
• Books
• Notes
• Emails
• Comments
• Logs
• Events
• Statistics
• Alerts
• Metrics

5. And it’s growing…
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
2010 2016
Exabytes
Year
machine social voip enterprise

6. But there is a gap
Storing
Data
Actionable
Insight

7. Data From
Any Source
Instantly
Analyze
Actionable
Insights
The promise of “Big Data”

8. Massive
effort in
cleaning
the data.
Takes
weeks to
analyze.
Insights
come too
late.
The reality
“I’m a data janitor.”
Senior Director of Data Science, major Apache Hadoop distro

9. Why?

10. Balancing Structure and Scale
SQL
Rela)onships
Structure
Can
it
Scale?
SQL
Yes
Yes
No
NoSQL
No
Yes
Yes
Hadoop
No
No
Yes
We got excited about Scale,
but now suffer for lack of Structure.
Which means: We can store a lot of data, but it takes a ton of work to
search, analyze and get insights out of it.
Hadoop Also Brings:
Intensive Engineering and a Large Upfront Investment

11. Where we’re going

12. Your Big Data pipeline
• Incredibly crowded landscape in both the
open source and proprietary worlds
• Most big data tools were built for people
with Massive Data
• Tools now focus on making working with
both structured & unstructured data easier

13. Why open source?
http://www.flickr.com/photos/16462767@N00/3984507322/
flexibility

14. Data From
Any Source
Instantly
Store, Search
& Analyze
Actionable
Insights
Elasticsearch:
Search, Analytics, Logging & Visualization
Logstash Elasticsearch Kibana
The ELK Stack

15. Data Democratization

16. Big Data for our business
• Measuring our community health metrics
Demo time….
[see appendix for screenshots from demo]

17. Be a data superhero, not a janitor!

18. Questions?
Leslie Hawthorn
Director of Developer Relations
leslie.hawthorn@elasticsearch.com
@lhawthorn

19. Appendix – Demo Slides