In this webinar we covered how to improve search with analytics using the Elastic Stack: ElasticSearch, Logstash, Kibana. Check out our upcoming events: www.mcplusa.com/events

1. MC+A - Confidential and Proprietary
“Simplicity is the ultimate sophistication.”
- Leonardo da Vinci
MC+A - Confidential and Proprietary

2. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
Search Analytics with ELK (Elastic Stack)
Finding value in your wealth of data

3. MC+A - Confidential and Proprietary
Ground rules
+ To ensure audio quality, all attendees are muted.
+ Feel free to put any questions into the GotoWebinar chat area.
+ Having issues? Email marketing@mcplusa.com or Follow
@mcplusa and we’ll respond via Direct Message or by email.

4. MC+A - Confidential and Proprietary
Agenda
1. An Approach to Search Analytics
2. Overview of ELK
+ ElasticSearch
+ Logstash
+ Kibana
3. Reference Architecture
4. Tooling for ELK development
5. Demo

5. MC+A - Confidential and Proprietary
Today’s Speakers
Michael Cizmar
Managing Director
MC+A
@michaelcizmar
John Cizmar
Director
MC+A
@johncizmar
Nicole Millalaf
Full Stack Developer
MC+A
@niccoleme

6. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
Search Analytics with ELK (Elastic Stack)
Finding value in your wealth of data

7. MC+A - Confidential and Proprietary
An Approach to Search Analytics

8. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
Framework for Improving Search through Analytics
Tracking and Improving Relevancy

9. MC+A - Confidential and Proprietary
+ Analytics drives insight and tuning
+ Not all content is equal
+ User feedback is not as important as you think (hint: User Action is)
+ Ongoing Governance is essential
Improving Search Through Analytics
Foundational Principles

10. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
What Determines Relevant Content
Objective and Opinionated

11. MC+A - Confidential and Proprietary
+ Search scoring by SME
+ Use Cases
+ Requirements
+ What is your Baseline
What is Relevant Content
Framing the Question

12. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
Data Driven Decisions
Define. Model. Iterate.

13. MC+A - Confidential and Proprietary
+ The need for a Clipboard
+ Experimentation
+ Testable
+ Governance
Data Driven Decisions
Removing Subjectivity

14. MC+A - Confidential and Proprietary
Not all Data Sources are Right for You
Logging vs. Click logging (GA)
+ Application Logs
+ Uses of a log: troubleshoot issues, security, monitoring
+ Know Gaps

15. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
Overview of ELK (The Elastic Stack)
Make Sense of Data

16. MC+A - Confidential and Proprietary
+ Log and event data collection
+ Provides Visualization Platform
+ Allows for exploration
What is ELK (The Elastic Stack)
What does it do?

17. MC+A - Confidential and Proprietary
The Elastic Stack
High Level Architecture
+ General architecture
+ Scaled architecture Elastic
Kibana
Logstash
User Interface
Index & Analyze
Ingest

18. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
ElasticSearch
Scale beyond limits.
MC+A - Confidential and Proprietary

19. MC+A - Confidential and Proprietary
The Elastic Stack
Elastic Search
+ Java
+ Search and index
+ Distributed
+ Clustering
+ API - JSON/RESTful
+ Apache Lucene

20. MC+A - Confidential and Proprietary

21. MC+A - Confidential and Proprietary
+ Scaling / Data Transport
+ Parsing of log files
+ Creating queries for dashboards
Architecture
Keypoints - Tasks

22. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
Logstash
Plumbing for your logs.
MC+A - Confidential and Proprietary

23. MC+A - Confidential and Proprietary
The Elastic Stack
Logstash
+ Multiple input / Multiple output
+ Centralize logs
+ Collect
+ Parse
+ Store / Forward

24. MC+A - Confidential and Proprietary
Logstash

25. MC+A - Confidential and Proprietary
Logstash
What does a log look like
58.160.163.109!205.175.241.30!58.160.163.109 - - [11/Jun/2016:00:04:04 -0600] "GET
/search?client=literature&filter=0&ie=UTF-8&oe=UTF-8&output=xml_no_dtd&proxystylesheet=literatu
re&site=literature&getfields=*&lang=en&hl=en&inbound=Products&requiredfields=-xproductcategory:
Not+Specified&sort=date:D:R:d1&ulang=en&access=p&entqr=3&entqrm=2&wc=200&wc_mc=1&ud=1&
&q=suva+inmeta:ddescription%3DCertification&dnavs=suva+inmeta:ddescription%3DCertification&ip=
58.160.163.109 HTTP/1.1" 200 26168 22 25.00

26. MC+A - Confidential and Proprietary
Logstash
What does a log look like
58.160.163.109!205.175.241.30!58.160.163.109 - - [11/Jun/2016:00:04:04 -0600] "GET
/search?client=literature&filter=0&ie=UTF-8&oe=UTF-8&output=xml_no_dtd&proxystylesheet=literatu
re&site=literature&getfields=*&lang=en&hl=en&inbound=Products&requiredfields=-xproductcategory:
Not+Specified&sort=date:D:R:d1&ulang=en&access=p&entqr=3&entqrm=2&wc=200&wc_mc=1&ud=1&
&q=suva+inmeta:ddescription%3DCertification&dnavs=suva+inmeta:ddescription%3DCertification&ip=
58.160.163.109 HTTP/1.1" 200 26168 22 25.00

27. MC+A - Confidential and Proprietary
Logstash
#--inmeta: split
if event['params']['q'].to_s.include? 'inmeta:'
event['params']['q'].gsub! '%2520',' '
event['params']['q'].gsub! '%3D','='
event['params']['q'].gsub! '%2528','('
event['params']['q'].gsub! '%252D','-'
event['params']['q'].gsub! '%2529',')'
event['params']['q'].gsub! '%252C',','
event['params']['q_inmeta'] = event['params']['q'].to_s[7+event['params']['q'].to_s.index('inmeta:')..-1].split('inmeta:')
end

28. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
Kibana
Real-time dashboards.
MC+A - Confidential and Proprietary

29. MC+A - Confidential and Proprietary
The Elastic Stack
Kibana
+ Real-time charting
+ Flexible visualizations

30. MC+A - Confidential and Proprietary
Kibana

31. MC+A - Confidential and Proprietary
Tooling
Getting Assistance.
+ Grok Debugger (https://grokdebug.herokuapp.com/)
+ Logstash Cookbook
+ Logstash Book

32. MC+A - Confidential and ProprietaryMC+A - Confidential and Proprietary
The Initial Questions / Demo
MC+A - Confidential and Proprietary

33. MC+A - Confidential and Proprietary
Key questions
Initial questions to seed discovery
+ What are most common queries?
+ What queries produce no results?
+ What interaction in the search interface are being used?
○ Filters
○ Sorts
+ Result click analysis
○ What result number did the user click on
○ Results with clicks /without

34. MC+A - Confidential and Proprietary
Questions?
MC+A - Confidential and Proprietary

35. MC+A - Confidential and Proprietary
Connecting With Us
www.mcplusa.com
plus.google.com/+mcplusa @mcplusa fb.com/mcplusa Channel MC+A
linkedin.com/company/mc-amarketing@mcplusa.com
MC+A - Confidential and Proprietary