6. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
WHO‘S WHO
Bernd Ahlers
@berndahlers
German, 34, Graylog2 Developer
Graylog2 Team since 2014
Developer @ TORCH GmbH
Michael Friedrich
@dnsmichi
Austrian, 31, Icinga Developer
Icinga Team since May 2009
Application Developer @ NETWAYS
8. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
TOOLS: GRAYLOG2
•Started as open source project by Lennart Koopmann in 2010
–Developed entirely in his free time
–Free & open source log management tool
•TORCH GmbH founded as company behind Graylog2 in late 2012
–after seeing massive growth and worldwide distribution in large
scale setups
•Team of 8 engineers working full-time on it
9. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
TOOLS: GRAYLOG2
•Big rewrite of Graylog2 started in 2012
•Finished with releasing a final v0.20.0 in February 2014
•Addresses what we learnt from our first customers and all users
•Unified REST API communication
–easy extending and integrating with other products, tools and
scripts
•New web interface focusing on powerful analytics
•Current stable version: 0.91.3
18. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
LOG & MONITORING
•Monitor your logs
–Call check plugin or receive passive events
–Generate alerts based on thresholds (configuration)
–Notifications based on alerts
–Visualize the current state & history for SLA reporting
–Trigger event handlers (e.g. iptables on flood)
•Popular plugins
–check_logfiles
–check_splunk
•Collector APIs & Hooks
–Graylog2 alert API & alert callback plugin
–Logstash Nagios output
20. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
STRATEGY
•Out-of-the-box support or external addons?
•Add hook to streams for passive event sending?
•Query a defined API for alerts?
•Visualize alerts, and where? (we want dashboards!)
•Re-usable & customizable URL for notifications
•Combine Log Events & Monitoring notifications and handlers
26. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
POLL: ICINGA CHECK
# ./check-graylog2-stream
usage:
-condition="<ID>": Condition ID, set only to check a single alert (optional)
-password="<password>": API password (mandatory)
-stream="<ID>": Stream ID (mandatory)
-url="http://localhost:12900": URL to Graylog2 api (optional)
-user="<username>": API username (mandatory)
28. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
COMBINING GRAYLOG2 & ICINGA 2
•Events triggered by Icinga 2
–Check results
–State changes
–Notifications
•Sent to Graylog2 using `GelfWriter` feature
# icinga2 feature enable gelf && service icinga2 restart
•Visualize in Graylog2
–Filter based on type (e.g. state != OK)
–Alert streams based on counts, etc
29. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
NOTIFICATIONS
•„Default Monitoring Alerts are awful“
http://holyhandgrenade.org/blog/2012/11/default-monitoring-alerts-are-awful/
–You want to see what‘s wrong. No additional click on your mobile.
•Icinga 2 triggers a notification
–Fetch additional information from Graylog2 API
–Include ‚notes_url‘ with stream id in notification
•Requirements
–Custom notification script
–Stream ids as custom attributes
–Icinga2 v2.2 Apply For Rules
30. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
MONITOR THE MONITORING CORE
•Check Plugin
–Query Graylog2 Alert Stream API for Icinga 2 alerts
–Use Stream ID for notifications & notes_url
•See what‘s happening in Icinga 2
–Restrict views based on user roles
–Debug plugin & check problems
–Combine cluster mal-function log
–Filter events
–Additional dashboard
35. WWW.GRAYLOG2.ORG | WWW.ICINGA.ORG
#OSMC #GRAYLOG2 #ICINGA
„THE FUTURE“
•Build your own stack
•Combine existing interfaces into one
–Graylog2 streams in Icinga Web 2 (ask Tom!)
–Icinga 2 Events in Graylog2 (more? We want more!)
•Correlate your monitoring events with events & logs of any kind
•Think about
–Simple and secure event receiver
–Auto-Discover checkable objects from log alerts
–Alert stream rules for monitoring