This document introduces the TICK stack, which is a collection of open source software tools for collecting, processing, storing, and visualizing metrics and events. It summarizes the main components: Telegraf collects metrics from servers and services and writes them to InfluxDB; InfluxDB is a time series database that stores metrics; Chronograf provides visualization of metrics stored in InfluxDB; and Kapacitor processes data from InfluxDB to perform tasks like anomaly detection and alerting. Examples are provided of how these tools can be used together in a workflow to monitor systems and applications.
10. WHAT ARE TIME SERIES?
TIME SERIES EXPLAINED
▸ A time series is a series of data
points indexed (or listed or
graphed) in time order
▸ Time series are used in statistics,
signal processing, pattern
recognition, weather forecasting
and largely in any domain of
applied science and engineering
▸ Time series are used for analysis
and forecasting
11. SOME QUESTIONS
TIME SERIES IN PRACTICE
▸ What offer is best to present to a visitor based on their behaviour, in real-time?
▸ What patterns can I detect in financial markets that I can use to execute faster, more
intelligent transactions?
▸ Can I predict how long visitors will stay and why they’ll drop off?
▸ Can I track sensors on my fleet vehicles over time to optimise delivery schedules and
fuel economy?
▸ Can I predict if my elastic infrastructure will scale on events like Black Friday?
▸ Can I weave through petabytes of machine to machine data over time to detect
malicious patterns in my network?
▸ Can I increase the yield of my crops and lower my costs by adjusting water and
fertiliser in real-time, based on environmental conditions?
14. USE CASES
IN THE REAL LIFE
▸ eBay’s Experimentation environment enables users to
answer important analytics and business questions.
▸ Mozilla uses InfluxDB to store important metrics about the
performance of applications and devices running Firefox
OS
▸ Facile.it collects a huge amount of data from their
customers that trigger a number of actions that are an
order of magnitude bigger than the data itself
16. WHY INFLUXDATA?
INFLUXDATA FEATURES
▸ Ease of Use – up and running in minutes, not days or weeks
▸ Scalable – write thousand of points per second, store billions of points for analysis
▸ Open Source – MIT licensed, extensible by design
▸ Integrated – Data collection, storage, visualisation and alerting designed to work together seamlessly
▸ Highly Available – Platform components can be distributed and clustered
▸ Real-Time Downsampling – Continuous queries precompute large amounts of data on-the-fly before
being written
▸ Efficient Storage – High compression and retention polices lower storage footprints and costs
▸ Purpose Built – InfluxData is designed from the ground up to do one thing, manage time-series data
at scale
▸ Choice of Deployment – Run InfluxData in your datacenter, a public cloud or on our managed hosting
service
18. TICK: TELEGRAF
TELEGRAF
▸ Telegraf is an agent written in Go
for collecting metrics and writing
them into InfluxDB or other
possible outputs.
▸ It’s a standalone module
▸ It can be customised with a
configuration file
23. TICK: INFLUXDB
INFLUXDB
▸ InfluxDB is a time series database
built from the ground up to handle
high write and query loads.
▸ InfluxDB is meant to be used as a
backing store for any use case
involving large amounts of
timestamped data, including
DevOps monitoring, application
metrics, IoT sensor data, and real-
time analytics.
24. TICK: INFLUXDB
FEATURES
▸ Specific query language (Influx Query Language)
▸ NO-SQL and schemaless database
▸ Retention policies for single measurements
▸ Replication policies for each database
▸ HTTP REST API
25. TICK: INFLUXDB
QUERY LANGUAGE
▸ InfluxDB has its own DSL (Domain Specific Language) to
query and explore data
▸ It has a set of functions to use in conjunction with the
common SELECT statement
▸ Mathematical operations can be done right into the query
▸ Continuous Queries are InfluxQL queries that run
automatically and periodically on realtime data and store
query results in a specified measurement.
27. TICK: CHRONOGRAF
CHRONOGRAF
▸ Chronograf is a graphing and
visualization application that you
use to perform ad hoc exploration
of your InfluxDB data.
▸ It’s a standalone module used in
conjunction with InfluxDB
28. TICK: CHRONOGRAF
KEY FEATURES
▸ Simple installation
▸ Smart query builder designed to work with large datasets
▸ Collect multiple graphs into dashboards
▸ Support for templates
29.
30. TICK: KAPACITOR
KAPACITOR
▸ Kapacitor is an open source data
processing framework that makes
it easy to create alerts, run ETL jobs
and detect anomalies.
▸ It’s a standalone module
▸ It can be configured with a Domain
Specific Language (DSL)
31. TICK: KAPACITOR
KEY FEATURES
▸ Process both streaming data and batch data.
▸ Query data from InfluxDB on a schedule, and receive data via
the line protocol and any other method InfluxDB supports.
▸ Perform any transformation currently possible in InfluxQL.
▸ Store transformed data back in InfluxDB.
▸ Add custom user defined functions to detect anomalies.
▸ Integrate with HipChat, OpsGenie, Alerta, Sensu, PagerDuty,
Slack, and more.
32. TICK: KAPACITOR
TICKSCRIPT
▸ The TICKscript language is an invocation chaining
language. Each script has a flat scope and each variable in
the scope defines methods that can be called on it.
▸ Kapacitor uses TICKscripts to define data processing
pipelines. A pipeline is set of nodes that process data and
edges that connect the nodes. Pipelines in Kapacitor are
directed acyclic graphs (DAGs) meaning each edge has a
direction that data flows and there cannot be any cycles in
the pipeline.