The good, the bad, and the ugly of creating a time series database on the Apache Cassandra/Scylla platform. This session covers the challenges and successes faced while creating the time series database KairosDB. The information covered will help anyone interested in storing their own time series data either in a home grown application or in KairosDB.

1. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Learn How to Build
a Time Series Database
Staff Engineer, Proofpoint
Brian Hawkins

2. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Brian Hawkins
2
Brian is a Staff Engineer at Proofpoint. He created
KairosDB, a time series solution that runs on top of
Cassandra or ScyllaDB.

3. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
KairosDB
▪ Fast open source time series database
▪ Cassandra is primary backend (GridDB and H2)
▪ Extremely pluggable
▪ Scales really well (10,000,000,000/day)
▪ Saves you $$$$
3

4. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Build Your Own TS Application
▪ Why Build Your Own?
o Specialized Query Patterns
o Indexing Requirements
▪ Use KairosDB
o Plugable Storage Model for Custom Data Point Types
o Easy to Extend
o Easy to Embed
4

5. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Storage
HashMap<partition_key, TreeMap<cluster_key, value>>
5

6. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Storage
HashMap<partition_key, TreeMap<cluster_key, value>>
6

7. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Storage
HashMap<partition_key, TreeMap<cluster_key, value>>
7

8. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Schema
public static final String DATA_POINTS = "" +
"CREATE TABLE IF NOT EXISTS data_points (" +
" metric text, " +
" row_time timestamp, " +
" data_type text, " +
" tags frozen<map<text, text>>, " +
" offset int, "+
" value blob, " +
" PRIMARY KEY ((metric, row_time, data_type, tags), offset)" +
") WITH COMPACT STORAGE";
8

9. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
General Tips for Ingest
▪ Batch (10k is sweet spot)
o Need to adjust batch_size_fail_threshold on C* server
▪ Add LZ4 Compression
▪ Set native_transport_max_threads: 2000
▪ Don’t cannon ball your batch
9

10. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
Is Cassandra/Scylla the Right Platform for TS?
▪ The Good
o Incredibly fast ingest
o Excellent scaling
o Moderately compact storage
o Write once data
▪ The Bad
o No multi dimensional lookup
o No in place processing of data
10

11. PRESENTATION TITLE ON ONE LINE
AND ON TWO LINES
First and last name
Position, company
THANK YOU
brianhks1@gmail.com
Please stay in touch
Any questions?