SlideShare une entreprise Scribd logo
1  sur  82
Télécharger pour lire hors ligne
CQL Under the Hood 
Robbie Strickland
whoami? 
Robbie Strickland 
Software Development Manager 
@rs_atl
Target Audience 
• Veterans who don’t use or don’t understand CQL 
• Newcomers who only know CQL, but don’t know what’s happening 
underneath
Definitions 
• Thrift: legacy RPC protocol + code generation tool 
• batch_mutate, get_range_slices, multiget_slice, etc. 
• Deprecated in favor of native protocol 
• Native protocol: replacement for Thrift 
• Only works with CQL
Definitions 
• Storage rows: keys + columns as stored on disk 
• CQL rows: abstraction layer on top of storage rows 
• Not usually a direct mapping to storage rows 
• Make use of predefined schema 
• Data still sparse – no space used for null columns
In the Old Days … 
• Lots of clients with lots of APIs 
• No common language for describing schemas or queries 
• Steep learning curve
In the Old Days … 
• No cursors, so entire result set must fit in memory on client and server 
• Hard to add new features 
• Lag time between release of new features and client library adoption
Solution: CQL? 
SELECT * FROM mytable WHERE mykey = ‘foo’; 
WAT???!!!
Solution: CQL? 
Veterans 
“What happened to 
my NoSQL??!!”
Solution: CQL? 
Newbies 
“Sweet, SQL! I don’t 
have to learn 
anything new!”
Reality 
• Don’t panic 
• Thrift problems solved 
• You didn’t lose anything 
• Underlying storage is 
unchanged 
• Don’t get lazy 
• CQL is not SQL 
• You need to know what 
you’re doing 
• No, you can’t just index 
everything
Simple CQL 
CREATE TABLE Books (! 
title varchar,! 
author varchar,! 
year int,! 
PRIMARY KEY (title)! 
SELECT * FROM Books;! 
);! 
! 
INSERT INTO Books (title, author, year) ! 
VALUES ('Patriot Games', 'Tom Clancy', 1987);! 
INSERT INTO Books (title, author, year) ! 
VALUES ('Without Remorse', 'Tom Clancy', 1993);! 
! 
title | author | year! 
-----------------+------------+------! 
Without Remorse | Tom Clancy | 1993! 
Patriot Games | Tom Clancy | 1987!
Storage Rows 
[default@unknown] create keyspace Library;! 
[default@unknown] use Library;! 
[default@Library] create column family Books! 
...! with key_validation_class=UTF8Type! 
...! and comparator=UTF8Type! 
...! and default_validation_class=UTF8Type;! 
[default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! 
[default@Library] set Books['Patriot Games']['year'] = '1987';! 
[default@Library] list Books;! 
! 
RowKey: Patriot Games! 
=> (name=author, value=Tom Clancy, timestamp=1393102991499000)! 
=> (name=year, value=1987, timestamp=1393103015955000)!
Storage Rows 
[default@unknown] create keyspace Library;! 
[default@unknown] use Library;! 
[default@Library] create column family Books! 
...! with key_validation_class=UTF8Type! 
...! and comparator=UTF8Type! 
...! and default_validation_class=UTF8Type;! 
[default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! 
[default@Library] set Books['Patriot Games']['year'] = '1987';! 
[default@Library] list Books;! 
! 
RowKey: Patriot Games! 
=> (name=author, value=Tom Clancy, timestamp=1393102991499000)! 
=> (name=year, value=1987, timestamp=1393103015955000)! 
Random hash (no ordering)
Storage Rows 
[default@unknown] create keyspace Library;! 
[default@unknown] use Library;! 
[default@Library] create column family Books! 
...! with key_validation_class=UTF8Type! 
...! and comparator=UTF8Type! 
...! and default_validation_class=UTF8Type;! 
[default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! 
[default@Library] set Books['Patriot Games']['year'] = '1987';! 
[default@Library] list Books;! 
! 
Ordered by name 
RowKey: Patriot Games! 
=> (name=author, value=Tom Clancy, timestamp=1393102991499000)! 
=> (name=year, value=1987, timestamp=1393103015955000)!
Compound Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Compound Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
Partition key (row key) 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Compound Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
Clustering columns 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
== Composite Columns 
[default@Library] list authors;! 
! 
Partition key (row key) 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)!
== Composite Columns 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
Clustering columns
== Composite Columns 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
Ordered by name
Reversing Sort Order 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
) WITH CLUSTERING ORDER BY (year DESC);! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
Reversing Sort Order 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Composite Partition Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY ((name, year), title)! 
);! 
Composite partition key 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Composite Partition Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY ((name, year), title)! 
);! 
Clustering column 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
== Composite Keys 
[default@Library] list authors;! 
! 
Partition keys (row key) 
RowKey: Tom Clancy:1993! 
=> (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! 
=> (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! 
-------------------! 
RowKey: Tom Clancy:1987! 
=> (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! 
=> (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
== Composite Keys 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy:1993! 
=> (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! 
=> (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! 
-------------------! 
RowKey: Tom Clancy:1987! 
=> (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! 
=> (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)! 
Clustering column
Why This Matters 
• Queries must respect underlying storage, else they will either be slow or 
impossible 
• You have to know your partition key at query time 
• If you want fast multi-record queries, select a range, in storage order 
• With clustering columns, order matters
Example 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
) WITH CLUSTERING ORDER BY (year DESC);!
Example 
name | year | title | isbn | publisher! 
------------+------+-----------------------------+---------------+-----------! 
Tom Clancy | 1996 | Executive Orders | 0-399-13825-0 | Putnam! 
Tom Clancy | 1994 | Debt of Honor | 0-399-13826-1 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13927-0 | Putnam! 
Tom Clancy | 1991 | The Sum of All Fears | 0-399-12341-6 | Putnam! 
Tom Clancy | 1989 | Clear and Present Danger | 0-399-13341-1 | Putnam! 
Tom Clancy | 1988 | The Cardinal of the Kremlin | 0-399-13241-4 | Putnam! 
Tom Clancy | 1987 | Patriot Games | 0-399-13231-4 | Putnam! 
Tom Clancy | 1986 | Red Storm Rising | 0-399-13230-2 | Putnam! 
Tom Clancy | 1984 | The Hunt for Red October | 0-399-13251-1 | Putnam! 
!!
Query by Key 
SELECT * FROM authors 
WHERE name = ‘Tom Clancy’ 
(CL = QUORUM) 
RF = 3 
Tom 
Clancy 
Tom 
Clancy 
Tom 
Clancy
Query by Key 
Find partition key 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Query by Key 
Scan all columns in order 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Range Query 
SELECT * FROM authors 
WHERE name = ‘Tom Clancy’ 
AND year >= 1990 
(CL = QUORUM) 
Tom 
Clancy 
Tom 
Clancy 
Tom 
Clancy
Range Query 
Find partition key 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Range Query 
Scan until < 1990 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Querying Tail of Range 
SELECT * FROM authors 
WHERE name = ‘Tom Clancy’ 
AND year <= 1990 
(CL = QUORUM) 
Tom 
Clancy 
Tom 
Clancy 
Tom 
Clancy
Querying Tail of Range 
Find partition key 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Querying Tail of Range 
Scan all, 
then filter <= 1990 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Multiple Keys 
DK 
DK 
MG 
MG 
TC 
MG 
TC 
TC 
DK 
SELECT * FROM authors 
WHERE name IN 
(‘Tom Clancy’, 
‘Dean Koontz’, 
‘Malcolm Gladwell’) 
(CL = QUORUM)
Lessons Learned 
• Sequential == fast 
• Query by key / clustering column (range) == Sequential 
• Multi-key query often == lots of nodes 
• Write in the intended read order
Collections 
• Sets: unordered, unique 
• Lists: ordered, allow duplicates 
• Maps: key/value pairs – can be a good substitute for dynamic columns 
• Max 64k items, 64k per item 
• Always returns entire collection
Sets 
CREATE TABLE authors (! 
!name text,! 
!books set<text>,! 
PRIMARY KEY (name)! 
name | books! 
------------+--------------------------------------! 
Tom Clancy | {'Patriot Games', 'Without Remorse'}! 
);! 
! 
INSERT INTO authors (name, books) ! 
VALUES ('Tom Clancy', {'Without Remorse', 'Patriot Games'});!
Sets 
RowKey: Tom Clancy! 
=> (name=books:50617472696f742047616d6573, value=, ...)! 
=> (name=books:576974686f75742052656d6f727365, value=, ...)! 
Set name 
Set item 
Empty value
Lists 
CREATE TABLE authors (! 
!name text,! 
!books list<text>,! 
PRIMARY KEY (name)! 
name | books! 
------------+--------------------------------------! 
Tom Clancy | ['Without Remorse’, 'Patriot Games']! 
);! 
! 
INSERT INTO authors (name, books) ! 
VALUES ('Tom Clancy', ['Without Remorse', 'Patriot Games']);!
Lists 
RowKey: Tom Clancy! 
=> (name=books:d36de8b0305011e4a0dddbbeade718be, value=576974686f75742052656d6f727365, ...)! 
=> (name=books:d36de8b1305011e4a0dddbbeade718be, value=50617472696f742047616d6573, ...)! 
List name 
Ordering ID 
List item
Maps 
CREATE TABLE authors (! 
!name text,! 
!books map<text, int>,! 
PRIMARY KEY (name)! 
name | books! 
------------+-------------------------------------------------! 
Tom Clancy | {'Patriot Games': 1987, 'Without Remorse': 1993}! 
);! 
! 
INSERT INTO authors (name, books) ! 
VALUES ('Tom Clancy', ! 
{'Without Remorse' : 1993, 'Patriot Games' : 1987});!
Maps 
RowKey: Tom Clancy! 
=> (name=books:50617472696f742047616d6573, value=000007c3, ...)! 
=> (name=books:576974686f75742052656d6f727365, value=000007c9, ...)! 
Map name 
Map key 
Map value
Indices 
CREATE INDEX author_publisher! 
ON author (publisher);!
Indices 
• Allow query by value in certain cases 
• Partitioned based on row key of indexed table 
• Are updated atomically along with the data being inserted 
• Must be low cardinality, or it won’t scale well (to large cluster sizes) 
• But not too low, or it’s sort of pointless
Index Distribution 
Node 1 
Authors 
“Tom Clancy” : “Putnam”! 
“Mark Twain” : “Putnam”! 
! 
Index 
“Putnam” : “Tom Clancy”! 
“Putnam” : “Mark Twain”! 
! 
Node 2 
Authors 
“Mark Twain” : “Putnam”! 
“Dan Brown” : “Putnam”! 
! 
Index 
“Putnam” : “Mark Twain”! 
“Putnam” : “Dan Brown”! 
! 
Node 3 
Authors 
“Dan Brown” : “Putnam” 
“Tom Clancy” : “Putnam”! 
! 
Index 
“Putnam” : “Dan Brown”! 
“Putnam” : “Tom Clancy”! 
! 
Index key == indexed column value
Index Distribution 
Node 1 
Authors 
“Tom Clancy” : “Putnam”! 
“Mark Twain” : “Putnam”! 
! 
Index 
“Putnam” : “Tom Clancy”! 
“Putnam” : “Mark Twain”! 
! 
Node 2 
Authors 
“Mark Twain” : “Putnam”! 
“Dan Brown” : “Putnam”! 
! 
Index 
“Putnam” : “Mark Twain”! 
“Putnam” : “Dan Brown”! 
! 
Node 3 
Authors 
“Dan Brown” : “Putnam” 
“Tom Clancy” : “Putnam”! 
! 
Index 
“Putnam” : “Dan Brown”! 
“Putnam” : “Tom Clancy”! 
! 
… but node distribution based on original table key
Querying by Value 
pub idx 
pub idx 
pub idx pub idx 
pub idx 
pub idx 
SELECT * FROM authors 
WHERE publisher = ‘Putnam’ 
(CL = QUORUM)
Deletes 
DELETE FROM authors WHERE name = 'Tom Clancy';! 
! 
INSERT INTO authors (title, name) 
VALUES ('Patriot Games', 'Tom Clancy') 
USING TTL 86400;! 
! 
INSERT INTO authors (title, name, year) 
VALUES ('Patriot Games', 'Tom Clancy', null);! 
! 
UPDATE authors SET publisher = null 
WHERE name = 'Tom Clancy';! 
!
Deletes 
• Log-structured storage, so writes are immutable 
• Deletes create tombstones, one for each deleted column 
• Cassandra must read the tombstones to make sure it doesn’t revive 
deleted data 
• Lots of deletes is an anti-pattern
Missing Columns 
INSERT INTO authors (title, name, year)! 
VALUES ('Without Remorse', 'Tom Clancy', 1993);! 
! 
name | year | title | isbn | publisher! 
------------+------+-----------------+------+-----------! 
Tom Clancy | 1993 | Without Remorse | null | null! 
! 
RowKey: Tom Clancy! 
=> (name=1993:Without Remorse:, value=, timestamp=1409936754170000)!
Missing Columns 
activity | timestamp | source | source_elapsed! 
---------------------------------------------------------------------------+--------------+-----------+----------------! 
execute_cql3_query | 11:51:55,975 | 127.0.0.1 | 0! 
Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:51:55,975 | 127.0.0.1 | 47! 
Preparing statement | 11:51:55,975 | 127.0.0.1 | 105! 
Executing single-partition query on authors | 11:51:55,975 | 127.0.0.1 | 307! 
Acquiring sstable references | 11:51:55,975 | 127.0.0.1 | 315! 
Merging memtable tombstones | 11:51:55,975 | 127.0.0.1 | 328! 
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:51:55,975 | 127.0.0.1 | 374! 
Merging data from memtables and 0 sstables | 11:51:55,975 | 127.0.0.1 | 383! 
Read 1 live and 0 tombstoned cells | 11:51:55,975 | 127.0.0.1 | 420! 
Request complete | 11:51:55,975 | 127.0.0.1 | 585! 
Only 1 read required
Null Columns 
INSERT INTO authors (title, name, year, isbn, publisher) 
VALUES ('Without Remorse', 'Tom Clancy', 1993, null, null);! 
! 
name | year | title | isbn | publisher! 
------------+------+-----------------+------+-----------! 
Tom Clancy | 1993 | Without Remorse | null | null!
Null Columns 
activity | timestamp | source | source_elapsed! 
---------------------------------------------------------------------------+--------------+-----------+----------------! 
execute_cql3_query | 11:57:31,623 | 127.0.0.1 | 0! 
Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:57:31,623 | 127.0.0.1 | 41! 
Preparing statement | 11:57:31,623 | 127.0.0.1 | 101! 
Executing single-partition query on authors | 11:57:31,623 | 127.0.0.1 | 532! 
Acquiring sstable references | 11:57:31,623 | 127.0.0.1 | 544! 
Merging memtable tombstones | 11:57:31,623 | 127.0.0.1 | 571! 
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:57:31,623 | 127.0.0.1 | 596! 
Merging data from memtables and 0 sstables | 11:57:31,623 | 127.0.0.1 | 605! 
Read 1 live and 2 tombstoned cells | 11:57:31,624 | 127.0.0.1 | 669! 
Request complete | 11:57:31,623 | 127.0.0.1 | 777! 
3 reads required!
Why Queries Fail 
SELECT name FROM authors WHERE title = 'Patriot Games';! 
Bad Request: PRIMARY KEY part title cannot be 
restricted (preceding part year is either not 
restricted or by a non-EQ relation)!
Why Queries Fail 
• Failure to provide the full partition key 
• Querying by value without an index 
• Misunderstanding clustering columns
Missing Key Parts 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY ((name, year), title)! 
);! 
! 
! 
SELECT name FROM authors WHERE title
Missing Key Parts 
[default@Library] list authors;! 
! 
Partition keys (row key) 
RowKey: Tom Clancy:1993! 
=> (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! 
=> (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! 
-------------------! 
RowKey: Tom Clancy:1987! 
=> (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! 
=> (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
Missing Key Parts 
SELECT * FROM authors WHERE name = 'Tom Clancy’;! 
Bad Request: Partition key part year must be restricted 
since preceding part is!
Missing Key Parts 
SELECT * FROM authors WHERE year = 1987;! 
Bad Request: partition key part year cannot be 
restricted (preceding part name is either not 
restricted or by a non-EQ relation)!
Missing Key Parts 
SELECT * FROM authors WHERE year >= 1987;! 
Bad Request: partition key part year cannot be 
restricted (preceding part name is either not 
restricted or by a non-EQ relation)!
Missing Key Parts 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy' and year >= 1987;! 
! 
Bad Request: Only EQ and IN relation are supported on 
the partition key (unless you use the token() function)!
Querying by Value 
SELECT * FROM authors WHERE isbn = '0-399-13241-4';! 
Bad Request: No indexed columns present in by-columns 
clause with Equal operator!
Querying Clustering Columns 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
! 
! 
SELECT name FROM authors WHERE
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
Clustering columns
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy’! 
AND year = 1993;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy’! 
AND year <= 1993;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy’! 
AND title = 'Patriot Games';! 
! 
Bad Request: PRIMARY KEY part title cannot be restricted 
(preceding part year is either not restricted or by a 
non-EQ relation)! 
!!
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
?
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE year = 1987;! 
! 
Bad Request: Cannot execute this query as it might 
involve data filtering and thus may have unpredictable 
performance. If you want to execute this query despite 
the performance unpredictability, use ALLOW FILTERING! 
!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE year = 1987 ALLOW FILTERING;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE year = 1987 LIMIT 1 ALLOW FILTERING;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
Summary 
• Stop using Thrift; use CQL instead 
• … but know what your model/query is doing
Thanks! 
Robbie Strickland 
Software Development Manager 
@rs_atl
CQL Under the Hood

Contenu connexe

Tendances

AWS (Amazon Redshift) presentation
AWS (Amazon Redshift) presentationAWS (Amazon Redshift) presentation
AWS (Amazon Redshift) presentation
Volodymyr Rovetskiy
 

Tendances (20)

MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective IndexingMongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
 
RocksDB Performance and Reliability Practices
RocksDB Performance and Reliability PracticesRocksDB Performance and Reliability Practices
RocksDB Performance and Reliability Practices
 
Guaranteeing Memory Safety in Rust
Guaranteeing Memory Safety in RustGuaranteeing Memory Safety in Rust
Guaranteeing Memory Safety in Rust
 
Introduction to Cassandra Architecture
Introduction to Cassandra ArchitectureIntroduction to Cassandra Architecture
Introduction to Cassandra Architecture
 
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
 
Webinar: Secrets of ClickHouse Query Performance, by Robert Hodges
Webinar: Secrets of ClickHouse Query Performance, by Robert HodgesWebinar: Secrets of ClickHouse Query Performance, by Robert Hodges
Webinar: Secrets of ClickHouse Query Performance, by Robert Hodges
 
Linux Systems Performance 2016
Linux Systems Performance 2016Linux Systems Performance 2016
Linux Systems Performance 2016
 
Build a Complex, Realtime Data Management App with Postgres 14!
Build a Complex, Realtime Data Management App with Postgres 14!Build a Complex, Realtime Data Management App with Postgres 14!
Build a Complex, Realtime Data Management App with Postgres 14!
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage Engine
 
FS2 mongo reactivestreams
FS2 mongo reactivestreamsFS2 mongo reactivestreams
FS2 mongo reactivestreams
 
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdfDeep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
 
Automating the Cloud with Terraform, and Ansible
Automating the Cloud with Terraform, and AnsibleAutomating the Cloud with Terraform, and Ansible
Automating the Cloud with Terraform, and Ansible
 
Spark streaming
Spark streamingSpark streaming
Spark streaming
 
Fun with click house window functions webinar slides 2021-08-19
Fun with click house window functions webinar slides  2021-08-19Fun with click house window functions webinar slides  2021-08-19
Fun with click house window functions webinar slides 2021-08-19
 
Sql
SqlSql
Sql
 
20180726 AWS KRUG - RDS Aurora에 40억건 데이터 입력하기
20180726 AWS KRUG - RDS Aurora에 40억건 데이터 입력하기20180726 AWS KRUG - RDS Aurora에 40억건 데이터 입력하기
20180726 AWS KRUG - RDS Aurora에 40억건 데이터 입력하기
 
Modeling Data and Queries for Wide Column NoSQL
Modeling Data and Queries for Wide Column NoSQLModeling Data and Queries for Wide Column NoSQL
Modeling Data and Queries for Wide Column NoSQL
 
AWS (Amazon Redshift) presentation
AWS (Amazon Redshift) presentationAWS (Amazon Redshift) presentation
AWS (Amazon Redshift) presentation
 
Parquet Hadoop Summit 2013
Parquet Hadoop Summit 2013Parquet Hadoop Summit 2013
Parquet Hadoop Summit 2013
 
Network policy @ k8s day
Network policy @ k8s dayNetwork policy @ k8s day
Network policy @ k8s day
 

En vedette

An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
DataStax
 
Asterix and the cauldron
Asterix and the cauldronAsterix and the cauldron
Asterix and the cauldron
LUIS NARBONA
 
Understanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data StructureUnderstanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data Structure
DataStax
 

En vedette (16)

Building a distributed Key-Value store with Cassandra
Building a distributed Key-Value store with CassandraBuilding a distributed Key-Value store with Cassandra
Building a distributed Key-Value store with Cassandra
 
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
 
CQL3 in depth
CQL3 in depthCQL3 in depth
CQL3 in depth
 
Batman and robin 036
Batman and robin 036Batman and robin 036
Batman and robin 036
 
Thor 002
Thor 002Thor 002
Thor 002
 
Dc comics essentials batman - year one
Dc comics essentials   batman - year oneDc comics essentials   batman - year one
Dc comics essentials batman - year one
 
Asterix and the cauldron
Asterix and the cauldronAsterix and the cauldron
Asterix and the cauldron
 
DataStax: A deep look at the CQL WHERE clause
DataStax: A deep look at the CQL WHERE clauseDataStax: A deep look at the CQL WHERE clause
DataStax: A deep look at the CQL WHERE clause
 
Batman
BatmanBatman
Batman
 
Understanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data StructureUnderstanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data Structure
 
Indexing in Cassandra
Indexing in CassandraIndexing in Cassandra
Indexing in Cassandra
 
Marvel Comics
Marvel ComicsMarvel Comics
Marvel Comics
 
Cassandra By Example: Data Modelling with CQL3
Cassandra By Example: Data Modelling with CQL3Cassandra By Example: Data Modelling with CQL3
Cassandra By Example: Data Modelling with CQL3
 
Advanced data modeling with apache cassandra
Advanced data modeling with apache cassandraAdvanced data modeling with apache cassandra
Advanced data modeling with apache cassandra
 
Linux Profiling at Netflix
Linux Profiling at NetflixLinux Profiling at Netflix
Linux Profiling at Netflix
 
The Walking Dead volume 5
The Walking Dead volume 5The Walking Dead volume 5
The Walking Dead volume 5
 

Plus de Robbie Strickland

Plus de Robbie Strickland (6)

Always On: Building Highly Available Applications on Cassandra
Always On: Building Highly Available Applications on CassandraAlways On: Building Highly Available Applications on Cassandra
Always On: Building Highly Available Applications on Cassandra
 
Lambda at Weather Scale - Cassandra Summit 2015
Lambda at Weather Scale - Cassandra Summit 2015Lambda at Weather Scale - Cassandra Summit 2015
Lambda at Weather Scale - Cassandra Summit 2015
 
New Analytics Toolbox DevNexus 2015
New Analytics Toolbox DevNexus 2015New Analytics Toolbox DevNexus 2015
New Analytics Toolbox DevNexus 2015
 
New Analytics Toolbox
New Analytics ToolboxNew Analytics Toolbox
New Analytics Toolbox
 
Big Data Grows Up - A (re)introduction to Cassandra
Big Data Grows Up - A (re)introduction to CassandraBig Data Grows Up - A (re)introduction to Cassandra
Big Data Grows Up - A (re)introduction to Cassandra
 
Online Analytics with Hadoop and Cassandra
Online Analytics with Hadoop and CassandraOnline Analytics with Hadoop and Cassandra
Online Analytics with Hadoop and Cassandra
 

Dernier

Dernier (20)

Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Strategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering TeamsStrategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering Teams
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdf
 
The UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, OcadoThe UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, Ocado
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 

CQL Under the Hood

  • 1. CQL Under the Hood Robbie Strickland
  • 2. whoami? Robbie Strickland Software Development Manager @rs_atl
  • 3. Target Audience • Veterans who don’t use or don’t understand CQL • Newcomers who only know CQL, but don’t know what’s happening underneath
  • 4. Definitions • Thrift: legacy RPC protocol + code generation tool • batch_mutate, get_range_slices, multiget_slice, etc. • Deprecated in favor of native protocol • Native protocol: replacement for Thrift • Only works with CQL
  • 5. Definitions • Storage rows: keys + columns as stored on disk • CQL rows: abstraction layer on top of storage rows • Not usually a direct mapping to storage rows • Make use of predefined schema • Data still sparse – no space used for null columns
  • 6. In the Old Days … • Lots of clients with lots of APIs • No common language for describing schemas or queries • Steep learning curve
  • 7. In the Old Days … • No cursors, so entire result set must fit in memory on client and server • Hard to add new features • Lag time between release of new features and client library adoption
  • 8. Solution: CQL? SELECT * FROM mytable WHERE mykey = ‘foo’; WAT???!!!
  • 9. Solution: CQL? Veterans “What happened to my NoSQL??!!”
  • 10. Solution: CQL? Newbies “Sweet, SQL! I don’t have to learn anything new!”
  • 11. Reality • Don’t panic • Thrift problems solved • You didn’t lose anything • Underlying storage is unchanged • Don’t get lazy • CQL is not SQL • You need to know what you’re doing • No, you can’t just index everything
  • 12. Simple CQL CREATE TABLE Books (! title varchar,! author varchar,! year int,! PRIMARY KEY (title)! SELECT * FROM Books;! );! ! INSERT INTO Books (title, author, year) ! VALUES ('Patriot Games', 'Tom Clancy', 1987);! INSERT INTO Books (title, author, year) ! VALUES ('Without Remorse', 'Tom Clancy', 1993);! ! title | author | year! -----------------+------------+------! Without Remorse | Tom Clancy | 1993! Patriot Games | Tom Clancy | 1987!
  • 13. Storage Rows [default@unknown] create keyspace Library;! [default@unknown] use Library;! [default@Library] create column family Books! ...! with key_validation_class=UTF8Type! ...! and comparator=UTF8Type! ...! and default_validation_class=UTF8Type;! [default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! [default@Library] set Books['Patriot Games']['year'] = '1987';! [default@Library] list Books;! ! RowKey: Patriot Games! => (name=author, value=Tom Clancy, timestamp=1393102991499000)! => (name=year, value=1987, timestamp=1393103015955000)!
  • 14. Storage Rows [default@unknown] create keyspace Library;! [default@unknown] use Library;! [default@Library] create column family Books! ...! with key_validation_class=UTF8Type! ...! and comparator=UTF8Type! ...! and default_validation_class=UTF8Type;! [default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! [default@Library] set Books['Patriot Games']['year'] = '1987';! [default@Library] list Books;! ! RowKey: Patriot Games! => (name=author, value=Tom Clancy, timestamp=1393102991499000)! => (name=year, value=1987, timestamp=1393103015955000)! Random hash (no ordering)
  • 15. Storage Rows [default@unknown] create keyspace Library;! [default@unknown] use Library;! [default@Library] create column family Books! ...! with key_validation_class=UTF8Type! ...! and comparator=UTF8Type! ...! and default_validation_class=UTF8Type;! [default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! [default@Library] set Books['Patriot Games']['year'] = '1987';! [default@Library] list Books;! ! Ordered by name RowKey: Patriot Games! => (name=author, value=Tom Clancy, timestamp=1393102991499000)! => (name=year, value=1987, timestamp=1393103015955000)!
  • 16. Compound Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 17. Compound Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! Partition key (row key) name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 18. Compound Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! Clustering columns name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 19. == Composite Columns [default@Library] list authors;! ! Partition key (row key) RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)!
  • 20. == Composite Columns [default@Library] list authors;! ! RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! Clustering columns
  • 21. == Composite Columns [default@Library] list authors;! ! RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! Ordered by name
  • 22. Reversing Sort Order CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! ) WITH CLUSTERING ORDER BY (year DESC);! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
  • 23. Reversing Sort Order [default@Library] list authors;! ! RowKey: Tom Clancy! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 24. Composite Partition Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY ((name, year), title)! );! Composite partition key name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 25. Composite Partition Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY ((name, year), title)! );! Clustering column name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 26. == Composite Keys [default@Library] list authors;! ! Partition keys (row key) RowKey: Tom Clancy:1993! => (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! => (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! -------------------! RowKey: Tom Clancy:1987! => (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! => (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
  • 27. == Composite Keys [default@Library] list authors;! ! RowKey: Tom Clancy:1993! => (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! => (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! -------------------! RowKey: Tom Clancy:1987! => (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! => (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)! Clustering column
  • 28. Why This Matters • Queries must respect underlying storage, else they will either be slow or impossible • You have to know your partition key at query time • If you want fast multi-record queries, select a range, in storage order • With clustering columns, order matters
  • 29. Example CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! ) WITH CLUSTERING ORDER BY (year DESC);!
  • 30. Example name | year | title | isbn | publisher! ------------+------+-----------------------------+---------------+-----------! Tom Clancy | 1996 | Executive Orders | 0-399-13825-0 | Putnam! Tom Clancy | 1994 | Debt of Honor | 0-399-13826-1 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13927-0 | Putnam! Tom Clancy | 1991 | The Sum of All Fears | 0-399-12341-6 | Putnam! Tom Clancy | 1989 | Clear and Present Danger | 0-399-13341-1 | Putnam! Tom Clancy | 1988 | The Cardinal of the Kremlin | 0-399-13241-4 | Putnam! Tom Clancy | 1987 | Patriot Games | 0-399-13231-4 | Putnam! Tom Clancy | 1986 | Red Storm Rising | 0-399-13230-2 | Putnam! Tom Clancy | 1984 | The Hunt for Red October | 0-399-13251-1 | Putnam! !!
  • 31. Query by Key SELECT * FROM authors WHERE name = ‘Tom Clancy’ (CL = QUORUM) RF = 3 Tom Clancy Tom Clancy Tom Clancy
  • 32. Query by Key Find partition key RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 33. Query by Key Scan all columns in order RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 34. Range Query SELECT * FROM authors WHERE name = ‘Tom Clancy’ AND year >= 1990 (CL = QUORUM) Tom Clancy Tom Clancy Tom Clancy
  • 35. Range Query Find partition key RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 36. Range Query Scan until < 1990 RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 37. Querying Tail of Range SELECT * FROM authors WHERE name = ‘Tom Clancy’ AND year <= 1990 (CL = QUORUM) Tom Clancy Tom Clancy Tom Clancy
  • 38. Querying Tail of Range Find partition key RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 39. Querying Tail of Range Scan all, then filter <= 1990 RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 40. Multiple Keys DK DK MG MG TC MG TC TC DK SELECT * FROM authors WHERE name IN (‘Tom Clancy’, ‘Dean Koontz’, ‘Malcolm Gladwell’) (CL = QUORUM)
  • 41. Lessons Learned • Sequential == fast • Query by key / clustering column (range) == Sequential • Multi-key query often == lots of nodes • Write in the intended read order
  • 42. Collections • Sets: unordered, unique • Lists: ordered, allow duplicates • Maps: key/value pairs – can be a good substitute for dynamic columns • Max 64k items, 64k per item • Always returns entire collection
  • 43. Sets CREATE TABLE authors (! !name text,! !books set<text>,! PRIMARY KEY (name)! name | books! ------------+--------------------------------------! Tom Clancy | {'Patriot Games', 'Without Remorse'}! );! ! INSERT INTO authors (name, books) ! VALUES ('Tom Clancy', {'Without Remorse', 'Patriot Games'});!
  • 44. Sets RowKey: Tom Clancy! => (name=books:50617472696f742047616d6573, value=, ...)! => (name=books:576974686f75742052656d6f727365, value=, ...)! Set name Set item Empty value
  • 45. Lists CREATE TABLE authors (! !name text,! !books list<text>,! PRIMARY KEY (name)! name | books! ------------+--------------------------------------! Tom Clancy | ['Without Remorse’, 'Patriot Games']! );! ! INSERT INTO authors (name, books) ! VALUES ('Tom Clancy', ['Without Remorse', 'Patriot Games']);!
  • 46. Lists RowKey: Tom Clancy! => (name=books:d36de8b0305011e4a0dddbbeade718be, value=576974686f75742052656d6f727365, ...)! => (name=books:d36de8b1305011e4a0dddbbeade718be, value=50617472696f742047616d6573, ...)! List name Ordering ID List item
  • 47. Maps CREATE TABLE authors (! !name text,! !books map<text, int>,! PRIMARY KEY (name)! name | books! ------------+-------------------------------------------------! Tom Clancy | {'Patriot Games': 1987, 'Without Remorse': 1993}! );! ! INSERT INTO authors (name, books) ! VALUES ('Tom Clancy', ! {'Without Remorse' : 1993, 'Patriot Games' : 1987});!
  • 48. Maps RowKey: Tom Clancy! => (name=books:50617472696f742047616d6573, value=000007c3, ...)! => (name=books:576974686f75742052656d6f727365, value=000007c9, ...)! Map name Map key Map value
  • 49. Indices CREATE INDEX author_publisher! ON author (publisher);!
  • 50. Indices • Allow query by value in certain cases • Partitioned based on row key of indexed table • Are updated atomically along with the data being inserted • Must be low cardinality, or it won’t scale well (to large cluster sizes) • But not too low, or it’s sort of pointless
  • 51. Index Distribution Node 1 Authors “Tom Clancy” : “Putnam”! “Mark Twain” : “Putnam”! ! Index “Putnam” : “Tom Clancy”! “Putnam” : “Mark Twain”! ! Node 2 Authors “Mark Twain” : “Putnam”! “Dan Brown” : “Putnam”! ! Index “Putnam” : “Mark Twain”! “Putnam” : “Dan Brown”! ! Node 3 Authors “Dan Brown” : “Putnam” “Tom Clancy” : “Putnam”! ! Index “Putnam” : “Dan Brown”! “Putnam” : “Tom Clancy”! ! Index key == indexed column value
  • 52. Index Distribution Node 1 Authors “Tom Clancy” : “Putnam”! “Mark Twain” : “Putnam”! ! Index “Putnam” : “Tom Clancy”! “Putnam” : “Mark Twain”! ! Node 2 Authors “Mark Twain” : “Putnam”! “Dan Brown” : “Putnam”! ! Index “Putnam” : “Mark Twain”! “Putnam” : “Dan Brown”! ! Node 3 Authors “Dan Brown” : “Putnam” “Tom Clancy” : “Putnam”! ! Index “Putnam” : “Dan Brown”! “Putnam” : “Tom Clancy”! ! … but node distribution based on original table key
  • 53. Querying by Value pub idx pub idx pub idx pub idx pub idx pub idx SELECT * FROM authors WHERE publisher = ‘Putnam’ (CL = QUORUM)
  • 54. Deletes DELETE FROM authors WHERE name = 'Tom Clancy';! ! INSERT INTO authors (title, name) VALUES ('Patriot Games', 'Tom Clancy') USING TTL 86400;! ! INSERT INTO authors (title, name, year) VALUES ('Patriot Games', 'Tom Clancy', null);! ! UPDATE authors SET publisher = null WHERE name = 'Tom Clancy';! !
  • 55. Deletes • Log-structured storage, so writes are immutable • Deletes create tombstones, one for each deleted column • Cassandra must read the tombstones to make sure it doesn’t revive deleted data • Lots of deletes is an anti-pattern
  • 56. Missing Columns INSERT INTO authors (title, name, year)! VALUES ('Without Remorse', 'Tom Clancy', 1993);! ! name | year | title | isbn | publisher! ------------+------+-----------------+------+-----------! Tom Clancy | 1993 | Without Remorse | null | null! ! RowKey: Tom Clancy! => (name=1993:Without Remorse:, value=, timestamp=1409936754170000)!
  • 57. Missing Columns activity | timestamp | source | source_elapsed! ---------------------------------------------------------------------------+--------------+-----------+----------------! execute_cql3_query | 11:51:55,975 | 127.0.0.1 | 0! Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:51:55,975 | 127.0.0.1 | 47! Preparing statement | 11:51:55,975 | 127.0.0.1 | 105! Executing single-partition query on authors | 11:51:55,975 | 127.0.0.1 | 307! Acquiring sstable references | 11:51:55,975 | 127.0.0.1 | 315! Merging memtable tombstones | 11:51:55,975 | 127.0.0.1 | 328! Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:51:55,975 | 127.0.0.1 | 374! Merging data from memtables and 0 sstables | 11:51:55,975 | 127.0.0.1 | 383! Read 1 live and 0 tombstoned cells | 11:51:55,975 | 127.0.0.1 | 420! Request complete | 11:51:55,975 | 127.0.0.1 | 585! Only 1 read required
  • 58. Null Columns INSERT INTO authors (title, name, year, isbn, publisher) VALUES ('Without Remorse', 'Tom Clancy', 1993, null, null);! ! name | year | title | isbn | publisher! ------------+------+-----------------+------+-----------! Tom Clancy | 1993 | Without Remorse | null | null!
  • 59. Null Columns activity | timestamp | source | source_elapsed! ---------------------------------------------------------------------------+--------------+-----------+----------------! execute_cql3_query | 11:57:31,623 | 127.0.0.1 | 0! Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:57:31,623 | 127.0.0.1 | 41! Preparing statement | 11:57:31,623 | 127.0.0.1 | 101! Executing single-partition query on authors | 11:57:31,623 | 127.0.0.1 | 532! Acquiring sstable references | 11:57:31,623 | 127.0.0.1 | 544! Merging memtable tombstones | 11:57:31,623 | 127.0.0.1 | 571! Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:57:31,623 | 127.0.0.1 | 596! Merging data from memtables and 0 sstables | 11:57:31,623 | 127.0.0.1 | 605! Read 1 live and 2 tombstoned cells | 11:57:31,624 | 127.0.0.1 | 669! Request complete | 11:57:31,623 | 127.0.0.1 | 777! 3 reads required!
  • 60. Why Queries Fail SELECT name FROM authors WHERE title = 'Patriot Games';! Bad Request: PRIMARY KEY part title cannot be restricted (preceding part year is either not restricted or by a non-EQ relation)!
  • 61. Why Queries Fail • Failure to provide the full partition key • Querying by value without an index • Misunderstanding clustering columns
  • 62. Missing Key Parts CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY ((name, year), title)! );! ! ! SELECT name FROM authors WHERE title
  • 63. Missing Key Parts [default@Library] list authors;! ! Partition keys (row key) RowKey: Tom Clancy:1993! => (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! => (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! -------------------! RowKey: Tom Clancy:1987! => (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! => (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
  • 64. Missing Key Parts SELECT * FROM authors WHERE name = 'Tom Clancy’;! Bad Request: Partition key part year must be restricted since preceding part is!
  • 65. Missing Key Parts SELECT * FROM authors WHERE year = 1987;! Bad Request: partition key part year cannot be restricted (preceding part name is either not restricted or by a non-EQ relation)!
  • 66. Missing Key Parts SELECT * FROM authors WHERE year >= 1987;! Bad Request: partition key part year cannot be restricted (preceding part name is either not restricted or by a non-EQ relation)!
  • 67. Missing Key Parts SELECT * FROM authors ! WHERE name = 'Tom Clancy' and year >= 1987;! ! Bad Request: Only EQ and IN relation are supported on the partition key (unless you use the token() function)!
  • 68. Querying by Value SELECT * FROM authors WHERE isbn = '0-399-13241-4';! Bad Request: No indexed columns present in by-columns clause with Equal operator!
  • 69. Querying Clustering Columns CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! ! ! SELECT name FROM authors WHERE
  • 70. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! Clustering columns
  • 71. Querying Clustering Columns SELECT * FROM authors ! WHERE name = 'Tom Clancy’! AND year = 1993;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 72. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
  • 73. Querying Clustering Columns SELECT * FROM authors ! WHERE name = 'Tom Clancy’! AND year <= 1993;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 74. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
  • 75. Querying Clustering Columns SELECT * FROM authors ! WHERE name = 'Tom Clancy’! AND title = 'Patriot Games';! ! Bad Request: PRIMARY KEY part title cannot be restricted (preceding part year is either not restricted or by a non-EQ relation)! !!
  • 76. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! ?
  • 77. Querying Clustering Columns SELECT * FROM authors ! WHERE year = 1987;! ! Bad Request: Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING! !
  • 78. Querying Clustering Columns SELECT * FROM authors ! WHERE year = 1987 ALLOW FILTERING;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
  • 79. Querying Clustering Columns SELECT * FROM authors ! WHERE year = 1987 LIMIT 1 ALLOW FILTERING;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
  • 80. Summary • Stop using Thrift; use CQL instead • … but know what your model/query is doing
  • 81. Thanks! Robbie Strickland Software Development Manager @rs_atl