SlideShare une entreprise Scribd logo
1  sur  82
Télécharger pour lire hors ligne
CQL Under the Hood 
Robbie Strickland
whoami? 
Robbie Strickland 
Software Development Manager 
@rs_atl
Target Audience 
• Veterans who don’t use or don’t understand CQL 
• Newcomers who only know CQL, but don’t know what’s happening 
underneath
Definitions 
• Thrift: legacy RPC protocol + code generation tool 
• batch_mutate, get_range_slices, multiget_slice, etc. 
• Deprecated in favor of native protocol 
• Native protocol: replacement for Thrift 
• Only works with CQL
Definitions 
• Storage rows: keys + columns as stored on disk 
• CQL rows: abstraction layer on top of storage rows 
• Not usually a direct mapping to storage rows 
• Make use of predefined schema 
• Data still sparse – no space used for null columns
In the Old Days … 
• Lots of clients with lots of APIs 
• No common language for describing schemas or queries 
• Steep learning curve
In the Old Days … 
• No cursors, so entire result set must fit in memory on client and server 
• Hard to add new features 
• Lag time between release of new features and client library adoption
Solution: CQL? 
SELECT * FROM mytable WHERE mykey = ‘foo’; 
WAT???!!!
Solution: CQL? 
Veterans 
“What happened to 
my NoSQL??!!”
Solution: CQL? 
Newbies 
“Sweet, SQL! I don’t 
have to learn 
anything new!”
Reality 
• Don’t panic 
• Thrift problems solved 
• You didn’t lose anything 
• Underlying storage is 
unchanged 
• Don’t get lazy 
• CQL is not SQL 
• You need to know what 
you’re doing 
• No, you can’t just index 
everything
Simple CQL 
CREATE TABLE Books (! 
title varchar,! 
author varchar,! 
year int,! 
PRIMARY KEY (title)! 
SELECT * FROM Books;! 
);! 
! 
INSERT INTO Books (title, author, year) ! 
VALUES ('Patriot Games', 'Tom Clancy', 1987);! 
INSERT INTO Books (title, author, year) ! 
VALUES ('Without Remorse', 'Tom Clancy', 1993);! 
! 
title | author | year! 
-----------------+------------+------! 
Without Remorse | Tom Clancy | 1993! 
Patriot Games | Tom Clancy | 1987!
Storage Rows 
[default@unknown] create keyspace Library;! 
[default@unknown] use Library;! 
[default@Library] create column family Books! 
...! with key_validation_class=UTF8Type! 
...! and comparator=UTF8Type! 
...! and default_validation_class=UTF8Type;! 
[default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! 
[default@Library] set Books['Patriot Games']['year'] = '1987';! 
[default@Library] list Books;! 
! 
RowKey: Patriot Games! 
=> (name=author, value=Tom Clancy, timestamp=1393102991499000)! 
=> (name=year, value=1987, timestamp=1393103015955000)!
Storage Rows 
[default@unknown] create keyspace Library;! 
[default@unknown] use Library;! 
[default@Library] create column family Books! 
...! with key_validation_class=UTF8Type! 
...! and comparator=UTF8Type! 
...! and default_validation_class=UTF8Type;! 
[default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! 
[default@Library] set Books['Patriot Games']['year'] = '1987';! 
[default@Library] list Books;! 
! 
RowKey: Patriot Games! 
=> (name=author, value=Tom Clancy, timestamp=1393102991499000)! 
=> (name=year, value=1987, timestamp=1393103015955000)! 
Random hash (no ordering)
Storage Rows 
[default@unknown] create keyspace Library;! 
[default@unknown] use Library;! 
[default@Library] create column family Books! 
...! with key_validation_class=UTF8Type! 
...! and comparator=UTF8Type! 
...! and default_validation_class=UTF8Type;! 
[default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! 
[default@Library] set Books['Patriot Games']['year'] = '1987';! 
[default@Library] list Books;! 
! 
Ordered by name 
RowKey: Patriot Games! 
=> (name=author, value=Tom Clancy, timestamp=1393102991499000)! 
=> (name=year, value=1987, timestamp=1393103015955000)!
Compound Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Compound Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
Partition key (row key) 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Compound Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
Clustering columns 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
== Composite Columns 
[default@Library] list authors;! 
! 
Partition key (row key) 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)!
== Composite Columns 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
Clustering columns
== Composite Columns 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
Ordered by name
Reversing Sort Order 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
) WITH CLUSTERING ORDER BY (year DESC);! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
Reversing Sort Order 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Composite Partition Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY ((name, year), title)! 
);! 
Composite partition key 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Composite Partition Key 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY ((name, year), title)! 
);! 
Clustering column 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
== Composite Keys 
[default@Library] list authors;! 
! 
Partition keys (row key) 
RowKey: Tom Clancy:1993! 
=> (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! 
=> (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! 
-------------------! 
RowKey: Tom Clancy:1987! 
=> (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! 
=> (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
== Composite Keys 
[default@Library] list authors;! 
! 
RowKey: Tom Clancy:1993! 
=> (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! 
=> (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! 
-------------------! 
RowKey: Tom Clancy:1987! 
=> (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! 
=> (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)! 
Clustering column
Why This Matters 
• Queries must respect underlying storage, else they will either be slow or 
impossible 
• You have to know your partition key at query time 
• If you want fast multi-record queries, select a range, in storage order 
• With clustering columns, order matters
Example 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
) WITH CLUSTERING ORDER BY (year DESC);!
Example 
name | year | title | isbn | publisher! 
------------+------+-----------------------------+---------------+-----------! 
Tom Clancy | 1996 | Executive Orders | 0-399-13825-0 | Putnam! 
Tom Clancy | 1994 | Debt of Honor | 0-399-13826-1 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13927-0 | Putnam! 
Tom Clancy | 1991 | The Sum of All Fears | 0-399-12341-6 | Putnam! 
Tom Clancy | 1989 | Clear and Present Danger | 0-399-13341-1 | Putnam! 
Tom Clancy | 1988 | The Cardinal of the Kremlin | 0-399-13241-4 | Putnam! 
Tom Clancy | 1987 | Patriot Games | 0-399-13231-4 | Putnam! 
Tom Clancy | 1986 | Red Storm Rising | 0-399-13230-2 | Putnam! 
Tom Clancy | 1984 | The Hunt for Red October | 0-399-13251-1 | Putnam! 
!!
Query by Key 
SELECT * FROM authors 
WHERE name = ‘Tom Clancy’ 
(CL = QUORUM) 
RF = 3 
Tom 
Clancy 
Tom 
Clancy 
Tom 
Clancy
Query by Key 
Find partition key 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Query by Key 
Scan all columns in order 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Range Query 
SELECT * FROM authors 
WHERE name = ‘Tom Clancy’ 
AND year >= 1990 
(CL = QUORUM) 
Tom 
Clancy 
Tom 
Clancy 
Tom 
Clancy
Range Query 
Find partition key 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Range Query 
Scan until < 1990 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Querying Tail of Range 
SELECT * FROM authors 
WHERE name = ‘Tom Clancy’ 
AND year <= 1990 
(CL = QUORUM) 
Tom 
Clancy 
Tom 
Clancy 
Tom 
Clancy
Querying Tail of Range 
Find partition key 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Querying Tail of Range 
Scan all, 
then filter <= 1990 
RowKey: Tom Clancy! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! 
...! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
Multiple Keys 
DK 
DK 
MG 
MG 
TC 
MG 
TC 
TC 
DK 
SELECT * FROM authors 
WHERE name IN 
(‘Tom Clancy’, 
‘Dean Koontz’, 
‘Malcolm Gladwell’) 
(CL = QUORUM)
Lessons Learned 
• Sequential == fast 
• Query by key / clustering column (range) == Sequential 
• Multi-key query often == lots of nodes 
• Write in the intended read order
Collections 
• Sets: unordered, unique 
• Lists: ordered, allow duplicates 
• Maps: key/value pairs – can be a good substitute for dynamic columns 
• Max 64k items, 64k per item 
• Always returns entire collection
Sets 
CREATE TABLE authors (! 
!name text,! 
!books set<text>,! 
PRIMARY KEY (name)! 
name | books! 
------------+--------------------------------------! 
Tom Clancy | {'Patriot Games', 'Without Remorse'}! 
);! 
! 
INSERT INTO authors (name, books) ! 
VALUES ('Tom Clancy', {'Without Remorse', 'Patriot Games'});!
Sets 
RowKey: Tom Clancy! 
=> (name=books:50617472696f742047616d6573, value=, ...)! 
=> (name=books:576974686f75742052656d6f727365, value=, ...)! 
Set name 
Set item 
Empty value
Lists 
CREATE TABLE authors (! 
!name text,! 
!books list<text>,! 
PRIMARY KEY (name)! 
name | books! 
------------+--------------------------------------! 
Tom Clancy | ['Without Remorse’, 'Patriot Games']! 
);! 
! 
INSERT INTO authors (name, books) ! 
VALUES ('Tom Clancy', ['Without Remorse', 'Patriot Games']);!
Lists 
RowKey: Tom Clancy! 
=> (name=books:d36de8b0305011e4a0dddbbeade718be, value=576974686f75742052656d6f727365, ...)! 
=> (name=books:d36de8b1305011e4a0dddbbeade718be, value=50617472696f742047616d6573, ...)! 
List name 
Ordering ID 
List item
Maps 
CREATE TABLE authors (! 
!name text,! 
!books map<text, int>,! 
PRIMARY KEY (name)! 
name | books! 
------------+-------------------------------------------------! 
Tom Clancy | {'Patriot Games': 1987, 'Without Remorse': 1993}! 
);! 
! 
INSERT INTO authors (name, books) ! 
VALUES ('Tom Clancy', ! 
{'Without Remorse' : 1993, 'Patriot Games' : 1987});!
Maps 
RowKey: Tom Clancy! 
=> (name=books:50617472696f742047616d6573, value=000007c3, ...)! 
=> (name=books:576974686f75742052656d6f727365, value=000007c9, ...)! 
Map name 
Map key 
Map value
Indices 
CREATE INDEX author_publisher! 
ON author (publisher);!
Indices 
• Allow query by value in certain cases 
• Partitioned based on row key of indexed table 
• Are updated atomically along with the data being inserted 
• Must be low cardinality, or it won’t scale well (to large cluster sizes) 
• But not too low, or it’s sort of pointless
Index Distribution 
Node 1 
Authors 
“Tom Clancy” : “Putnam”! 
“Mark Twain” : “Putnam”! 
! 
Index 
“Putnam” : “Tom Clancy”! 
“Putnam” : “Mark Twain”! 
! 
Node 2 
Authors 
“Mark Twain” : “Putnam”! 
“Dan Brown” : “Putnam”! 
! 
Index 
“Putnam” : “Mark Twain”! 
“Putnam” : “Dan Brown”! 
! 
Node 3 
Authors 
“Dan Brown” : “Putnam” 
“Tom Clancy” : “Putnam”! 
! 
Index 
“Putnam” : “Dan Brown”! 
“Putnam” : “Tom Clancy”! 
! 
Index key == indexed column value
Index Distribution 
Node 1 
Authors 
“Tom Clancy” : “Putnam”! 
“Mark Twain” : “Putnam”! 
! 
Index 
“Putnam” : “Tom Clancy”! 
“Putnam” : “Mark Twain”! 
! 
Node 2 
Authors 
“Mark Twain” : “Putnam”! 
“Dan Brown” : “Putnam”! 
! 
Index 
“Putnam” : “Mark Twain”! 
“Putnam” : “Dan Brown”! 
! 
Node 3 
Authors 
“Dan Brown” : “Putnam” 
“Tom Clancy” : “Putnam”! 
! 
Index 
“Putnam” : “Dan Brown”! 
“Putnam” : “Tom Clancy”! 
! 
… but node distribution based on original table key
Querying by Value 
pub idx 
pub idx 
pub idx pub idx 
pub idx 
pub idx 
SELECT * FROM authors 
WHERE publisher = ‘Putnam’ 
(CL = QUORUM)
Deletes 
DELETE FROM authors WHERE name = 'Tom Clancy';! 
! 
INSERT INTO authors (title, name) 
VALUES ('Patriot Games', 'Tom Clancy') 
USING TTL 86400;! 
! 
INSERT INTO authors (title, name, year) 
VALUES ('Patriot Games', 'Tom Clancy', null);! 
! 
UPDATE authors SET publisher = null 
WHERE name = 'Tom Clancy';! 
!
Deletes 
• Log-structured storage, so writes are immutable 
• Deletes create tombstones, one for each deleted column 
• Cassandra must read the tombstones to make sure it doesn’t revive 
deleted data 
• Lots of deletes is an anti-pattern
Missing Columns 
INSERT INTO authors (title, name, year)! 
VALUES ('Without Remorse', 'Tom Clancy', 1993);! 
! 
name | year | title | isbn | publisher! 
------------+------+-----------------+------+-----------! 
Tom Clancy | 1993 | Without Remorse | null | null! 
! 
RowKey: Tom Clancy! 
=> (name=1993:Without Remorse:, value=, timestamp=1409936754170000)!
Missing Columns 
activity | timestamp | source | source_elapsed! 
---------------------------------------------------------------------------+--------------+-----------+----------------! 
execute_cql3_query | 11:51:55,975 | 127.0.0.1 | 0! 
Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:51:55,975 | 127.0.0.1 | 47! 
Preparing statement | 11:51:55,975 | 127.0.0.1 | 105! 
Executing single-partition query on authors | 11:51:55,975 | 127.0.0.1 | 307! 
Acquiring sstable references | 11:51:55,975 | 127.0.0.1 | 315! 
Merging memtable tombstones | 11:51:55,975 | 127.0.0.1 | 328! 
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:51:55,975 | 127.0.0.1 | 374! 
Merging data from memtables and 0 sstables | 11:51:55,975 | 127.0.0.1 | 383! 
Read 1 live and 0 tombstoned cells | 11:51:55,975 | 127.0.0.1 | 420! 
Request complete | 11:51:55,975 | 127.0.0.1 | 585! 
Only 1 read required
Null Columns 
INSERT INTO authors (title, name, year, isbn, publisher) 
VALUES ('Without Remorse', 'Tom Clancy', 1993, null, null);! 
! 
name | year | title | isbn | publisher! 
------------+------+-----------------+------+-----------! 
Tom Clancy | 1993 | Without Remorse | null | null!
Null Columns 
activity | timestamp | source | source_elapsed! 
---------------------------------------------------------------------------+--------------+-----------+----------------! 
execute_cql3_query | 11:57:31,623 | 127.0.0.1 | 0! 
Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:57:31,623 | 127.0.0.1 | 41! 
Preparing statement | 11:57:31,623 | 127.0.0.1 | 101! 
Executing single-partition query on authors | 11:57:31,623 | 127.0.0.1 | 532! 
Acquiring sstable references | 11:57:31,623 | 127.0.0.1 | 544! 
Merging memtable tombstones | 11:57:31,623 | 127.0.0.1 | 571! 
Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:57:31,623 | 127.0.0.1 | 596! 
Merging data from memtables and 0 sstables | 11:57:31,623 | 127.0.0.1 | 605! 
Read 1 live and 2 tombstoned cells | 11:57:31,624 | 127.0.0.1 | 669! 
Request complete | 11:57:31,623 | 127.0.0.1 | 777! 
3 reads required!
Why Queries Fail 
SELECT name FROM authors WHERE title = 'Patriot Games';! 
Bad Request: PRIMARY KEY part title cannot be 
restricted (preceding part year is either not 
restricted or by a non-EQ relation)!
Why Queries Fail 
• Failure to provide the full partition key 
• Querying by value without an index 
• Misunderstanding clustering columns
Missing Key Parts 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY ((name, year), title)! 
);! 
! 
! 
SELECT name FROM authors WHERE title
Missing Key Parts 
[default@Library] list authors;! 
! 
Partition keys (row key) 
RowKey: Tom Clancy:1993! 
=> (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! 
=> (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! 
-------------------! 
RowKey: Tom Clancy:1987! 
=> (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! 
=> (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
Missing Key Parts 
SELECT * FROM authors WHERE name = 'Tom Clancy’;! 
Bad Request: Partition key part year must be restricted 
since preceding part is!
Missing Key Parts 
SELECT * FROM authors WHERE year = 1987;! 
Bad Request: partition key part year cannot be 
restricted (preceding part name is either not 
restricted or by a non-EQ relation)!
Missing Key Parts 
SELECT * FROM authors WHERE year >= 1987;! 
Bad Request: partition key part year cannot be 
restricted (preceding part name is either not 
restricted or by a non-EQ relation)!
Missing Key Parts 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy' and year >= 1987;! 
! 
Bad Request: Only EQ and IN relation are supported on 
the partition key (unless you use the token() function)!
Querying by Value 
SELECT * FROM authors WHERE isbn = '0-399-13241-4';! 
Bad Request: No indexed columns present in by-columns 
clause with Equal operator!
Querying Clustering Columns 
CREATE TABLE authors (! 
!name text,! 
!year int,! 
!title text,! 
!isbn text,! 
!publisher text,! 
!PRIMARY KEY (name, year, title)! 
);! 
! 
! 
SELECT name FROM authors WHERE
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
Clustering columns
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy’! 
AND year = 1993;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy’! 
AND year <= 1993;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! 
Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE name = 'Tom Clancy’! 
AND title = 'Patriot Games';! 
! 
Bad Request: PRIMARY KEY part title cannot be restricted 
(preceding part year is either not restricted or by a 
non-EQ relation)! 
!!
Querying Clustering Columns 
RowKey: Tom Clancy! 
=> (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! 
=> (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! 
=> (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! 
=> (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! 
=> (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! 
?
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE year = 1987;! 
! 
Bad Request: Cannot execute this query as it might 
involve data filtering and thus may have unpredictable 
performance. If you want to execute this query despite 
the performance unpredictability, use ALLOW FILTERING! 
!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE year = 1987 ALLOW FILTERING;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
Querying Clustering Columns 
SELECT * FROM authors ! 
WHERE year = 1987 LIMIT 1 ALLOW FILTERING;! 
name | year | title | isbn | publisher! 
------------+------+-----------------+---------------+-----------! 
Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
Summary 
• Stop using Thrift; use CQL instead 
• … but know what your model/query is doing
Thanks! 
Robbie Strickland 
Software Development Manager 
@rs_atl
CQL Under the Hood

Contenu connexe

Tendances

Real data models of silicon valley
Real data models of silicon valleyReal data models of silicon valley
Real data models of silicon valleyPatrick McFadin
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling StrategiesWebscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling StrategiesJonathan Katz
 
High Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal CloudHigh Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal CloudMongoDB
 
Linux Performance Tools 2014
Linux Performance Tools 2014Linux Performance Tools 2014
Linux Performance Tools 2014Brendan Gregg
 
The MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer TraceThe MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer Traceoysteing
 
Broken Linux Performance Tools 2016
Broken Linux Performance Tools 2016Broken Linux Performance Tools 2016
Broken Linux Performance Tools 2016Brendan Gregg
 
The Apache Spark File Format Ecosystem
The Apache Spark File Format EcosystemThe Apache Spark File Format Ecosystem
The Apache Spark File Format EcosystemDatabricks
 
Apache Calcite (a tutorial given at BOSS '21)
Apache Calcite (a tutorial given at BOSS '21)Apache Calcite (a tutorial given at BOSS '21)
Apache Calcite (a tutorial given at BOSS '21)Julian Hyde
 
MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6MYXPLAIN
 
Deep dive to PostgreSQL Indexes
Deep dive to PostgreSQL IndexesDeep dive to PostgreSQL Indexes
Deep dive to PostgreSQL IndexesIbrar Ahmed
 
memcached Binary Protocol in a Nutshell
memcached Binary Protocol in a Nutshellmemcached Binary Protocol in a Nutshell
memcached Binary Protocol in a NutshellToru Maesaka
 
Indexing and Performance Tuning
Indexing and Performance TuningIndexing and Performance Tuning
Indexing and Performance TuningMongoDB
 
Apache Calcite overview
Apache Calcite overviewApache Calcite overview
Apache Calcite overviewJulian Hyde
 
Introduction to Apache Calcite
Introduction to Apache CalciteIntroduction to Apache Calcite
Introduction to Apache CalciteJordan Halterman
 
BI, Reporting and Analytics on Apache Cassandra
BI, Reporting and Analytics on Apache CassandraBI, Reporting and Analytics on Apache Cassandra
BI, Reporting and Analytics on Apache CassandraVictor Coustenoble
 

Tendances (20)

ORC Deep Dive 2020
ORC Deep Dive 2020ORC Deep Dive 2020
ORC Deep Dive 2020
 
ZeroMQ
ZeroMQZeroMQ
ZeroMQ
 
Real data models of silicon valley
Real data models of silicon valleyReal data models of silicon valley
Real data models of silicon valley
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling StrategiesWebscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
 
FASTER Key-Value Store and Log
FASTER Key-Value Store and LogFASTER Key-Value Store and Log
FASTER Key-Value Store and Log
 
Postgres index types
Postgres index typesPostgres index types
Postgres index types
 
High Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal CloudHigh Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal Cloud
 
Linux Performance Tools 2014
Linux Performance Tools 2014Linux Performance Tools 2014
Linux Performance Tools 2014
 
The MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer TraceThe MySQL Query Optimizer Explained Through Optimizer Trace
The MySQL Query Optimizer Explained Through Optimizer Trace
 
Broken Linux Performance Tools 2016
Broken Linux Performance Tools 2016Broken Linux Performance Tools 2016
Broken Linux Performance Tools 2016
 
The Apache Spark File Format Ecosystem
The Apache Spark File Format EcosystemThe Apache Spark File Format Ecosystem
The Apache Spark File Format Ecosystem
 
eBPF Workshop
eBPF WorkshopeBPF Workshop
eBPF Workshop
 
Apache Calcite (a tutorial given at BOSS '21)
Apache Calcite (a tutorial given at BOSS '21)Apache Calcite (a tutorial given at BOSS '21)
Apache Calcite (a tutorial given at BOSS '21)
 
MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6MySQL Indexing - Best practices for MySQL 5.6
MySQL Indexing - Best practices for MySQL 5.6
 
Deep dive to PostgreSQL Indexes
Deep dive to PostgreSQL IndexesDeep dive to PostgreSQL Indexes
Deep dive to PostgreSQL Indexes
 
memcached Binary Protocol in a Nutshell
memcached Binary Protocol in a Nutshellmemcached Binary Protocol in a Nutshell
memcached Binary Protocol in a Nutshell
 
Indexing and Performance Tuning
Indexing and Performance TuningIndexing and Performance Tuning
Indexing and Performance Tuning
 
Apache Calcite overview
Apache Calcite overviewApache Calcite overview
Apache Calcite overview
 
Introduction to Apache Calcite
Introduction to Apache CalciteIntroduction to Apache Calcite
Introduction to Apache Calcite
 
BI, Reporting and Analytics on Apache Cassandra
BI, Reporting and Analytics on Apache CassandraBI, Reporting and Analytics on Apache Cassandra
BI, Reporting and Analytics on Apache Cassandra
 

En vedette

Building a distributed Key-Value store with Cassandra
Building a distributed Key-Value store with CassandraBuilding a distributed Key-Value store with Cassandra
Building a distributed Key-Value store with Cassandraaaronmorton
 
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...DataStax
 
Batman and robin 036
Batman and robin 036Batman and robin 036
Batman and robin 036combookdude
 
Dc comics essentials batman - year one
Dc comics essentials   batman - year oneDc comics essentials   batman - year one
Dc comics essentials batman - year onecombookdude
 
Asterix and the cauldron
Asterix and the cauldronAsterix and the cauldron
Asterix and the cauldronLUIS NARBONA
 
DataStax: A deep look at the CQL WHERE clause
DataStax: A deep look at the CQL WHERE clauseDataStax: A deep look at the CQL WHERE clause
DataStax: A deep look at the CQL WHERE clauseDataStax Academy
 
Understanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data StructureUnderstanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data StructureDataStax
 
Indexing in Cassandra
Indexing in CassandraIndexing in Cassandra
Indexing in CassandraEd Anuff
 
Cassandra By Example: Data Modelling with CQL3
Cassandra By Example: Data Modelling with CQL3Cassandra By Example: Data Modelling with CQL3
Cassandra By Example: Data Modelling with CQL3Eric Evans
 
Advanced data modeling with apache cassandra
Advanced data modeling with apache cassandraAdvanced data modeling with apache cassandra
Advanced data modeling with apache cassandraPatrick McFadin
 
Linux Profiling at Netflix
Linux Profiling at NetflixLinux Profiling at Netflix
Linux Profiling at NetflixBrendan Gregg
 
The Walking Dead volume 5
The Walking Dead volume 5The Walking Dead volume 5
The Walking Dead volume 5dmxallen
 

En vedette (16)

Building a distributed Key-Value store with Cassandra
Building a distributed Key-Value store with CassandraBuilding a distributed Key-Value store with Cassandra
Building a distributed Key-Value store with Cassandra
 
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
An Effective Approach to Migrate Cassandra Thrift to CQL (Yabin Meng, Pythian...
 
CQL3 in depth
CQL3 in depthCQL3 in depth
CQL3 in depth
 
Batman and robin 036
Batman and robin 036Batman and robin 036
Batman and robin 036
 
Thor 002
Thor 002Thor 002
Thor 002
 
Dc comics essentials batman - year one
Dc comics essentials   batman - year oneDc comics essentials   batman - year one
Dc comics essentials batman - year one
 
Asterix and the cauldron
Asterix and the cauldronAsterix and the cauldron
Asterix and the cauldron
 
DataStax: A deep look at the CQL WHERE clause
DataStax: A deep look at the CQL WHERE clauseDataStax: A deep look at the CQL WHERE clause
DataStax: A deep look at the CQL WHERE clause
 
Batman
BatmanBatman
Batman
 
Understanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data StructureUnderstanding How CQL3 Maps to Cassandra's Internal Data Structure
Understanding How CQL3 Maps to Cassandra's Internal Data Structure
 
Indexing in Cassandra
Indexing in CassandraIndexing in Cassandra
Indexing in Cassandra
 
Marvel Comics
Marvel ComicsMarvel Comics
Marvel Comics
 
Cassandra By Example: Data Modelling with CQL3
Cassandra By Example: Data Modelling with CQL3Cassandra By Example: Data Modelling with CQL3
Cassandra By Example: Data Modelling with CQL3
 
Advanced data modeling with apache cassandra
Advanced data modeling with apache cassandraAdvanced data modeling with apache cassandra
Advanced data modeling with apache cassandra
 
Linux Profiling at Netflix
Linux Profiling at NetflixLinux Profiling at Netflix
Linux Profiling at Netflix
 
The Walking Dead volume 5
The Walking Dead volume 5The Walking Dead volume 5
The Walking Dead volume 5
 

Plus de Robbie Strickland

Always On: Building Highly Available Applications on Cassandra
Always On: Building Highly Available Applications on CassandraAlways On: Building Highly Available Applications on Cassandra
Always On: Building Highly Available Applications on CassandraRobbie Strickland
 
Lambda at Weather Scale - Cassandra Summit 2015
Lambda at Weather Scale - Cassandra Summit 2015Lambda at Weather Scale - Cassandra Summit 2015
Lambda at Weather Scale - Cassandra Summit 2015Robbie Strickland
 
New Analytics Toolbox DevNexus 2015
New Analytics Toolbox DevNexus 2015New Analytics Toolbox DevNexus 2015
New Analytics Toolbox DevNexus 2015Robbie Strickland
 
Big Data Grows Up - A (re)introduction to Cassandra
Big Data Grows Up - A (re)introduction to CassandraBig Data Grows Up - A (re)introduction to Cassandra
Big Data Grows Up - A (re)introduction to CassandraRobbie Strickland
 
Online Analytics with Hadoop and Cassandra
Online Analytics with Hadoop and CassandraOnline Analytics with Hadoop and Cassandra
Online Analytics with Hadoop and CassandraRobbie Strickland
 

Plus de Robbie Strickland (6)

Always On: Building Highly Available Applications on Cassandra
Always On: Building Highly Available Applications on CassandraAlways On: Building Highly Available Applications on Cassandra
Always On: Building Highly Available Applications on Cassandra
 
Lambda at Weather Scale - Cassandra Summit 2015
Lambda at Weather Scale - Cassandra Summit 2015Lambda at Weather Scale - Cassandra Summit 2015
Lambda at Weather Scale - Cassandra Summit 2015
 
New Analytics Toolbox DevNexus 2015
New Analytics Toolbox DevNexus 2015New Analytics Toolbox DevNexus 2015
New Analytics Toolbox DevNexus 2015
 
New Analytics Toolbox
New Analytics ToolboxNew Analytics Toolbox
New Analytics Toolbox
 
Big Data Grows Up - A (re)introduction to Cassandra
Big Data Grows Up - A (re)introduction to CassandraBig Data Grows Up - A (re)introduction to Cassandra
Big Data Grows Up - A (re)introduction to Cassandra
 
Online Analytics with Hadoop and Cassandra
Online Analytics with Hadoop and CassandraOnline Analytics with Hadoop and Cassandra
Online Analytics with Hadoop and Cassandra
 

Dernier

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilV3cube
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesBoston Institute of Analytics
 

Dernier (20)

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 

CQL Under the Hood

  • 1. CQL Under the Hood Robbie Strickland
  • 2. whoami? Robbie Strickland Software Development Manager @rs_atl
  • 3. Target Audience • Veterans who don’t use or don’t understand CQL • Newcomers who only know CQL, but don’t know what’s happening underneath
  • 4. Definitions • Thrift: legacy RPC protocol + code generation tool • batch_mutate, get_range_slices, multiget_slice, etc. • Deprecated in favor of native protocol • Native protocol: replacement for Thrift • Only works with CQL
  • 5. Definitions • Storage rows: keys + columns as stored on disk • CQL rows: abstraction layer on top of storage rows • Not usually a direct mapping to storage rows • Make use of predefined schema • Data still sparse – no space used for null columns
  • 6. In the Old Days … • Lots of clients with lots of APIs • No common language for describing schemas or queries • Steep learning curve
  • 7. In the Old Days … • No cursors, so entire result set must fit in memory on client and server • Hard to add new features • Lag time between release of new features and client library adoption
  • 8. Solution: CQL? SELECT * FROM mytable WHERE mykey = ‘foo’; WAT???!!!
  • 9. Solution: CQL? Veterans “What happened to my NoSQL??!!”
  • 10. Solution: CQL? Newbies “Sweet, SQL! I don’t have to learn anything new!”
  • 11. Reality • Don’t panic • Thrift problems solved • You didn’t lose anything • Underlying storage is unchanged • Don’t get lazy • CQL is not SQL • You need to know what you’re doing • No, you can’t just index everything
  • 12. Simple CQL CREATE TABLE Books (! title varchar,! author varchar,! year int,! PRIMARY KEY (title)! SELECT * FROM Books;! );! ! INSERT INTO Books (title, author, year) ! VALUES ('Patriot Games', 'Tom Clancy', 1987);! INSERT INTO Books (title, author, year) ! VALUES ('Without Remorse', 'Tom Clancy', 1993);! ! title | author | year! -----------------+------------+------! Without Remorse | Tom Clancy | 1993! Patriot Games | Tom Clancy | 1987!
  • 13. Storage Rows [default@unknown] create keyspace Library;! [default@unknown] use Library;! [default@Library] create column family Books! ...! with key_validation_class=UTF8Type! ...! and comparator=UTF8Type! ...! and default_validation_class=UTF8Type;! [default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! [default@Library] set Books['Patriot Games']['year'] = '1987';! [default@Library] list Books;! ! RowKey: Patriot Games! => (name=author, value=Tom Clancy, timestamp=1393102991499000)! => (name=year, value=1987, timestamp=1393103015955000)!
  • 14. Storage Rows [default@unknown] create keyspace Library;! [default@unknown] use Library;! [default@Library] create column family Books! ...! with key_validation_class=UTF8Type! ...! and comparator=UTF8Type! ...! and default_validation_class=UTF8Type;! [default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! [default@Library] set Books['Patriot Games']['year'] = '1987';! [default@Library] list Books;! ! RowKey: Patriot Games! => (name=author, value=Tom Clancy, timestamp=1393102991499000)! => (name=year, value=1987, timestamp=1393103015955000)! Random hash (no ordering)
  • 15. Storage Rows [default@unknown] create keyspace Library;! [default@unknown] use Library;! [default@Library] create column family Books! ...! with key_validation_class=UTF8Type! ...! and comparator=UTF8Type! ...! and default_validation_class=UTF8Type;! [default@Library] set Books['Patriot Games']['author'] = 'Tom Clancy';! [default@Library] set Books['Patriot Games']['year'] = '1987';! [default@Library] list Books;! ! Ordered by name RowKey: Patriot Games! => (name=author, value=Tom Clancy, timestamp=1393102991499000)! => (name=year, value=1987, timestamp=1393103015955000)!
  • 16. Compound Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 17. Compound Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! Partition key (row key) name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 18. Compound Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! Clustering columns name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 19. == Composite Columns [default@Library] list authors;! ! Partition key (row key) RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)!
  • 20. == Composite Columns [default@Library] list authors;! ! RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! Clustering columns
  • 21. == Composite Columns [default@Library] list authors;! ! RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! Ordered by name
  • 22. Reversing Sort Order CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! ) WITH CLUSTERING ORDER BY (year DESC);! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
  • 23. Reversing Sort Order [default@Library] list authors;! ! RowKey: Tom Clancy! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 24. Composite Partition Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY ((name, year), title)! );! Composite partition key name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 25. Composite Partition Key CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY ((name, year), title)! );! Clustering column name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 26. == Composite Keys [default@Library] list authors;! ! Partition keys (row key) RowKey: Tom Clancy:1993! => (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! => (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! -------------------! RowKey: Tom Clancy:1987! => (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! => (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
  • 27. == Composite Keys [default@Library] list authors;! ! RowKey: Tom Clancy:1993! => (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! => (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! -------------------! RowKey: Tom Clancy:1987! => (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! => (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)! Clustering column
  • 28. Why This Matters • Queries must respect underlying storage, else they will either be slow or impossible • You have to know your partition key at query time • If you want fast multi-record queries, select a range, in storage order • With clustering columns, order matters
  • 29. Example CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! ) WITH CLUSTERING ORDER BY (year DESC);!
  • 30. Example name | year | title | isbn | publisher! ------------+------+-----------------------------+---------------+-----------! Tom Clancy | 1996 | Executive Orders | 0-399-13825-0 | Putnam! Tom Clancy | 1994 | Debt of Honor | 0-399-13826-1 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13927-0 | Putnam! Tom Clancy | 1991 | The Sum of All Fears | 0-399-12341-6 | Putnam! Tom Clancy | 1989 | Clear and Present Danger | 0-399-13341-1 | Putnam! Tom Clancy | 1988 | The Cardinal of the Kremlin | 0-399-13241-4 | Putnam! Tom Clancy | 1987 | Patriot Games | 0-399-13231-4 | Putnam! Tom Clancy | 1986 | Red Storm Rising | 0-399-13230-2 | Putnam! Tom Clancy | 1984 | The Hunt for Red October | 0-399-13251-1 | Putnam! !!
  • 31. Query by Key SELECT * FROM authors WHERE name = ‘Tom Clancy’ (CL = QUORUM) RF = 3 Tom Clancy Tom Clancy Tom Clancy
  • 32. Query by Key Find partition key RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 33. Query by Key Scan all columns in order RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 34. Range Query SELECT * FROM authors WHERE name = ‘Tom Clancy’ AND year >= 1990 (CL = QUORUM) Tom Clancy Tom Clancy Tom Clancy
  • 35. Range Query Find partition key RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 36. Range Query Scan until < 1990 RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 37. Querying Tail of Range SELECT * FROM authors WHERE name = ‘Tom Clancy’ AND year <= 1990 (CL = QUORUM) Tom Clancy Tom Clancy Tom Clancy
  • 38. Querying Tail of Range Find partition key RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 39. Querying Tail of Range Scan all, then filter <= 1990 RowKey: Tom Clancy! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1994:Debt of Honor:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1994:Debt of Honor:ISBN, value=0-399-13826-1, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1991:The Sum of All Fears:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1991:The Sum of All Fears:ISBN, value=0-399-13241-6, timestamp=1393104011458000)! ...! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)!
  • 40. Multiple Keys DK DK MG MG TC MG TC TC DK SELECT * FROM authors WHERE name IN (‘Tom Clancy’, ‘Dean Koontz’, ‘Malcolm Gladwell’) (CL = QUORUM)
  • 41. Lessons Learned • Sequential == fast • Query by key / clustering column (range) == Sequential • Multi-key query often == lots of nodes • Write in the intended read order
  • 42. Collections • Sets: unordered, unique • Lists: ordered, allow duplicates • Maps: key/value pairs – can be a good substitute for dynamic columns • Max 64k items, 64k per item • Always returns entire collection
  • 43. Sets CREATE TABLE authors (! !name text,! !books set<text>,! PRIMARY KEY (name)! name | books! ------------+--------------------------------------! Tom Clancy | {'Patriot Games', 'Without Remorse'}! );! ! INSERT INTO authors (name, books) ! VALUES ('Tom Clancy', {'Without Remorse', 'Patriot Games'});!
  • 44. Sets RowKey: Tom Clancy! => (name=books:50617472696f742047616d6573, value=, ...)! => (name=books:576974686f75742052656d6f727365, value=, ...)! Set name Set item Empty value
  • 45. Lists CREATE TABLE authors (! !name text,! !books list<text>,! PRIMARY KEY (name)! name | books! ------------+--------------------------------------! Tom Clancy | ['Without Remorse’, 'Patriot Games']! );! ! INSERT INTO authors (name, books) ! VALUES ('Tom Clancy', ['Without Remorse', 'Patriot Games']);!
  • 46. Lists RowKey: Tom Clancy! => (name=books:d36de8b0305011e4a0dddbbeade718be, value=576974686f75742052656d6f727365, ...)! => (name=books:d36de8b1305011e4a0dddbbeade718be, value=50617472696f742047616d6573, ...)! List name Ordering ID List item
  • 47. Maps CREATE TABLE authors (! !name text,! !books map<text, int>,! PRIMARY KEY (name)! name | books! ------------+-------------------------------------------------! Tom Clancy | {'Patriot Games': 1987, 'Without Remorse': 1993}! );! ! INSERT INTO authors (name, books) ! VALUES ('Tom Clancy', ! {'Without Remorse' : 1993, 'Patriot Games' : 1987});!
  • 48. Maps RowKey: Tom Clancy! => (name=books:50617472696f742047616d6573, value=000007c3, ...)! => (name=books:576974686f75742052656d6f727365, value=000007c9, ...)! Map name Map key Map value
  • 49. Indices CREATE INDEX author_publisher! ON author (publisher);!
  • 50. Indices • Allow query by value in certain cases • Partitioned based on row key of indexed table • Are updated atomically along with the data being inserted • Must be low cardinality, or it won’t scale well (to large cluster sizes) • But not too low, or it’s sort of pointless
  • 51. Index Distribution Node 1 Authors “Tom Clancy” : “Putnam”! “Mark Twain” : “Putnam”! ! Index “Putnam” : “Tom Clancy”! “Putnam” : “Mark Twain”! ! Node 2 Authors “Mark Twain” : “Putnam”! “Dan Brown” : “Putnam”! ! Index “Putnam” : “Mark Twain”! “Putnam” : “Dan Brown”! ! Node 3 Authors “Dan Brown” : “Putnam” “Tom Clancy” : “Putnam”! ! Index “Putnam” : “Dan Brown”! “Putnam” : “Tom Clancy”! ! Index key == indexed column value
  • 52. Index Distribution Node 1 Authors “Tom Clancy” : “Putnam”! “Mark Twain” : “Putnam”! ! Index “Putnam” : “Tom Clancy”! “Putnam” : “Mark Twain”! ! Node 2 Authors “Mark Twain” : “Putnam”! “Dan Brown” : “Putnam”! ! Index “Putnam” : “Mark Twain”! “Putnam” : “Dan Brown”! ! Node 3 Authors “Dan Brown” : “Putnam” “Tom Clancy” : “Putnam”! ! Index “Putnam” : “Dan Brown”! “Putnam” : “Tom Clancy”! ! … but node distribution based on original table key
  • 53. Querying by Value pub idx pub idx pub idx pub idx pub idx pub idx SELECT * FROM authors WHERE publisher = ‘Putnam’ (CL = QUORUM)
  • 54. Deletes DELETE FROM authors WHERE name = 'Tom Clancy';! ! INSERT INTO authors (title, name) VALUES ('Patriot Games', 'Tom Clancy') USING TTL 86400;! ! INSERT INTO authors (title, name, year) VALUES ('Patriot Games', 'Tom Clancy', null);! ! UPDATE authors SET publisher = null WHERE name = 'Tom Clancy';! !
  • 55. Deletes • Log-structured storage, so writes are immutable • Deletes create tombstones, one for each deleted column • Cassandra must read the tombstones to make sure it doesn’t revive deleted data • Lots of deletes is an anti-pattern
  • 56. Missing Columns INSERT INTO authors (title, name, year)! VALUES ('Without Remorse', 'Tom Clancy', 1993);! ! name | year | title | isbn | publisher! ------------+------+-----------------+------+-----------! Tom Clancy | 1993 | Without Remorse | null | null! ! RowKey: Tom Clancy! => (name=1993:Without Remorse:, value=, timestamp=1409936754170000)!
  • 57. Missing Columns activity | timestamp | source | source_elapsed! ---------------------------------------------------------------------------+--------------+-----------+----------------! execute_cql3_query | 11:51:55,975 | 127.0.0.1 | 0! Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:51:55,975 | 127.0.0.1 | 47! Preparing statement | 11:51:55,975 | 127.0.0.1 | 105! Executing single-partition query on authors | 11:51:55,975 | 127.0.0.1 | 307! Acquiring sstable references | 11:51:55,975 | 127.0.0.1 | 315! Merging memtable tombstones | 11:51:55,975 | 127.0.0.1 | 328! Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:51:55,975 | 127.0.0.1 | 374! Merging data from memtables and 0 sstables | 11:51:55,975 | 127.0.0.1 | 383! Read 1 live and 0 tombstoned cells | 11:51:55,975 | 127.0.0.1 | 420! Request complete | 11:51:55,975 | 127.0.0.1 | 585! Only 1 read required
  • 58. Null Columns INSERT INTO authors (title, name, year, isbn, publisher) VALUES ('Without Remorse', 'Tom Clancy', 1993, null, null);! ! name | year | title | isbn | publisher! ------------+------+-----------------+------+-----------! Tom Clancy | 1993 | Without Remorse | null | null!
  • 59. Null Columns activity | timestamp | source | source_elapsed! ---------------------------------------------------------------------------+--------------+-----------+----------------! execute_cql3_query | 11:57:31,623 | 127.0.0.1 | 0! Parsing select * from authors where name = 'Tom Clancy' LIMIT 10000; | 11:57:31,623 | 127.0.0.1 | 41! Preparing statement | 11:57:31,623 | 127.0.0.1 | 101! Executing single-partition query on authors | 11:57:31,623 | 127.0.0.1 | 532! Acquiring sstable references | 11:57:31,623 | 127.0.0.1 | 544! Merging memtable tombstones | 11:57:31,623 | 127.0.0.1 | 571! Skipped 0/0 non-slice-intersecting sstables, included 0 due to tombstones | 11:57:31,623 | 127.0.0.1 | 596! Merging data from memtables and 0 sstables | 11:57:31,623 | 127.0.0.1 | 605! Read 1 live and 2 tombstoned cells | 11:57:31,624 | 127.0.0.1 | 669! Request complete | 11:57:31,623 | 127.0.0.1 | 777! 3 reads required!
  • 60. Why Queries Fail SELECT name FROM authors WHERE title = 'Patriot Games';! Bad Request: PRIMARY KEY part title cannot be restricted (preceding part year is either not restricted or by a non-EQ relation)!
  • 61. Why Queries Fail • Failure to provide the full partition key • Querying by value without an index • Misunderstanding clustering columns
  • 62. Missing Key Parts CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY ((name, year), title)! );! ! ! SELECT name FROM authors WHERE title
  • 63. Missing Key Parts [default@Library] list authors;! ! Partition keys (row key) RowKey: Tom Clancy:1993! => (name=Without Remorse:isbn, value=0-399-13241-4, timestamp=1409344246457000)! => (name=Without Remorse:publisher, value=5075746e616d, timestamp=1409344246457000)! -------------------! RowKey: Tom Clancy:1987! => (name=Patriot Games:isbn, value=0-399-13825-0, timestamp=1409344245715000)! => (name=Patriot Games:publisher, value=5075746e616d, timestamp=1409344245715000)!
  • 64. Missing Key Parts SELECT * FROM authors WHERE name = 'Tom Clancy’;! Bad Request: Partition key part year must be restricted since preceding part is!
  • 65. Missing Key Parts SELECT * FROM authors WHERE year = 1987;! Bad Request: partition key part year cannot be restricted (preceding part name is either not restricted or by a non-EQ relation)!
  • 66. Missing Key Parts SELECT * FROM authors WHERE year >= 1987;! Bad Request: partition key part year cannot be restricted (preceding part name is either not restricted or by a non-EQ relation)!
  • 67. Missing Key Parts SELECT * FROM authors ! WHERE name = 'Tom Clancy' and year >= 1987;! ! Bad Request: Only EQ and IN relation are supported on the partition key (unless you use the token() function)!
  • 68. Querying by Value SELECT * FROM authors WHERE isbn = '0-399-13241-4';! Bad Request: No indexed columns present in by-columns clause with Equal operator!
  • 69. Querying Clustering Columns CREATE TABLE authors (! !name text,! !year int,! !title text,! !isbn text,! !publisher text,! !PRIMARY KEY (name, year, title)! );! ! ! SELECT name FROM authors WHERE
  • 70. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! Clustering columns
  • 71. Querying Clustering Columns SELECT * FROM authors ! WHERE name = 'Tom Clancy’! AND year = 1993;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 72. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
  • 73. Querying Clustering Columns SELECT * FROM authors ! WHERE name = 'Tom Clancy’! AND year <= 1993;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam! Tom Clancy | 1993 | Without Remorse | 0-399-13825-0 | Putnam!
  • 74. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)!
  • 75. Querying Clustering Columns SELECT * FROM authors ! WHERE name = 'Tom Clancy’! AND title = 'Patriot Games';! ! Bad Request: PRIMARY KEY part title cannot be restricted (preceding part year is either not restricted or by a non-EQ relation)! !!
  • 76. Querying Clustering Columns RowKey: Tom Clancy! => (name=1987:Patriot Games:ISBN, value=0-399-13241-4, timestamp=1393104011458000)! => (name=1987:Patriot Games:publisher, value=Putnam, timestamp=1393103948577000)! => (name=1993:Without Remorse:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1993:Without Remorse:publisher, value=Putnam, timestamp=1393104083773000)! => (name=1996:Executive Orders:ISBN, value=0-399-13825-0, timestamp=1393104109214000)! => (name=1996:Executive Orders:publisher, value=Putnam, timestamp=1393104083773000)! ?
  • 77. Querying Clustering Columns SELECT * FROM authors ! WHERE year = 1987;! ! Bad Request: Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING! !
  • 78. Querying Clustering Columns SELECT * FROM authors ! WHERE year = 1987 ALLOW FILTERING;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
  • 79. Querying Clustering Columns SELECT * FROM authors ! WHERE year = 1987 LIMIT 1 ALLOW FILTERING;! name | year | title | isbn | publisher! ------------+------+-----------------+---------------+-----------! Tom Clancy | 1987 | Patriot Games | 0-399-13241-4 | Putnam!
  • 80. Summary • Stop using Thrift; use CQL instead • … but know what your model/query is doing
  • 81. Thanks! Robbie Strickland Software Development Manager @rs_atl