Distributed data days 2018 San Francisco presentation

2. Relational model in Cassandra:
Will it fit?
Distributed Data Days SF, September 2018
Matija Gobec
matija.gobec@smartcat.io
@mad_max0204

3. Why this talk?

4. Agenda
Cassandra data model
Options and alternatives
UDT use case and Apache Spark

5. Who is using Cassandra?

6. What are 3 major Cassandra issues
Data model
Over-expectations
Poor resource planning

7. What are 3 major Cassandra issues
Data model
Over-expectations
Poor resource planning

8. Cassandra data model
It’s simple

9. Cassandra data model
It’s simple
Map[k, Map[k, v]]

10. Cassandra data model
It’s simple
Map[k, Map[k, v]]
It sucks

11. Cassandra data model
It’s simple
Map[k, Map[k, v]]
It sucks
Or not...

12. Data model

13. Cassandra data model
Primary key DATA
Slim row

14. Cassandra data model
Partition key
DATA
Wide row
Clustering key
DATA
Clustering key
DATA
Clustering key
...

15. But my data model looks like this

16. Or even...

17. What are my options?

18. Denormalization
Query based data model
Employee
EmployeeID
OrganizationID
Name
OrganizationID Name
Employee name
EmployeeID
1. Select all employees for a given organizationID
EmployeeID Name
2. Select employee for a given employeeID
OrganizationID
Organization
OrganizationID
Name
Relational model

19. Denormalization
Application level joins
Organization
OrganizationID
Name
Employee
EmployeeID
OrganizationID
Firstname
Lastname
Email
...
Relational model
OrganizationID Name
1. Select all employees for a given organization
2. Select employee for a given employeeID
EmployeeID Name OrganizationID
Results in multiple select statements
EmployeeID EmployeeID
...

20. Denormalization
Secondary indexes
1. Select all employees for a given organization
2. Select employee for a given employeeID
EmployeeID Name OrganizationID
Performance impact
...
CREATE SECONDARY INDEX
Organization
OrganizationID
Name
Employee
EmployeeID
OrganizationID
Firstname
Lastname
Email
...
Relational model

21. Denormalization
PROS
Fast reads
One query per request (usually)
Scalable (probably)
CONS
Complex data management
Can be extremely hard and complex on
insert/update/delete
Need to know all queries upfront

22. UDTs
CREATE TABLE keyspace.organization (
organizationid bigint PRIMARY KEY,
name text,
employees list<frozen<employee>>
);
CREATE TYPE test.employee (
employeeid bigint,
firstname text,
lastname text,
email text
);
OrganizationID Name
Employees
Employee Employee ...

23. UDTs
PROS
Fast(er) reads
One query per request
Scalable (should be!!)
Indexing?
CONS
Complex data management
No partial updates
Need to know all queries upfront
Indexing?

24. Blob data
CREATE TABLE keyspace.organization (
organizationid bigint PRIMARY KEY,
name text,
employees text / blob
);
OrganizationID Name
Employees
Employees list as a JSON text
or a serialized objects blob
JSON text or serialized objects

25. Blob data
PROS
Fast reads
One query per request
No need to serialize into JSON
CONS
Complex data management
No partial updates
Need to know all queries upfront
No indexing option

26. Relational database
PROS
It’s made for relational data
CONS
Scaling
Availability
Fault tolerance
Performance

27. Other options
Cassandra+Indexing
Cassandra+RDB
...

28. Leveraging UDTs

29. Use case
Highly nested data model
Impossible to denormalize
Fairly simple access patterns
Top level (root) entity

30. Data model
Root entity
Child entity Child entity Child entity
Child entityChild entity Child entity Child entity
Child entity Child entity
Child entity
Child entity
Child entity Child entity Child entity
Child entity
Child entity
Child entity
Child entity
Child entity
Child entity
Child entity
Child entity

31. How to insert data
Insert AS JSON (2.2+)
Inserted as string, stored as a column type
Easy to manage and debug
Keep track of the data size!!!

32. Spark dataframe UDT mapping
dataframe.as("parent").join(
child.groupBy(seq.map(col): _*)
.agg(collect_list(struct(columns.map(col): _*))
.alias(alias)), seq, joinType
)
dataframe.join(child
.withColumn(alias, struct(child.columns.map(col): _*))
.select(joinColumn, alias), Seq(joinColumn), joinType)
One to many
One to one

33. Inserting from Spark
// Save to cassandra
dataframe.write
.format("org.apache.spark.sql.cassandra")
.options(Map(
"keyspace" -> s"$keyspace",
"table" -> s"$table"
))
.mode(SaveMode.Append)
.save

34. Indexing UDTs
Not possible with just Cassandra
Lucene/Solr based secondary index
Indexing of fields on nested UDTs
Field analyzers

35. Solr schema example
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<schema name="solrSchema" version="1.5">
<types>
<fieldType class="org.apache.solr.schema.TrieIntField" name="TrieIntField"/>
<fieldType class="org.apache.solr.schema.TrieDateField" name="TrieDateField"/>
...
<fields>
<field docValues="true" indexed="true" multiValued="false" name="partition_key" stored="true"
type="TrieIntField"/>
<field docValues="true" indexed="true" multiValued="false" name="clustering_key" stored="true"
type="TrieDateField"/>
<field docValues="true" indexed="true" multiValued="false" name="some_type.id" stored="true"
type="TrieIntField" />
<field docValues="true" indexed="true" multiValued="false" name="some_type.some_other_type.id"
stored="true" type="TrieIntField" />
...
</fields>
<uniqueKey>(partition_key,clustering_key)</uniqueKey>
</schema>

36. But will it blend?

37. Closing notes
Cassandra data model supports a lot of use cases
Data modeling skills are required
Relational model is hard but not impossible
Additional tools in the ecosystem
Don’t be stubborn

38. Q&A

39. Matija Gobec
matija.gobec@smartcat.io
@mad_max0204
Thank you