KillrChat data modeling @ BreizhJUG

1. KillrChat Presentation
DuyHai DOAN, Technical Advocate
Amira LAKHAL, Agile Java Developer

2. @doanduyhai @miralak
KillrChat presentation
2
What is KillrChat ?
• scalable messaging app
Why KillrChat ?
• show real life de-normalization
• DIY exercise
• provide real application for attendees
• highlight Cassandra eco-system

3. @doanduyhai @miralak
Technology stack
3
C* Achilles
UI Bootstrap
AngularJS SockJS
Spring Boot
Spring Messaging
Spring REST
Spring Security

4. @doanduyhai @miralak
Front end layout
4

5. @doanduyhai @miralak
Getting started
5
Clone the Git repository
git clone https://github.com/doanduyhai/killrchat.git
Go into the ‘killrchat’ folder and launch in dev mode
cd killrchat
mvn spring-boot:run -Pdev

6. Step 1
User account management

7. @doanduyhai @miralak
Scalability
7
n1
n2
n3
n4
n5
n6
n7
n8
A
B
C
D
E
F
G
H
jdoe
oscar
alice
hsue
bob
Scaling by login

8. @doanduyhai @miralak
Data model
8
CREATE TABLE killrchat.users(
login text,
pass text, //password is not allowed because reserved word
lastname text,
firstname text,
bio text,
email text,
chat_rooms set<text>,
PRIMARY KEY(login));

9. @doanduyhai @miralak
User’s chat rooms
9

10. @doanduyhai @miralak
User’s chat rooms data model
10
How to store chat rooms for an user ?
CREATE TABLE killrchat.user_rooms(
login text,
room_name text,
PRIMARY KEY((login), room_name));
• pros: can store huge room count per user (106
)
• cons: separated table, needs 1 extra SELECT

11. @doanduyhai @miralak
User’s chat rooms data model
11
Best choice
• 1 SELECT fetches all data for a given user
• usually, 1 user is not in more that 1000 rooms at a time
• stores only room name
CREATE TABLE killrchat.users(
login text,
…
chat_rooms set<text>, //list of chat rooms for this user
PRIMARY KEY(login));

12. @doanduyhai @miralak
Lightweight Transaction
12
Avoid creating the same login by 2 different users ?
☞ use Lightweight Transaction
INSERT INTO killrchat.users(login, …)
VALUES (‘jdoe’, …) IF NOT EXISTS ;
Expensive operation
☞ do you create a new account every day ?

13. Demo

14. Step 2
Chat room management

15. @doanduyhai @miralak
Scalability
15
n1
n2
n3
n4
n5
n6
n7
n8
A
B
C
D
E
F
G
H
games
scala
java
politics
cassandra
Scaling by room name

16. @doanduyhai @miralak
Data model
16
CREATE TABLE killrchat.chat_rooms(
room_name text,
name text, // same as room_name
creation_date timestamp,
banner text,
creator text, // de-normalization
creator_login text,
participants set<text>, // de-normalization
PRIMARY KEY(room_name));

17. @doanduyhai @miralak
Room details
17

18. @doanduyhai @miralak
Room participants
18

19. @doanduyhai @miralak
De-normalization
19
CREATE TABLE killrchat.chat_rooms(
room_name text,
…
creator text, // JSON blob {login: …, firstname: …, lastname: …}
…
participants set<text>, // JSON blob {login: …, firstname: …, lastname: …}
PRIMARY KEY(room_name));

20. @doanduyhai @miralak
Lightweight Transaction
20
Avoid creating the same room by 2 different users ?
☞ use Lightweight Transaction
INSERT INTO killrchat.chat_rooms(room_name, …)
VALUES (‘games’, …) IF NOT EXISTS ;

21. Demo

22. Step 3
Participants management
Room deletion

23. @doanduyhai @miralak
Participant joining
23
Adding new participant
UPDATE killrchat.chat_rooms SET participants = participants + {…}
WHERE room_name = ‘games’;
❓
What if the creator deletes the room at the same time ?

24. @doanduyhai @miralak
Concurrent delete/update
24
UPDATE chat_rooms SET
participants =
participants + {login: ‘jdoe’, …}
WHERE room_name = ‘games’;
DELETE FROM chat_rooms
WHERE room_name= ‘games’;
result games
participants creator banner ...
{login: ‘jdoe’, …} ∅ ∅ ∅

25. @doanduyhai @miralak
Participant joining
25
Solution
☞ use Lightweight Transaction
UPDATE killrchat.chat_rooms SET participants = participants + {…}
WHERE room_name = ‘games’ IF name = ‘games’;
Why not use UPDATE … IF EXISTS ?

26. @doanduyhai @miralak
Participant joining
26
Solution
☞ use Lightweight Transaction
UPDATE killrchat.chat_rooms SET participants = participants + {…}
WHERE room_name = ‘games’ IF name = ‘games’;
Why not use UPDATE … IF EXISTS ?
• syntax not yet available
• use column name = room_name = partition key as condition
• see https://issues.apache.org/jira/browse/CASSANDRA-8610

27. @doanduyhai @miralak
Participant leaving
27
Removing participant (no read-before-write)
UPDATE killrchat.chat_rooms SET participants = participants - {…}
WHERE room_name = ‘games’; ❓
What if the creator deletes the room at the same time ?
• we’ll create a tombstone
• tombstone will be garbage-collected by compaction

28. @doanduyhai @miralak
Concurrent delete/update
28
UPDATE chat_rooms SET
participants =
participants - {login: ‘jdoe’, …}
WHERE room_name = ‘games’;
DELETE FROM chat_rooms
WHERE room_name= ‘games’;
result games
participants creator banner ...
∅ ∅ ∅ ∅

29. @doanduyhai @miralak
Deleting room
29
What if participant leaving at the same time ?
• not a problem, tombstone will be garbage
What if participant joining at the same time ?
☞ use Lightweight Transaction
Only room creator can delete room, no one else!
☞ use Lightweight Transaction again !

30. @doanduyhai @miralak
Deleting room
30
DELETE killrchat.chat_rooms
WHERE room_name = ‘games’
IF creator_login = <current_user_login>;
Solution
Advantages
• current user login coming from Security context, no cheating !
• slow but how often do you delete rooms ?

31. @doanduyhai @miralak
Concurrent delete/update
31
UPDATE chat_rooms SET
participants =
participants + {login: ‘jdoe’, …}
WHERE room_name = ‘games’
IF name = ‘games’;
DELETE FROM chat_rooms
WHERE room_name= ‘games’
IF creator_login = ‘jdoe’;
OK

32. @doanduyhai @miralak
Concurrent delete/update
32
UPDATE chat_rooms SET
participants =
participants + {login: ‘jdoe’, …}
WHERE room_name = ‘games’
IF name = ‘games’;
DELETE FROM chat_rooms
WHERE room_name= ‘games’
IF creator_login = ‘jdoe’;
Room deleted

33. Demo

34. Step 4
Chat messages management

35. @doanduyhai @miralak
Scalability
35
n1
n2
n3
n4
n5
n6
n7
n8
A
B
C
D
E
F
G
H
Scaling messages by room name
games
message1
… messageN
…
politics
message1
… messageN
…
java
message1
… messageN
…
cassandra
message1
… messageN
…
scala
message1
… messageN
…

36. @doanduyhai @miralak
Data model
36
CREATE TABLE killrchat.chat_room_messages(
room_name text,
message_id timeuuid,
content text,
author text, // JSON blob {login: …, firstname: …, lastname: …}
system_message boolean,
PRIMARY KEY((room_name), message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

37. Demo

38. Q & R

39. Thank You
@doanduyhai
duy_hai.doan@datastax.com
https://academy.datastax.com/
@miralak