This year, a set of technologies we’ve been developing for years, graduated together to bring you uncompromised performance, availability and elasticity. From Raft consensus to serverless and goodput. In this session Dor Laor, ScyllaDB’s CEO will cover the major innovations coming to ScyllaDB core and ScyllaDB Cloud.
7. Why ScyllaDB?
Best High Availability in the industry
Best Disaster Recovery in the industry
Best scalability in the industry
Best Price/Performance in the industry Auto-tune - out of the box performance
Fully compatible with Cassandra & DynamoDB
The power of Cassandra at the speed of Redis
No Lock-in
Open Source Software
8. ■ Quick recap for newbies
Agenda
■ Journey of 2022
■ The Future of Fast Data
29. ■ Auto Tombstone GC
■ Load & stream
■ Auto restore
■ RBNO
■ Consistent Schema
changes
Maintenance, Maintenance, Maintenance
Repair Based Node Operations is in for /all /default
■ Resumable
■ Consistent
■ Simplified
■ Fast
■ Unified
30. Top-K Queries
User defined aggregates = UDF +
WASM +
Distributed aggregates +
Workload prioritization
How many people follow Elon Musk?
What are the top-k products in shopping carts
31. Double Click on Database Drivers
■ Maintain Python, Java, Go
■ Huge boost w/ Rust
■ C++ driver, wrapped over Rust
■ Improved load balancing (shard, cache, zone aware)
■ Serverless support
■ Performance
33. ■ Raft
■ Consistent Schema
changes
■ Consistent topology
changes
Consistent Metadata by Raft
Protocol for state machine replication
Total order broadcast of state change commands
Single leader
X = 0 X += 1 CAS(X, 0, 1)
X = 0
38. ■ Abstract the servers, instances. Building block is VCPU
■ Decouple storage from compute. Still uses NVMe
■ Flexibility of annual commit
■ Encryption@rest + BYOK
■ Elasticity
■ API & Terraform
■ Advanced Network gateways
■ Self serve user management & SSO
■ Self serve billing & metering
Scylla Cloud Goes Serverless
39. Vision
Old: The power of Cassandra at the speed of Redis
New: The power of ScyllaDB, with the usability of DynamoDB
40. Gamechanger Revolution
Today
■ Eventual consistency
■ 3x replication, may not be identical
■ Storage is bound to compute
■ Full scan repair process is required
■ Cheap for high throughput,
expensive for high volume
Future
■ Raft based consistency
■ No need to repair
■ Single main replica, on S3
■ Can be cached for performance
■ Save arriving, fresh data in 3 replicas,
until sync w/ S3
■ Point in time recovery, extreme
elasticity
■ Cluster parking
■ ~Free disaster recovery backup
41. Future of Key Ranges
■ token = hash(partition key)
■ token range: set of partition keys
token
token
range
token
42. Each node has a set of tokens assigned during bootstrap
Token Metadata - vNodes
node A node B node C
A
C
B
C
A
B
43. Token Metadata - vNodes
A
C
B
C
A
B
{A, C}
{C, B}
{B, A}
{C, A}
{A, B}
{B, C}
token
metadata
replication
metadata
replication
strategy
44. Future of Key Ranges
Today
■ Key ranges =~ vNodes
■ Statically allocated @node
creation
■ Size of TBs
Coming - Tablets
■ Small range of keys
■ Size GBs-10GB
■ Dynamically controlled
■ Dynamically mapped/moves
between nodes
53. Today
■ Disaggregated, local NVMe
■ Gravity - time to stream
■ Ultra fast
■ Cheaper than EBS/SAN
■ Expensive if not queried
Object Store / S3 based
■ $ almost free
■ Completely decoupled
■ Fully consistent
■ Accessed by all nodes
■ Tiered Storage
■ Slow-> Local NVMe cache
Future of Storage
55. Future of Storage
tablet
Node
■ Compaction can happen off-query engine
■ Sstables NVME-cached by replicas (Cache-Replication-Factor)
■ Tiered Storage helps to control size of NVMe cache
■ Available for all users (not just DBaaS)
57. G.O.A.T Future
tablet
Node
■ Data in S3 - Low TCO
■ NVMe Configurable Caches == Fast, tiered storage
■ Elasticity
■ Remove nodes - Sync pending to S3 & change table ownership
■ Add nodes
■ many at once
■ New hardware added async from tablet ownership
■ Node can serve data from the first tablet assigned!
■ Get the local NVMe cache populated
■ Point in time Backup/restore
■ Add/remove data-centers easily - access local S3
■ Cluster parking
59. Bring RAFT to user tables!
CREATE TABLE foo WITH raft = TRUE;
Strongly-Consistent Tables
Do not have any databases
before ScyllaDB.
Always run nodetool cleanup
after bootstrapping a new
node.
Run repair within gc-grace
seconds.
Do not bootstrap nodes
concurrently, or make
any other topology change
Do not use SimpleStrategy in
a multi DC setup
Do not have any databases
before ScyllaDB.
Always run nodetool cleanup
after bootstrapping a new
node.
Run repair within gc-grace
seconds.
Do not bootstrap nodes
concurrently, or make
any other topology change
Do not use SimpleStrategy in
a multi DC setup
Do not have any databases
before ScyllaDB.
Always run nodetool cleanup
after bootstrapping a new
node.
Run repair within gc-grace
seconds.
Do not bootstrap nodes
concurrently, or make
any other topology change
Do not use SimpleStrategy in
a multi DC setup
Do not have any databases
before ScyllaDB.
Always run nodetool cleanup
after bootstrapping a new
node.
Run repair within gc-grace
seconds.
Do not bootstrap nodes
concurrently, or make
any other topology change
Do not use SimpleStrategy in
a multi DC setup
60. Strongly-Consistent Tables
LWT (paxos based)
■ Slow
■ 3 rounds to replicas per user
request
■ Concurrent conflicting requests
-> retries -> negative scaling
RAFT
■ Fast
■ 1 round to replicas (on leader)
■ No retries
61. 2022 ScyllaDB is great
Welcome to Serverless
Welcome to the Future
Thank You
@DorLaor