1.
ShareChat's Journey
Migrating 100TB of Data to
ScyllaDB with NO Downtime
Chinmoy Mahapatra - Software Engineer, Platforms
Anuraj Jain - Software Engineer, Platforms

2.
■ Live Migration Framework Overview
■ Dual Writes Deep Dive
■ Handling Conflicts
■ Handling Counters
■ Export Job Overview
■ Export Job Deep Dive
■ Challenges / Learnings
■ Results
Presentation Agenda

3.
Live Migration Architecture

4.
DB Driver?

5.
Dual Writes?

6.
Resolving Conflicts

7.
Migration Stages
Stage Read Write
Source DB ScyllaDB Source DB ScyllaDB
Dual writes enabled ✅ ❌ ✅ ✅
Export job Running ✅ ❌ ✅ ✅
Audit ✅ ❌ ✅ ✅
Validation ✅ ❌ ✅ ✅
Read Switch ❌ ✅ ✅ ✅
Write Switch ❌ ✅ ❌ ✅

8.
What About Counters?

9.
What About Counters?

10.
Migration Stages (Counters)
Stage Read Write
Source DB ScyllaDB Source DB ScyllaDB
Write to dirty keystore ✅ ❌ ✅ ❌
Export job Running ✅ ❌ ✅ ❌
Dual writes ✅ ❌ ✅ ✔
Migrate dirty keys ✅ ❌ ✅ ✔
Audit ✅ ❌ ✅ ✅
Validation ✅ ❌ ✅ ✅
Read Switch ❌ ✅ ✅ ✅
Write Switch ❌ ✅ ❌ ✅

11.
Exporting the Existing Data From
Source to Destination Table

12.
Let’s See Some Code
Our job has majorly 3 parts - Read, Transform, Write

13.
How To Do a Timestamped Query?
We provide a CustomMapperFactory to our CassandraIO PTransform, which then uses a
CustomObjectMapper (a mapper interface implementation) and writes to ScyllaDB with a
custom timestamped query using a accessor.

14.
How To Do a Timestamped Query?

15.
Challenges/Learnings
1. Write Consistency | Quorum vs One
2. Partial Data Export with Apache Beam
3. Segregation of base table and materialized
4. Choosing the right compaction strategy | Incremental vs Null
5. Validation for the migrated data from entire source and
destination table
6. Migrating Counters | ensuring consistency

16.
ShareChat’s Usage of ScyllaDB
■ We have migrated around 65+ TB of data already. Plan to
onboard additional 50 TB
■ Almost 35-40 services are using ScyllaDB at scale
■ Our biggest cluster size is around 28 TB and our max throughput
for one of our cluster is 1.5 Million Ops/sec

17.
Thank You
Stay in Touch
Chinmoy Mahapatra
chinmoymahapatra@sharechat.co
https://github.com/ChinmoyMahapatra
https://www.linkedin.com/in/chinmoy-
s-mahapatra-5458b517b
Anuraj Jain
anurajjain@sharechat.co
https://github.com/anuraj381
https://www.linkedin.com/in/anuraj-jain-3101