Use ScyllaDB Alternator to Use Amazon DynamoDB API, Everywhere, Better, More Affordable, All at Once

1. Use ScyllaDB to Replace Amazon DynamoDB: Everywhere, Better, More Affordable, All at Once Tzach Livyatan, VP of Product

2. Tzach Livyantan ■ ScyllaDB, VP Product ■ Tzach has a 20 year career in development, system engineering and product management. He has worked in the Telecom domain, focusing on carrier grade systems, signalling, policy and charging applications for Oracle and others.

3. ■ Head to Head: Comparing Amazon DynamoDB to ScyllaDB ■ ScyllaDB Alternator - DynamoDB Compatible API ■ Migrating from Amazon DynamoDB to ScyllaDB Agenda

4. Summary ■ ScyllaDB is compatible with Amazon DynamoDB and can run on multiple platforms ■ ScyllaDB is less expensive and has lower tail latency than Amazon DynamoDB ■ You can migrate from Amazon DynamoDB to ScyllaDB without downtime

5. Head to Head: Amazon DynamoDB Vs. ScyllaDB

6. Test Setup ■ YCSB 0.18.0+ ■ ScyllaDB Enterprise 2022.2 ■ ScyllaDB Cluster: 3 nodes - i4i.2xlarge on us-east-1 zones b,c,d ■ Loaders: 8 nodes of c5.2xlarge ■ Each loader machine runs 3 instances of YCSB with 40 threads total of 18 processes with parallelism of 720 (Tried with 50 threads- 900 parallelism and it is not better) ■ Preload of 1TB ■ Throughput was set to 70% of the maximum (burst) ■ Workload distribution: Uniform, Zipﬁan, Hotspot ■ Payload of 1.1kb ■ 10 columns of 100 bytes (YCSB's default)

7. ■ Choose the right number of loaders / threads/ connection to maximize the DB (not the loaders) ■ Choose the right distribution and parameters (e.g. Zipﬁan is very bad for DynamoDB) ■ Build a reproducible setup to run and collect results ■ Understand loader implementation and issues (e.g. Coordinated Omission) ■ Understand DB limitations (e.g. DynamoDB per partition rate limit) Why Testing is Hard

8. Results - ScyllaDB ScyllaDB with 70% capacity of max throughput Hotspot (0.013,0.95) Cost 29,172 $ Workload Split Throughput (r/s) P99 Latency in MS % Update % Reads Updates Reads 0.1 0.9 96,520 3.487 4.795 0.2 0.8 87,580 3.257 4.291 0.5 0.5 100,500 3.593 4.739 0.8 0.2 134,330 4.647 6.347 0.9 0.1 145,860 6.743 7.675

9. Results - ScyllaDB ScyllaDB with 70% capacity of max throughput ScyllaDB Cluster: 3 nodes - i4i.2xlarge Replication Factor: 3 Hotspot (0.013,0.95) Cost 29,172 $

10. Results - DynamoDB Hotspot (0.013,0.95) Workload Split Throughput (r/s) P99 Latency in MS % Update % Reads Updates Read 100% 0% 99,460 24.079 X 90% 10% 110,450 23.807 20.095 50% 50% 197,900 18.943 15.743 10% 90% 214,580 23.407 19.695 0% 100% 198,640 X 19.695

11. Results - DynamoDB Hotspot (0.013,0.95) Yearly cost: $527,328 Yearly cost: $106,872.00

12. ■ 50% reads, 50% writes ■ Hotspot distribution ■ Goal Throughput: 100K which is 70% of ScyllaDB max capacity Use Case Comparison Latency Measured Throughput Yearly Cost Mean Read Mean Update P99 Read P99 Update ScyllaDB Cloud 1.93 1.57 4.739 3.593 100.5K 29,172 $ DynamoDB 5.51 6.59 37.695 41.951 100.K (provisioned 120K) 278,172 $

13. DynamoDB with Different Distributions ■ Read / Update ratio: 50% / 50% ■ Provisioned Capacity: 200K ■ Loader Throughput 100K update, 100K read Distribution p99-Update p99-Read Throughput hotspot(0.013,0.95) 18.943 15.743 197,900 uniform 26.383 21.999 186,760 zipfian 499.455 22.159 35,220

14. DynamoDB with Different Distributions ■ Read / Update ratio: 50% / 50% ■ Provisioned Capacity: 200K ■ Target Capacity: 200K

15. DynamoDB Latency Across 3 Distributions distribution write% read% p99-Update p99-Read zipfian 100% 0% 498.43 NA zipfian 90% 10% 498.69 21.07 zipfian 50% 50% 499.46 22.16 zipfian 10% 90% 18.82 280.58 zipfian 0% 100% NA 295.42 uniform 100% 0% 22.11 NA uniform 90% 10% 25.70 22.19 uniform 50% 50% 26.38 22.00 uniform 10% 90% 22.99 19.63 uniform 0% 100% NA 19.22 hotspot(0.013,0.95) 100% 0% 24.08 NA hotspot(0.013,0.95) 90% 10% 23.81 20.10 hotspot(0.013,0.95) 50% 50% 18.94 15.74 hotspot(0.013,0.95) 10% 90% 23.41 19.70 hotspot(0.013,0.95) 0% 100% NA 19.70

16. DynamoDB hard limit: Partition throughput per sec: ■ 3,000 Read Capacity Units (RCUs) ■ 1,000 Write Capacity Units (WCUs) ■ Long (2.5 sec) wait for every fail request Why DynamoDB Might Not Match Provisioned Capacity? More info on DynamoDB limits: https://www.alexdebrie.com/posts/dynamodb-limits/

17. Cost of Reserved Capacity per Table

18. DynamoDB On Demand vs Provisioned

19. DynamoDB On Demand vs Provisioned 9 tables, each with 20K sustained capacity

20. DynamoDB On Demand vs Provisioned - Max capacity for *all* tables 9 * 100K = 900K ops: 1,299,423$ yearly cost - Base 20K with 100K peak (72h) 509,311$ yearly cost (assuming fast enough scaling) https://calculator.aws/#/estimate?id=5450b405778770d0816f2430eec10ef19523301c 9 tables, each with 20K sustained capacity And 100K peak!

21. DynamoDB On Demand vs Provisioned 9 tables, each with 20K sustained capacity And 100K peak! - 250K ops is more than enough to support bursts - $ 58,344 yearly on Scylla Cloud. x22 less expensive than Dynamo! https://price-calc.gh.scylladb.com/?writes=125000&reads=125000&storage=1&itemSize=1 &replication=3

22. ScyllaDB Alternator

23. Two Swords ScyllaDB! Amazon DynamoDB Compatible API + Production Ready + Protocol / Driver level compatibility + Drop and drop-in replacement + Available on K8s + REST / HTTP(S) Apache Cassandra Compatible API + Production Ready + Protocol / Driver level compatibility + Drop and replace + Available on K8s + Binary Protocol (CQL) 23

24. What is Alternator? ■ ScyllaDB started with Cassandra APIs (CQL). ■ Alternator: a DynamoDB-compatible API. ■ Alternator is part of ScyllaDB - not separate nodes or executable. ■ Run everywhere ■ Run locally ■ Run on any Cloud

25. Alternator Timeline September 2019 First open-source release April 2020 Open-source GA (ScyllaDB 4.0) June 2020 ScyllaDB Cloud July 2020 First paying customer August 2020 ScyllaDB Enterprise (2020.1) September 2020 Jan 2023 AWS Outposts ScyllaDB Enterprise (2022.2)

26. Alternator Customer Use Case AWS i3 instances 3 clusters - 18 nodes in total Peak query per second: around 400k/s Data volume ~8.5TB Estimated cost on ■ DynamoDB: 90K + 12*73,538 = 972K per year (reserved) ■ ScyllaDB Cloud: 12 * 17,019 = 204K per year (reserved)

27. Demo!

28. AWS CLI export SCYLLA_CLOUD='http://3.217.45.196:8000' aws --region=us-east-1 --endpoint-url $SCYLLA_CLOUD dynamodb delete-table --table-name MusicCollection aws --region=us-east-1 --endpoint-url $SCYLLA_CLOUD dynamodb create-table --table-name MusicCollection --attribute-definitions AttributeName=Artist,AttributeType=S AttributeName=SongTitle,AttributeType=S --key-schema AttributeName=Artist,KeyType=HASH AttributeName=SongTitle,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 aws --endpoint-url $SCYLLA_CLOUD dynamodb put-item --table-name MusicCollection --item '{"Artist": {"S": "No One You Know"}, "SongTitle": {"S": "Call Me Today"}, "AlbumTitle": {"S": "Somewhat Famous"}, "Awards": {"N": "1"}, "ttl": {"N": "'$(($(date +%s)+30))'"}}' aws --endpoint-url $SCYLLA_CLOUD dynamodb scan --table-name MusicCollection aws --region=us-east-1 --endpoint-url $SCYLLA_CLOUD dynamodb delete-table --table-name MusicCollection

29. Alternator Dashboard

30. State Of The Alternator

31. DynamoDB API Compatibility ■ Table Operations (Create, Delete, Describe, Delete…) ■ Batch operations ■ Item Operations (Get, Put, Delete…) ■ Query ■ Scan ■ Local Secondary Indexes ■ Global Secondary Indexes ■ TimeToLive ■ Data Types (String, Boolean, Date, Integer …)

32. Roadmap Features ■ Point in time Backup (backup is supported via ScyllaDB Manager) ■ Export and import to/from S3 ■ Pay per operation ■ Multi Item Transactions ■ DAX caching - not required with ScyllaDB superior performance

33. Trying Alternator To try Alternator quickly, you can: ■ Create an Alternator cluster on ScyllaDB Cloud ■ Run one Alternator node on your machine in 5 minutes, using docker: docker pull scylladb/scylla:latest docker run --name scylla -d -p 8000:8000 scylladb/scylla:latest --alternator-port=8000 --alternator-write-isolation=always

34. Migrating from Amazon DynamoDB to ScyllaDB

35. Write to DynamoDB Time Read from DynamoDB Live Migration 35 Updates from DynamoDB Stream Enable Streams Migrate Schema

36. Write to DynamoDB Migrate Schema Time Read from DynamoDB Write to ScyllaDB Live Migration 36 Updates from DynamoDB Stream

37. Write to DynamoDB Forklifting Existing Data DBs in Sync Time Read from DynamoDB Write to ScyllaDB Live Migration 37 Updates from DynamoDB Stream

38. Dual Reads Write to DynamoDB Forklifting Existing Data Validation DBs in Sync Time Read from ScyllaDB Read from DynamoDB Write to ScyllaDB Live Migration 38 Updates from DynamoDB Stream

39. Dual Reads Write to DynamoDB Forklifting Existing Data Validation DBs in Sync Time Read from ScyllaDB Fade off DynamoDB Read from DynamoDB Write to ScyllaDB Live Migration 39 Updates from DynamoDB Stream

40. ■ Highly resilient to failures ■ Access compatible Databases using a native connector ■ High performance parallelized reads and writes ■ Support transformations ■ Open Source! SQL NoSQL ScyllaDB Migrator 40

41. Streaming Migration Source https://www.scylladb.com/2020/09/02/one-step-streaming-migration-from-dynamodb-into-scylla/

42. Summary ■ ScyllaDB is compatible with Amazon DynamoDB and can run on multiple platforms ■ ScyllaDB is less expensive and has lower tail latency than Amazon DynamoDB ■ You can migrate from Amazon DynamoDB to ScyllaDB without downtime

43. Thank You Stay in Touch Tzach Livyatan tzach@scylladb.com @TzachL @tzach

Use ScyllaDB Alternator to Use Amazon DynamoDB API, Everywhere, Better, More Affordable, All at Once

Vous êtes en train de lire un aperçu.

Consultez-les par la suite

Use ScyllaDB Alternator to Use Amazon DynamoDB API, Everywhere, Better, More Affordable, All at Once

Plus De Contenu Connexe

Similaire à Use ScyllaDB Alternator to Use Amazon DynamoDB API, Everywhere, Better, More Affordable, All at Once (20)

Plus par ScyllaDB (20)

Plus récents (20)