Acidic clusters - Review of contemporary ACID-compliant databases with synchronous replication - Fossetcon 2014
•
0 likes•760 views
This talk reviews database clusters of our time which employ synchronous replication while being ACID compliant. ACID compliance implies ability to support transactions across nodes. As part of this talk, PXC (Percona XtraDB Cluster)/Galera, Google F1 based on Spanner/CFS and MySQL Cluster will be considered. Primary objective here is to expound features of each in order to highlight differentiating factors and commonality between them.
Recommended
Recommended
More Related Content
Similar to Acidic clusters - Review of contemporary ACID-compliant databases with synchronous replication - Fossetcon 2014
Similar to Acidic clusters - Review of contemporary ACID-compliant databases with synchronous replication - Fossetcon 2014 (20)
More from Raghavendra Prabhu
More from Raghavendra Prabhu (19)
Recently uploaded
Recently uploaded (20)
Acidic clusters - Review of contemporary ACID-compliant databases with synchronous replication - Fossetcon 2014
- 1. ACIDic Clusters Review of contemporary ACID-compliant databases with synchronous replication Fossetcon 2014 Raghavendra Prabhu raghavendra.prabhu@percona.com Percona 12 September, 2014 Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 1 / 42
- 2. Outline 1 Introduction Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 1 / 42
- 3. Outline 1 Introduction 2 Review Architecture Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 1 / 42
- 4. Outline 1 Introduction 2 Review Architecture 3 Epilogue Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 1 / 42
- 5. Introduction Seed quotes.. “’Network is reliable’ - a fallacy of the distributed system.” “A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable.” - Leslie Lamport “Those who would give up essential correctness, to purchase a little temporary scalability, deserve neither correctness nor scalability.” - Leif Walsh “A given row can’t be modified more than once per RTT." - Alexey Yurchenko Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 2 / 42
- 7. Introduction Introduction I A twist on CAP Pick any Two? ACID SQL Synchronous replication Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 4 / 42
- 8. Introduction Introduction I A twist on CAP Pick any Two? ACID SQL Synchronous replication I Solution: Don’t have to pick! Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 4 / 42
- 9. Introduction Introduction I A twist on CAP Pick any Two? ACID SQL Synchronous replication I Solution: Don’t have to pick! I CAP and latency - eventual consistency I Lambda architecture Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 4 / 42
- 11. Introduction RAS I Why Synchronous Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
- 12. Introduction RAS I Why Synchronous Symmetry ? Easier to manage and conceive ? Build once, deploy everywhere! Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
- 13. Introduction RAS I Why Synchronous Symmetry ? Easier to manage and conceive ? Build once, deploy everywhere! Latency! I Do we need ACID? Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
- 14. Introduction RAS I Why Synchronous Symmetry ? Easier to manage and conceive ? Build once, deploy everywhere! Latency! I Do we need ACID? Depends! Concurrent workload Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
- 15. Introduction RAS I Why Synchronous Symmetry ? Easier to manage and conceive ? Build once, deploy everywhere! Latency! I Do we need ACID? Depends! Concurrent workload I Relational database Can it be a KV/document store? Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
- 16. Introduction RAS I Why Synchronous Symmetry ? Easier to manage and conceive ? Build once, deploy everywhere! Latency! I Do we need ACID? Depends! Concurrent workload I Relational database Can it be a KV/document store? Everyone wants SQL though. Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
- 17. Introduction RAS I Why Synchronous Symmetry ? Easier to manage and conceive ? Build once, deploy everywhere! Latency! I Do we need ACID? Depends! Concurrent workload I Relational database Can it be a KV/document store? Everyone wants SQL though. Beats every other API! Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 6 / 42
- 18. Review Contemporary systems I PXC/Galera I NDB Cluster I Google F1 I Others - FoundationDB, CockroachDB I Not talking of comparisons/benchmarks. I Apples and Pineapples! (Since they are all clusters) Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 7 / 42
- 19. Review Family of Synchronous Systems I Transactional replication Overhead and workarounds One-copy equivalence NDB: 2 PC I Virtual Synchrony Extended Virtual Synchrony: Galera I Multi-phase Paxos: Google F1 Also 2PC Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 8 / 42
- 20. Review Architecture Layers I Necessity Strengths Degrees of freedom I Monolithic v/s Layered Monolithic - Galera Layered - F1/Spanner, NDB(?) I Implications Failures Debugging Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 9 / 42
- 22. Review Architecture Storage I Unbundling of translation and data I Different strategies Spanner ? CFS underneath ? Provides guarantees in replication Hierarchical storage NDB ? Separatation of data and management nodes ? Hybrid storage Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 11 / 42
- 23. Review Architecture Storage I WSREP Plugin system for Galera Provider ? InnoDB ? More tightly bound ? In-memory and ring-buffers Transaction awareness Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 12 / 42
- 24. Review Architecture Storage I Statelessness Virtual Synchrony ? Causality Loose/Tight Binding Consequences ? Co-ordination issue ? Flexibility ? Transitivity ? Uncoupled relation Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 13 / 42
- 25. Review Architecture Relational Database I NoSQL / NewSQL / OldSQL New wine in old bottle? Approach towards ACIDity. ? Pros and Cons ? The Fit Does NoSQL meet RAS ? Eventual (in)Consistency ? Strict definitions ? Instrumentation Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 14 / 42
- 27. Review Architecture Relational Database I Role of API Presence of Layer Nature of storage API defines data model? I NoSQL access to SQL I ORM Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 16 / 42
- 29. Review Architecture Degree of Synchronicity I Latency is a killer! Amortization I Synchronicity imply Causality? I Commutative writesets I Role of Applications/Architecture Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 18 / 42
- 30. Review Architecture ACIDity I ACIDity test? I BASE and NoSQL I MVCC and Synchronous Replication Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 19 / 42
- 32. Review Architecture ACIDity I Does ACID require synchronous OR I Does Synchronous demand ACIDity? Reconciliation requires it. ? Expensive without ? Rollbacks ACID possible without Relational semantics? Thought: Filesystems with replication Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 21 / 42
- 34. Review Architecture Locking I Optimistic Concurrency Reduce communication ? WWW/HTTP Software Transactional Memory ? Lock elision ? Natural to Databases Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 23 / 42
- 35. Review Architecture Locking I Conflicts and Deadlocks Limiting factor: Amdahl’s Scale: number of nodes and size of transactions = Quadratic or Quinary = Not strictly bound = Only a strict lower bound Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 24 / 42
- 36. Review Architecture Locking I Pessimistic Locking Performance Network Overhead ? More roundtrips ? Pipeline/Batching I Hybrid approach Switching I Granularity of Locking Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 25 / 42
- 38. Review Architecture Locking I Externalities GPS clock - Google F1 Controlled conditions Limitations Lock Managers: Zookeeper, Chubby I Lamport timestamps! A Zeitgeber! Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 27 / 42
- 39. Review Architecture Integration I Scaling the system upwards I Composable Transactions I XA Support I Extensibility Adding other storage engines: PSA I Geographic Replication Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 28 / 42
- 40. Review Architecture Sharding I Systems that support it I Issues of Quorum I Adding support externally Spider Engine Cluster of clusters MySQL Fabric? Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 29 / 42
- 41. Review Architecture Transition I From single node to a cluster I Idempotency of transactions I Integration with Async replication I Loss of key parts Degree of ACIDity Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 30 / 42
- 42. Review Architecture Operational usage I Resource Manager Moving parts Reduces the confusion I Backups Impact on cluster operations Logical and Physical backups NoSQL interface Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 31 / 42
- 43. Review Architecture Operational usage I Integration with other HA Load balancers and Proxies Fencing / STONITH Resource Managers ? Pacemaker Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 32 / 42
- 45. Review Architecture Eventual Consistency “not consistent right now, maybe come back later and things will be better” “Full transactional consistency is one of the most important properties of F1.” Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 34 / 42
- 46. Review Architecture EC: Qualitative Analysis I Active analysis System-centric Convergence time I Passive analysis Distributed tracing Dangling reads Client-centric Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 35 / 42
- 47. Review Architecture EC: Qualitative Analysis I Guarantees Latency and staleness Monotonic reads Causality k|delta - atomicity Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 36 / 42
- 49. Epilogue Limitations I None! Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 38 / 42
- 50. Epilogue Limitations I None! I Silver bullet? I Where you may want Async / delay I Hot updates/inserts I Schema changes I Compromise on strictness/features/performance Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 38 / 42
- 51. Epilogue Further Reading I Is this a solved problem? Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 39 / 42
- 52. Epilogue Further Reading I Is this a solved problem? May be not. I The Dangers of Replication and a Solution I Replication Using Group Communication Over a Partitioned Network I Transaction Processing: Concepts and Techniques I F1: A Distributed SQL Database That Scales I Eventually Consistent: Not What You Were Expecting? I The Layer Concept I Don’t Settle for Eventual Consistency I Lambda Architecture Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 39 / 42
- 53. Epilogue Image Credits I http://upload.wikimedia.org/wikipedia/commons/e/e8/Latency_map_ world.png I http://blog.wikimedia.org/2014/07/09/ how-ripe-atlas-helped-wikipedia-users/ I https://secure.flickr.com/photos/alexbrn/5584251627 I https://secure.flickr.com/photos/russmorris/407778776 I https://secure.flickr.com/photos/carlosluzz/561920999 I https://secure.flickr.com/photos/eiriknewth/282268782 I https://secure.flickr.com/photos/shaireproductions/5509387767 I https://secure.flickr.com/photos/grantmac/3049823203 I http://guide.couchdb.org/draft/consistency/01.png I http://www.yeeach.com/post/583 Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 40 / 42
- 54. Epilogue Summary I Questions I Theories Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 41 / 42
- 55. Epilogue About I /me: Raghavendra Prabhu, Product Lead, Percona XtraDB Cluster, Percona. I Slides will be at http://www.slideshare.net/slidunder I Twitter: randomsurfer I LinkedIn: rdprabhu I Github: ronin13 I Presentation under CC BY-SA 4.0 Raghavendra Prabhu (Percona) ACIDic Clusters 12 September, 2014 42 / 42