Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
2021.06.
Distributed Lock
Manager
Hao Chen
MegaEase
企业云化架构解决方案提供商/用技术推动商业进步
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
Outline
What’s topics
• What’s the problem?
• Possible Solutions
• Distribute...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
Consistency
What’s the real problem is?
Order
Ship the Order
Cancel the Order...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
4
Order
(Canceled)
Ship the Order
Cancel the Order
Seller
Buyer
Consistency
W...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
5
Locking
Locking Solutions
Shard Application Centralized Distributed
Lock Ma...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
6
DB Lock
MySQL InnoDB Locking Solutions
SELECT ... LOCK IN SHARE MODE
SELECT...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
7
Shard
HotSpots
indices
Operation
for Order A
Operation
for Order A
Shard Da...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
8
The Problems of Shard
Why the shard is not a good design?
Hotspots Multiple...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
9
Lock Manager
Centralized Lock Manager?
• Distributed Lock Manager (DLM)
• h...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
10
DLM Lock Modes
Six lock modes
Mode Requesting Process Other Processes
Null...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
11
Redlock
Distributed locks with Redis
SET resource_name my_random_value NX ...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
12
How to do distributed locking
https://martin.kleppmann.com/2016/02/08/how-...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
13
Lock Expired
How to deal with dead lock ?
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
14
Lock Expired
Making the lock safe with fencing?
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
15
Is Redlock Safe ?
A technical debates
http://antirez.com/news/101
TL;DR
• ...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
16
Google Chubby
Google Chubby lock service
• Two Components – Master & Clien...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
17
Zookeeper Lock
How to use Zookeeper as lock service
1
2
Create `_locknode/...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
18
Takeaways
Summary
• Concurrent transaction need be synchronized
• DB Lock ...
企业云原生架构解决方案提供商/用技术推动商业进步
MegaEase
企业云原生架构提供商
19
Question
Make a decision
Framework Service
vs
https://megaease.com
Thanks
Vous avez terminé ce document.
Télécharger et lire hors ligne.
Prochain SlideShare
What to Upload to SlideShare
Suivant
Prochain SlideShare
What to Upload to SlideShare
Suivant
Télécharger pour lire hors ligne et voir en mode plein écran

Partager

Distributed Lock Manager

Télécharger pour lire hors ligne

What’s the problem?
Possible Solutions
Distributed Lock Manager
- Redlock
- Google Chubby
- ZooKeeper
Takeaways

Livres associés

Gratuit avec un essai de 30 jours de Scribd

Tout voir

Distributed Lock Manager

  1. 1. 2021.06. Distributed Lock Manager Hao Chen MegaEase 企业云化架构解决方案提供商/用技术推动商业进步
  2. 2. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 Outline What’s topics • What’s the problem? • Possible Solutions • Distributed Lock Manager • Redlock • Google Chubby • ZooKeeper • Takeaways
  3. 3. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 Consistency What’s the real problem is? Order Ship the Order Cancel the Order Seller Buyer How to drive the State Machine? What’s the end state?
  4. 4. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 4 Order (Canceled) Ship the Order Cancel the Order Seller Buyer Consistency Why should we need the lock? Within Locked Transaction Ship the Order Order (Canceled) Waiting For Lock Within Locked Transaction Error Invalid State Order (Shipped) Ship the Order Cancel the Order Seller Buyer Within Locked Transaction Order (Return) Waiting For Lock Within Locked Transaction Cancel the Order Return Process
  5. 5. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 5 Locking Locking Solutions Shard Application Centralized Distributed Lock Manager Database Lock
  6. 6. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 6 DB Lock MySQL InnoDB Locking Solutions SELECT ... LOCK IN SHARE MODE SELECT ... FOR UPDATE Exclusive Lock Shared Lock Versioning Optimistic Lock SELECT id, data, version FROM table WHERE id = $ID UPDATE table SET data = $data version = version + 1 WHERE id = $ID AND version=$VER if {UPDATE} Failed, go to step 1 else Done! 1 3 4 2 { Business Code }
  7. 7. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 7 Shard HotSpots indices Operation for Order A Operation for Order A Shard Data Nodes Operation Gateway Operation Gateway forward f o r w a r d • Using the Shard data Solution • Forward all of the operations to a single node • Consistency can be easy to be guaranteed • Consistent Hash • Gossip the indices Operation for Order A Operation for Order A Operation for Order A forw ard forward fo rw a rd Uber Ringpop https://eng.uber.com/ringpop-open-source-nodejs-library/
  8. 8. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 8 The Problems of Shard Why the shard is not a good design? Hotspots Multiple Entities Transaction Node Failed Load is not balanced Transfer money from account A to account B Data Replication cause more consistent issues R+W > N CAP Theorem
  9. 9. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 9 Lock Manager Centralized Lock Manager? • Distributed Lock Manager (DLM) • https://en.wikipedia.org/wiki/Distributed_lock_manager • Provides a lock-leasing protocol • Transaction Service is stateless • Lock Manager must be designed for HA • No single point failure • The lock state can be persistent. • Dead Lock Detection • Lock expired • Keepalive heart beats DLM Storage Persistence Service lock unlock
  10. 10. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 10 DLM Lock Modes Six lock modes Mode Requesting Process Other Processes Null (NL) No access Read or write access Concurrent Read (CR) Read access only Read or write access Concurrent Write (CW) Read or write access Read or write access Protected Read (PR) Read access only Read access only Protected Write (PW) Read or write access Read access only Exclusive (EX) Read or write access No access
  11. 11. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 11 Redlock Distributed locks with Redis SET resource_name my_random_value NX PX 30000 Not Exist Expired of 30s Unique across all clients Algorithm 1. Client get the current time in milliseconds 2. Client tries to acquire the lock in all the N instances sequentially, using the same key name and random value in all the instances. 3. Successfully acquire the lock - majority Redis nodes accepts && elapsed time < lock validity time. 4. Actual lock validity time = initial validity time - elapsed time 5. If acquire the lock failed, try to unlock all instances https://redis.io/topics/distlock if redis.call(“get”,KEYS[1]) == ARGV[1] then redis.call("del",KEYS[1]) end
  12. 12. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 12 How to do distributed locking https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html Challenge Redlock Using time to solve consensus? 5 Redis nodes (A, B, C, D and E) -- 2 Clients (1, 2) A B C D E 1 2 OK OK OK OK OK OK 1. Node C time jumps à Lock expired 2. Client 1 hangs while Node C reply, and become live after Lock expired 3. Client 1 get very long network delay from Node C “the algorithm makes dangerous assumptions about timing and system clocks” Martin Kleppmann
  13. 13. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 13 Lock Expired How to deal with dead lock ?
  14. 14. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 14 Lock Expired Making the lock safe with fencing?
  15. 15. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 15 Is Redlock Safe ? A technical debates http://antirez.com/news/101 TL;DR • Fencing is great, with this, no need distributed lock! • Clock jump is a big problem, Redlock cannot work correctly under this problem. • For network delay & client hangs, they all will be fine.
  16. 16. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 16 Google Chubby Google Chubby lock service • Two Components – Master & Client SDK • Cluster – typically 5 nodes • Replicas - use Paxos protocol to elect a master and replicate the logs • Failover – Master fails, other replicas run the election • Session & Keeplive • Client request or end a session • Lease Interval and Lease Timeout • Performance is average Source: https://www.slideshare.net/romain_jacotin/the-google-chubby-lock-service-for-looselycoupled-distributed-systems (45s) (12s)
  17. 17. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 17 Zookeeper Lock How to use Zookeeper as lock service 1 2 Create `_locknode/lock-` with Sequence & Ephemeral flag https://zookeeper.apache.org/doc/r3.1.2/recipes.html#sc_recipes_Locks Get children to see to check if I am the lowest sequence number • Yes. Get the Lock • No. Set the watch locknode _c_8a27d9a2-d177-11eb-b8bc-0242ac130003-lock-0000000001 _c_8a27d9a2-d177-11eb-b8bc-0242ac130003-lock-0000000002 Client A Client B
  18. 18. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 18 Takeaways Summary • Concurrent transaction need be synchronized • DB Lock is fine, but the Optimistic Lock is great. • Sharding the data cannot solve the all of problem • Distributed Lock Service need the following features: • High Availability • Data Replicas - strong consistent protocol – Pasox, Raft, zab • Master Failover – Leader election • Deadlock Detection • Keepalive & Lease Timeout
  19. 19. 企业云原生架构解决方案提供商/用技术推动商业进步 MegaEase 企业云原生架构提供商 19 Question Make a decision Framework Service vs
  20. 20. https://megaease.com Thanks
  • YuanChangXu

    Jul. 28, 2021
  • xianlaioy

    Jul. 13, 2021

What’s the problem? Possible Solutions Distributed Lock Manager - Redlock - Google Chubby - ZooKeeper Takeaways

Vues

Nombre de vues

913

Sur Slideshare

0

À partir des intégrations

0

Nombre d'intégrations

1

Actions

Téléchargements

100

Partages

0

Commentaires

0

Mentions J'aime

2

×