Old version: [233]대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing
•
8 j'aime•1,407 vues
그림이 정상 출력되는 다음 링크의 자료를 확인해 주세요. https://www.slideshare.net/deview/233-network-load-balancing-maglev-hashing-scheduler-in-ipvs-linux-kernel
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Old version: [233]대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing
Similaire à Old version: [233]대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing (20)
Plus de NAVER D2
Plus de NAVER D2 (20)
Dernier
Dernier (20)
Old version: [233]대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing
- 4. Container Cluster(s) Container Cluster(s) Container Cluster(s) Container Host Container Host Container Host
- 6. L4-LB L4-LB L4-LB L4-LB VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N VIP: 10.0.0.1:80 Destination IP 1 Destination IP 2 Destination IP N
- 7. Container Cluster(s) Container Host Container Host Container Host restart restart restart stop stop stop start start start
- 8. L4-LB L4-LB L4-LB L4-LB VIP: 10.0.0.1:80 DIP 1 ... N VIP: 10.0.0.1:80 DIP 1 ... N VIP: 10.0.0.1:80 DIP 1 ... N VIP: 10.0.0.1:80 DIP 1 ... N Container Host Container Host Container Host restart restart restart stop stop stop start start start
- 10. Container Host Container Host Container Host Container Host L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB
- 12. Container Host Container Host Container Host Container Host L4-LB
- 15. L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB Host: 1.1.1.1 VIP: 10.0.0.1 Host: 1.1.1.2 VIP: 10.0.0.1 Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.4 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1 Host: 1.1.1.6 VIP: 10.0.0.1
- 16. Host: 1.1.1.1 IP Advertise L4-LB L4-LB L4-LB L4-LB L4-LB L4-LB VIP: 10.0.0.1 Host: 1.1.1.2 VIP: 10.0.0.1 Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.4 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1 Host: 1.1.1.6 VIP: 10.0.0.1
- 17. PATH= Hashing( IP.Src, IP.Dst, IP.Protocol, Port.Src, Port.Dst ) Mod N L4-LB L4-LB Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1
- 19. PATH= Hashing() Mod ( N - 1 ) L4-LBL4-LBL4-LB L4-LB Host: 1.1.1.1 VIP: 10.0.0.1 Host: 1.1.1.3 VIP: 10.0.0.1 Host: 1.1.1.5 VIP: 10.0.0.1 Host: 1.1.1.6 VIP: 10.0.0.1
- 20. 192.68.0.2 Container[0] Container[1] Container[2] Container[3] L4-LB 1. Connection Established With Container[2] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 4. Container[0] has no idea about connection with container[2] 5. Container[0] Send RST. Connection Closed
- 21. 192.68.0.3 Container[0] Container[1] Container[2] Container[3] L4-LB L4-LB 1. Connection Established With Container[3] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 although the serving LB is alive. 4. Container[1] has no idea about connection with container[3] 5. Container[1] Send RST. Connection Closed L4-LB
- 23. Container Host Container Host Container Host Container Host L4-LB
- 27. 192.68.0.1 Hashing(IP) % 2 Container[0] Container[1] Hashing(IP) % 2 Hashing(IP) % 2 Hashing(IP) % 2 192.68.0.2 192.68.0.3 192.68.0.4
- 28. 192.68.0.1 Hashing(IP) % 4 Hashing(IP) % 4 Hashing(IP) % 4 Hashing(IP) % 4 192.68.0.2 192.68.0.3 192.68.0.4 Container[0] Container[1] Container[2] Container[3]
- 32. Consistent Hashing Backend Selection Packet Processing Packet Forwarding
- 33. B0 B1 B2 3 0 3 0 2 4 4 4 5 1 6 6 5 1 0 2 3 1 6 5 2 Backend = 3 Table Size = 7 Permutate Hashing Permutation table ( 3x 7 ) Preference list of Backends Lookup_table ( size =7,empty )
- 34. Population Permutation table ( 3x 7 ) B1 B0 B1 B0 B2 B2 B0 Lookup_table ( size =7 ) B0 B1 B2 3 0 3 0 2 4 4 4 5 1 6 6 5 1 0 2 3 1 6 5 2 [0] [1] [2] [3] [4] [5] [6] Assign backends by preference list
- 35. B1 B0 B1 B0 B2 B2 B0 each backend will receive an almost equal numberof connections. [0] [1] [2] [3] [4] [5] [6] L4-LB L4-LB B0 =3 * connection B1 =2 * connection B2 =2 * connection
- 36. Population Permutation table ( 2x 7 ) B0 B2 3 3 0 4 4 5 1 6 5 0 2 1 6 2 Assign backends by preference list Before B1 B0 B1 B0 B2 B2 B2 B1 is Removed Re-Permutate Hashing Lookup_table ( size =7 ) After B1 -> B0 B0 B1 -> B0 B0 B2 B2 B2 Same Value By hashing Lookup_table ( size =7 )
- 38. Backend SelectionPacket ProcessingPacket Forwarding
- 42. Consistent Hashing Backend Selection: FUNCTION Packet Processing: NETFILTER Packet Forwarding: NAT, DR, IP Tunneling
- 43. Weighted Round Robin Round Robin Source Hashing least connection
- 45. Maglev Hashing Scheduling Round Robin Source Hashing least connection Weighted Round Robin
- 47. ... Lookup_table Size=251 ... Lookup_table Size=131071 Disruption % VS Memory usage
- 50. Container Host Container Host Container Host Container Host W: 10 W: 40 W: 80
- 51. Container Host Container Host Container Host Container Host W: 10 W: 40 -> 0 W: 80
- 52. IPVS 1 VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS 2 VIP: 10.0.0.1:80 MH DIP: 172.16.0.3:80 DIP: 172.16.0.1:80 DIP: 172.16.0.5:80 DIP: 172.16.0.7:80 ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 1 ip_vs_mh_permutate() & ip_vs_mh_populate() ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 2 ip_vs_mh_permutate() & ip_vs_mh_populate() IPVS MH IPVS MH
- 53. 192.68.0.2 IPVS IPVS MH IPVS IPVS MH IPVS IPVS MH Container[0] Container[1] Container[2] Container[3] [2] [2] [2]
- 55. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH Add IP of containers dynamically To IPVS Add IP of containers dynamically To IPVS
- 56. VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS 2 VIP: 10.0.0.1:80 MH DIP: 172.16.0.3:80 DIP: 172.16.0.1:80 DIP: 172.16.0.5:80 DIP: 172.16.0.7:80 K/n IPVS MH
- 57. Container Host Container Host Container Host Container Host L4-LB Only K/n Disruption
- 58. IPVS MH IPVS 1 VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS 2 VIP: 10.0.0.1:80 MH DIP: 172.16.0.1:80 DIP: 172.16.0.2:80 DIP: 172.16.0.3:80 DIP: 172.16.0.10:80 IPVS MH ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 1 ip_vs_mh_Lookup[ ] of 10.0.0.1:80in ipvs 2
- 59. Container Host Container Host Container Host Container Host IPVS No Disruption
- 60. 192.68.0.2 Container[0] Container[1] Container[2] Container[3] IPVS 1. Connection Established With Container[2] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 4. IPVS MH can forward the packet to same destination by hashing. IPVS MH 5. Continue the Established connection.
- 61. 192.68.0.3 Container[0] Container[1] Container[2] Container[3] IPVS 1. Connection Established With Container[3] 2. A Load Balancer Down 3. ECMP Disruption. Forward packet another l4 although the serving LB is alive. IPVS IPVS MH 4. IPVS MH can forward the packet to same destination by hashing With no connection info. 5. Continue the Established connection.
- 62. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH Add IP of containers dynamically To IPVS Add IP of containers dynamically To IPVS
- 63. IPVS MH IPVS MH IPVS MH IPVS MH IPVS MH Add IP of containers dynamically To IPVS Add IP of containers dynamically To IPVS
- 65. Linux Kernel >= 4.18 Choose M in Kernel menuconfig to use IP_VS_MH
- 66. echo 1 > /proc/sys/net/ipv4/vs/sloppy_tcp echo 2 > /proc/sys/net/ipv4/vs/conn_reuse_mode