Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909

1. Disaster Recovery in distant DC; people, hardware, latency.

2. • What is really the Allegro.pl? • Distaster Recovery - true story • Latency - why 20 ms matters • Data Centers topology agenda

3. Allegro ... QXL ? Serwisy Aukcyjne Serwisy Ogłoszeniowe Płatności .pl .cz .ro .ua.bg .hu .sk .lt .pl .pl Sklepy Internetowe

4. Allegro... beginings

5. o Allegro... today

6. o Allegro... today about 1000 servers

7. o Allegro... today about 1000 servers about 300 IT staff

8. o Allegro... today about 1000 servers about 300 IT staff about 40 in infrastructure

9. we are growing Source: http://www.aukcjostat.pl/

10. why bother?

11. Qvo vadis?

12. Qvo vadis?

13. Qvo vadis?

14. Qvo vadis?

15. Qvo vadis?

16. Qvo vadis?

17. 2 hours by plane... ~800 km Go west...

18. 2 hours by plane... ~800 km Go west...

19. WWCKD ?

20. WWCKD ? Hope is the mother of lost battles No guts, no glory. No pain, no game. No risk, no fun.

21. WWCKD ? Hope is the mother of lost battles No guts, no glory. No pain, no game. tiaaaa... No risk, no fun.

22. Building DC3

23. Building DC3

24. DC net topology

25. DC net topology ISP 1 ISP 2 DC2 , Poznań

26. DC net topology ISP 1 ISP 2 ISP 1 ISP 2 DC2 , Poznań DC3 , Frankfurt

27. DC net topology ISP 1 ISP 2 ISP 1 ISP 2 DC2 , Poznań DC3 , Frankfurt 2G, EoMPLS

28. DC net topology ISP 1 ISP 2 ISP 1 ISP 2 DC2 , Poznań DC3 , Frankfurt 2G, EoMPLS 10G, DWDM

29. Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS DNS $ host -t ns allegro.pl allegro.pl name server gtmdc3.allegro.pl. allegro.pl name server gtmdc2.allegro.pl.

30. Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl

31. Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl

32. Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request

39. Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS response

40. Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS req: www.allegro.pl HTTP trafﬁc

41. Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl

51. Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS req: www.allegro.pl HTTP trafﬁc

52. Latency vs tennis

53. SYN SYN/ACK ACK TCP/IP explained FIN ACK ACK DATA DATA ACK Control packets Data packets bandwidth RTT / latency window size = 64 kBytes Bandwidth-delay Product BDP (bytes) = bandwidth (KBytes/sec) x RTT (ms)

54. 0,0005sec SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,0005sec 0,0005sec ACK DATA DATA ACK 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec Control packets Data packets

55. 0,0005sec SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,0005sec 0,0005sec ACK DATA DATA ACK 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec Control packets Data packets Scenario #1 • bandwidth = 10Gbit/sec • RTT / latency = 1 ms (1/1000 sec) • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B?

56. 0,0005sec SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,0005sec 0,0005sec ACK DATA DATA ACK 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec Control packets Data packets Scenario #1 • bandwidth = 10Gbit/sec • RTT / latency = 1 ms (1/1000 sec) • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B? 1 sec !!! (640 000 kBytes) / (64 kBytes/ 1ms) = 1000 ms

57. 0,5sek SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,5sek 0,5sek ACK DATA DATA ACK 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek Control packets Data packets

58. 0,5sek SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,5sek 0,5sek ACK DATA DATA ACK 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek Control packets Data packets Scenario #2 • bandwidth = 10Gbit/sec • RTT / latency = 1 sek • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B?

59. 0,5sek SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,5sek 0,5sek ACK DATA DATA ACK 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek Control packets Data packets Scenario #2 • bandwidth = 10Gbit/sec • RTT / latency = 1 sek • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B? 1000 sec !!! (640 000 kBytes) / (64 kBytes/1000ms) = 1000 sec

60. 800km distance...

61. 800km distance... storage data base application / transactions AIX Oracle MySQLbackend daemons ﬁle transfers server deployment Linux

62. 800km distance... just 20ms...

63. co-operation

64. co-operation developers

65. co-operation developers infrastructure

68. Recipe for success Disaster recovery ingredients

69. Recipe for success

73. Thank You, Q&A

Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (9)

Similar to Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909

Similar to Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909 (20)

Recently uploaded

Recently uploaded (20)

Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909