Building & Managing 80+ Cloudflare Points of Presence Around the World
•
0 j'aime•613 vues
21 - IDNOG03 - Jimmy Halim (Cloudflare) - Brief Introduction of CloudFlare, the routing and benefit to Internet, plus network perpective
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (17)
En vedette
En vedette (20)
Similaire à Building & Managing 80+ Cloudflare Points of Presence Around the World
Similaire à Building & Managing 80+ Cloudflare Points of Presence Around the World (20)
Plus de Indonesia Network Operators Group
Plus de Indonesia Network Operators Group (20)
Dernier
Dernier (20)
Building & Managing 80+ Cloudflare Points of Presence Around the World
- 1. Jimmy Halim IDNOG3 jhalim@cloudflare.com Jakarta, 28 July 2016 Building & Managing 80+ PoPs
- 3. ● 4+ million zones/domains ● 43+ billion DNS queries/day ● How? ○ Orange cloud ○ Global distributed network in 80+ locations Still growing fast! ○ Anycast routing Protect and accelerate any website online
- 4. Benefit of orange cloud ● Direct visitors to the nearest entry point ○ Fast! ■ Lesser hops ■ Reduced latency ■ Improved performance ● Save bandwidth! ○ Lesser requests to origin ■ Typically 50% of the resources on any given web page are cacheable ○ Mitigate malicious visitors or DDoS ■ Stop them before get to the origin web server ● Resiliency ○ 80+ locations!
- 5. Grey cloud vs orange cloud
- 7. 1 new PoP per week!
- 8. Strategic Planning ● Agreement/Negotiation ● Location ○ Peering Exchanges ○ Cost ○ Support ● Size ○ Traffic analysis ■ Number of Racks ■ Equipment types ■ Transits/Peering Exchanges ● How many? ● How big are the pipes?
- 9. Challenges ● Installation ○ Regulation ■ Import policy ○ Transits ■ Different carriers have different setup/policies ○ Language barriers ● Human factors ○ Configuration errors! ■ Anycast ● Traffic turnup ○ How to ensure it is not impacting ■ No outages please!
- 10. Solutions ● Out of band network is a must! ○ Acting as last resort ○ Upgrade/downgrade ○ Maintenances ● Configuration template ○ Auto configuration ■ Anycast! ○ Peer review ● Global Network Engineering ○ Round the clock deployment ■ Reduced bottleneck
- 11. Testing with providers ● Circuit testing ○ Point to point extended ping test ■ Test all physical ports ○ Failover Testing ■ Redundancy ● Do not create a blackhole instead! ● Use testing prefix ○ Global versus domestic ■ RIPE Atlas measurement ■ Public route servers ○ Good related BGP configuration ■ It does what is supposed to do
- 12. Traffic Turnup ● Do not send all prefixes at 1 go! ○ Start with few prefixes ○ Check the routing to these few prefixes ■ Global traffic analysis ● No big drop of traffic in other location ● Traffic comes from the right countries ○ Monitor for 24 hours ■ Confirms there are no anomalies observed ● On the new location ● Globally ○ Announce all prefixes ■ In batches ■ Repeat the same steps above!
- 13. Traffic Turnup ● Get the providers to be involved ○ Especially if it is a single homed ○ Inform them the schedule ■ Get them to understand what to expect ■ Troubleshoot and fix the problem faster! ○ Their users might be able to see problem faster
- 15. ● 80+ locations ● 500+ transit/exchange ports ● 500+ network equipments ● Uncountable alerts! Challenges
- 16. Building Resilience Network ● Stable hardware and software ● Automatic configuration template/peer review ● Solid monitoring system ● Network automation ● Global network engineering
- 17. Hardware and Software ● Proper evaluation and testing ○ Fits requirement ○ Bugs free ○ Scalable ● Global standardization ○ Same models of hardware ○ Same software codes ● No mass software upgrade! ○ Small PoP first ○ Deploy in batches
- 18. Solid Monitoring System ● Reduced unwanted alerts ○ Only gets relevant alerts ○ Silence PoP/ports during maintenances ● Monitor the performance of transit providers ○ Detects packet loss on their backbone ○ Provides automatic related traceroutes ○ Actions based on severity ■ Disabling the PoP automatically ■ Disabling traffic on related transit provider automatically ■ Suggests on actions to do
- 19. Alerts Channel and Dashboard
- 20. Alerts Channel and Dashboard
- 21. Alerts Channel and Dashboard
- 22. Alerts Channel and Dashboard
- 23. Network Automation ● Open source recipe: napalm-salt ripe72-NetworkAutomation-SaltandNAPALM-MirceaUlinic-CloudFlare
- 24. Network Automation | NAPALM-Salt (examples) ● salt "edge*" net.cli "show version" ● salt -G "os:junos" net.cli "show chassis hardware” ● salt -G "os:iosxr" net.arp ● salt-run net.find [target_device] ● salt-run net.find [mac_address] ● salt-run bgp.neighbors [bgp_asn] ● salt [target_device] [anycast.disable | anycast.enable] ● salt [target_device] [transit.disable | transit.enable] [transit_name]
- 25. Network Automation | NAPALM-Salt (examples)
- 26. Global Network Engineering ● Follow the sun approach ○ San Francisco -> Singapore -> London -> San Francisco ● Doing all stuffs ○ Technical operations ○ Network engineering ○ Network expansion projects ○ New PoPs deployment ○ Peering stuffs ● Very fast response to network issues and escalation
- 27. Statistics
- 31. Q&A