This document discusses the problem of bufferbloat on the internet. It explains that bufferbloat occurs when routers and switches have too much buffering capacity, which leads to increased latency. The causes of bufferbloat include increased bandwidth usage from applications, large TCP initial windows, and routers being configured to avoid dropping packets. Solutions presented are queue management techniques like CoDel and PIE that aim to control latency while maintaining high throughput. The document notes that while awareness of bufferbloat has increased, legacy equipment and congestion on backbones continue to cause issues.
TechTAC® CFD Report Summary: A Comparison of Two Types of Tubing Anchor Catchers
Llnw bufferbloat
1. BufferbloatBufferbloat
Why is the Internet slow?Why is the Internet slow?
Stephen HemmingerStephen Hemminger
stephen@networkplumber.orgstephen@networkplumber.org
Limelight Networks May 2015
2.
3. BufferbloatBufferbloat
● What is the problemWhat is the problem
● What causes the problemWhat causes the problem
● SolutionsSolutions
– DemonstrationDemonstration
● Current statusCurrent status
– The GoodThe Good
– The BadThe Bad
– The UglyThe Ugly
8. Queuing Theory BasicsQueuing Theory Basics
averagetimeinqueue
utilization
servicerate
1 utilization
Graphic courtesy Sprint, Apricot 2004
From Fred Baker: Bufferbloat!
13. Blame LinuxBlame Linux
● Windows XPWindows XP
– Maximum window 64kMaximum window 64k
● Windows 7Windows 7
– Bandwidth limit to 80 mbitsBandwidth limit to 80 mbits
● AndroidAndroid
– Receive window limitedReceive window limited
14. Blame the customerBlame the customer
● Customers callCustomers call
supportsupport
● Applications areApplications are
using moreusing more
bandwidthbandwidth
● Block and chargeBlock and charge
22. Ideal Active Queue ManagementIdeal Active Queue Management
● FairFair
– All flows get some bandwidthAll flows get some bandwidth
● SimpleSimple
– No tuningNo tuning
● Easy to deployEasy to deploy
– No special hardware, no protocol changesNo special hardware, no protocol changes
● ReasonableReasonable
– Won't create multi-second latencyWon't create multi-second latency
36. Bufferbloat: The GoodBufferbloat: The Good
● Industry awarenessIndustry awareness
● Standards progressStandards progress
● Active researchActive research
● Open Source SolutionsOpen Source Solutions
37. Once you start looking,Once you start looking,
Bufferbloat can be everywhereBufferbloat can be everywhere
● EdgeEdge
– Home routerHome router
● ProviderProvider
● NetworkNetwork
● AcceleratorsAccelerators
– CachesCaches
– Load balancers,...Load balancers,...
● ServersServers
38. How is Bufferbloat Solved on Linux?How is Bufferbloat Solved on Linux?
● Queuing disciplinesQueuing disciplines
– Codel, PIE, ...Codel, PIE, ...
● Linux internalLinux internal
– BQL, FQ, PacingBQL, FQ, Pacing
● Enable good defaultsEnable good defaults
– ecnecn
– fq_codelfq_codel
39. Bufferbloat: the BadBufferbloat: the Bad
● Legacy equipmentLegacy equipment
– OEM focus on cost not AQMOEM focus on cost not AQM
● Congestion on backboneCongestion on backbone
– It's not a bug it's a feature?It's not a bug it's a feature?
– Problem is political not technicalProblem is political not technical
40. Bufferbloat: the UglyBufferbloat: the Ugly
● WirelessWireless
– AggregationAggregation
– MulticastMulticast
– Bad benchmarksBad benchmarks
– Proprietary firmwareProprietary firmware
43. Bufferbloat resources
Bufferbloat.net: http://bufferbloat.net
Email Lists: http://lists.bufferbloat.net
CeroWrt: http://www.bufferbloat.net/projects/cerowrt
Other talks: http://mirrors.bufferbloat.net/Talks
Jim Gettys Blog – http://gettys.wordpress.com
A big thanks to the bloat mailing list, Jim, Kathie, Van, Dave,
Eric, ISC, and all the other the CeroWrt/OpenWrt contributors
Editor's Notes
Bufferbloat has been known about in the academic community but was not generally measured because many tests only measured one dimension
150ms – VOIP
20ms – Desktop lag
100ms – DNS lookup fail
ARP, ND – Can't find machine
.5sec – User's give up
2 sec–scp copy fails
DHCP – Machine falls off network
Bufferbloat:
The sometimes extreme delays in the Internet caused by excess queuing.
As the utilization goes to 100% the delay goes to infinity.
Green = T1
Blue = 10Mbit
<number>
Effective Window: the amount of data TCP sends each RTT
Knee: the lowest window that makes throughput approximate capacity
Cliff: the largest window that makes throughput approximate capacity
Note that throughput is the same at knee and cliff. Increasing the window merely increases RTT, by increasing queue depth