Innovation is back is the transport and network layers

1. Future Internet protocols
Innovation is back in the transport and network layers
Olivier Bonaventure
https://inl.info.ucl.ac.be
December 2018

2. Agenda
• Today's Internet
• Transport Layer : evolution or revolution ?
– Multipath TCP
– QUIC
• Network Layer evolution
– IPv6 Segment Routing

3. The Protocol Hourglasss
Steve Deering, IETF 51, London, August 2001

4. Innovation in the protocol hourglass
• Higher layers
– Lots of innovation in the applications even if HTTP and
TLS play a very important role
• Lower layers
– Lots of innovation with new wireless and fixed local
area network technologies
• Network and transport layers
– It took us 20 years to deploy IPv6
– SCTP, DCCP, HIP, shim6, did not get enough traction

5. Why is innovation so difficult
in the network layer ?
• The Internet is already too large
• Any change in the network layer involves a too
large number of stakeholders
• Why was IPv6 eventually deployed ?
Source http://www.potaroo.net/ispcol/2018-01/addr2017.html

6. Why is innovation so difficult
in the transport layer ?
• Application developers will use a new
transport protocol if it gives benefits and has a
large installed based
• Kernel developers will implement and deploy
a new transport if it gives benefits and is
requested by many application developers

7. But there is another hurdle …
• Internet architecture as seen by students
Physical
Datalink
Network
Transport
Application
Physical
Physical
Datalink
Physical
Datalink
Network
O. Bonaventure, Computer networking : Principles, Protocols and Practice, open ebook, http://cnp3book.info.ucl.ac.be

8. The end-to-end principle
Physical
Datalink
Network
Transport
Application
Physical
Datalink
Network
Transport
Application
Physical
Datalink
Network
Physical
Datalink
TCP

9. In reality
– almost as many middleboxes as routers
– various types of middleboxes are deployed
Sherry, Justine, et al. "Making middleboxes someone else's problem: Network processing as a cloud service."
Proceedings of the ACM SIGCOMM 2012 conference. ACM, 2012.

10. A middlebox zoo
http://www.cisco.com/web/about/ac50/ac47/2.html
Web Security
Appliance
NAC Appliance
ACE XML
Gateway
Streamer
VPN Concentrator
SSL
Terminator
Cisco IOS Firewall
IP Telephony
Router
PIX Firewall
Right and Left
Voice
GatewayVVVV
Content
Engine
NAT

11. How to model those middleboxes ?
• In the official architecture, they do not exist
• In reality...
Physical
Datalink
Network
Transport
Application
Physical
Datalink
Network
Transport
Application
Physical
Datalink
Network
TCP
Physical
Datalink
Network
Transport
Application

12. TCP segments processed by a router
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
IP
TCP

13. TCP segments processed by a NAT
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options

14. Problems caused by middleboxes
• Most middleboxes make assumptions about
the format of protocol headers
– Headers changes are almost impossible
• Many middleboxes modify some fields of the
network and/or transport header
– A field set by the client may not reach the server
• Many middleboxes restrict the utilisation of
protocol extensions that they do not support
– Firewalls are the classical example

15. End-to-end transparency today
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Source port Destination port
Checksum Urgent pointer
THL Reserved Flags
Acknowledgment number
Sequence number
Window
Ver IHL ToS Total length
ChecksumTTL Protocol
Flags Frag. Offset
Source IP address
Identification
Destination IP address
Payload
Options
Middleboxes don't change
the Protocol field, but
many discard packets with an
unknown Protocol field

16. Agenda
• Today's Internet
• Transport Layer : evolution or revolution ?
– Multipath TCP
– QUIC
• Network Layer evolution
– IPv6 Segment Routing

17. Design objectives
• Multipath TCP is an evolution of TCP
• Design objectives
– Support unmodified applications
– Work over today’s networks (IPv4 and IPv6)
– Works in all networks where regular TCP works

18. A naïve Multipath TCP
SYN+ACK+Option
ACK
seq=123, "abc"
seq=126, "def"
SYN+Option

19. A naïve Multipath TCP
In today's Internet ?
SYN+Option
SYN+ACK+Option
ACK
seq=123, "abc"
seq=126, "def"
There is no
corresponding
TCP connection

20. Design decision
– A Multipath TCP connection is composed of one or
more regular TCP subflows that are combined
• Each host maintains state that glues the TCP subflows
that compose a Multipath TCP connection together
• Each TCP subflow is sent over a single path and appears
like a regular TCP connection along this path

21. Multipath TCP and the architecture
Physical
Datalink
Network
Transport
Application Multipath TCP
TCP1
socket
TCP2 TCPn...
Application
A. Ford, C. Raiciu, M. Handley, S. Barre, and J. Iyengar, “Architectural guidelines for multipath TCP
development", RFC6182 2011.

22. A regular TCP connection
• What is a regular TCP connection ?
– It starts with a three-way handshake
• SYN segments may contain special options
– All data segments are sent in sequence
• There is no gap in the sequence numbers
– It is terminated by using FIN or RST

23. Multipath TCP
SYN+MP_CAPABLE
SYN+ACK+MP_CAPABLE
ACK
SYN+MP_JOIN
SYN+ACK+MP_JOIN
ACK

24. TCP subflows
• Which subflows can be associated to a
Multipath TCP connection ?
– At least one of the elements of the four-tuple
needs to differ between two subflows
• Local IP address
• Remote IP address
• Local port
• Remote port
• Number of subflows can change during the
lifetime of a Multipath TCP connection

25. How to transfer data ?
seq=123,"a"
seq=124,"b"
seq=125,"c"
seq=126,"d"
ack=124
ack=126
ack=125
ack=127

26. How to transfer data
in today's Internet ?
seq=123,"a"
seq=124,"b"
seq=125,"c"
ack=124
ack=126
ack=125
Gap in sequence numbering space
Some DPI will not allow this !

27. Multipath TCP Data transfer
• Two levels of sequence numbers
Multipath TCP
TCP1
socket
TCP2
Multipath TCP
TCP1
socket
TCP2
ABCDEF
Data sequence #
TCP1 sequence #
TCP2 sequence #
A. Ford, C. Raiciu, M. J. Handley, and O. Bonaventure, “TCP Extensions for
Multipath Operation with Multiple Addresses,” RFC6824, 2013

28. Multipath TCP
Data transfer
Dseq=0,seq=123,"a"
DSeq=1, seq=456,"b"
DSeq=2, seq=124,"c"
DAck=1,ack=124
DAck=3, ack=125
DAck=2,ack=457

29. Multipath TCP
How to deal with losses ?
• Data losses over one TCP subflow
– Fast retransmit and timeout as in regular TCP
Dseq=0,seq=123,"a"
DAck=1,ack=12
4Dseq=0,seq=123,"a"
DAck=1,ack=124

30. Multipath TCP
• What happens when a TCP subflow fails ?
Dseq=0,seq=123,"a"
DSeq=1, seq=456,"b"
DAck=0,ack=457
Dseq=0,seq=457,"a"
DAck=2,ack=458

31. Multipath TCP use cases
High bandwidth on smartphones
• Koreans want 800+ Mbps on smartphones
WiFi
4G/LTE
Multipath TCP Regular TCP
SOCKS
O. Bonaventure, S. Seo, Multipath TCP Deployments, IETF Journal, November 2016,
http://www.ietfjournal.org/multipath-tcp-deployments/

32. Multipath TCP use cases
Low latency for Siri since 2013
• Long-lived TLS connections
WiFi
3G/LTE
Voice samples
Voice samples
On iOS11, any app can
Use Multipath TCP
O. Bonaventure, S. Seo, Multipath TCP Deployments, IETF Journal, November 2016,
http://www.ietfjournal.org/multipath-tcp-deployments/

33. Faster broadband networks ?

34. Multipath TCP use cases
Hybrid Access Networks
DSL
4G/LTE
Multipath TCP Regular TCP
Hybrid Access
Gateway
TCP
TCP
See https://www.tessares.net and
O. Bonaventure, M. Boucadair, B. Peirens, S. Seo, A. Nandugudi, 0-RTT TCP Converter, draft-ietf-tcpm-converters-00,
Feb. 2018

35. Why is Multipath TCP interesting for
networking researchers ?
• Availability of multiple paths opens new questions
– Path management
• When should a host create a subflow ?
• When should a host terminate a subflow ?
– Packet scheduling
• Which data should be sent over which subflow ?
• How to perform retransmissions when there are n subflows ?
– Congestion control
• Becomes a spatial problem and not anymore a temporal one
• Commercial deployment and open-source implem.
– http://www.multipath-tcp.org

36. Agenda
• Today's Internet
• Transport Layer : evolution or revolution ?
– Multipath TCP
– QUIC
• Network Layer evolution
– IPv6 Segment Routing

37. Google's quest for a faster web
• How to reduce web request latencies ?
– Deploy Google caches everywhere
– Tune TCP within the IETF
• Increase TCP's initial congestion window
• Tail Loss Probe
• TCP Fast Open
– Tune HTTP
• SPDY
• HTTP/2
– Tune TLS with TLS1.3
[RFC6829]
[RFC7413]
[RFC7540]
[RFC8446]

38. Can you improve web performance by
influencing both endpoints ?

39. A conversation at IETF'93

40. The QUIC revolution
• What are the benefits ?
– Deploy without convincing kernel developers/ SDO
HTTP/2
TLS
TCP
IP
Application
QUIC
IP
Application
UDP

41. QUIC evolves quickly
J. Ruth, et al. "A First Look at QUIC in the Wild" PAM2018 https://arxiv.org/abs/1801.05168

42. QUIC handshake
• No delay (0-rtt) between connection
establishment and web request/response
– Needs to remain secure
Client Hello
Server Hello
Encrypted request
Server Response

43. QUIC Handshake (2)
• Full handshake
Client Hello[incomplete]
Reject[Cert, H(CIP,Secret), ServerDHpubkey]
Server Hello [ServerDHkey, …]initialkey
Client Hello[H(CIP,Secret),ClientDHkey…]
Encrypted Request[GET …] initialkey
Encrypted Response[ServerDHkey, …]sessionkey

44. Main features of QUIC
• Support for multiple bytestreams
– Required for HTTP/2, but opens many possibilities
• Almost everything is encrypted (most headers)
– Prevents ossification caused by middleboxes
• Extensible protocol design
– Easy to add new frames
• Flexible packet numbering schemes
– Sequence numbers are never reused
• Flow control, congestion control
– Easy to add a new congestion control scheme

45. Why is QUIC interesting
for networking researchers
• QUIC is implemented in user space
– This makes experimentation much easier than
kernel space protocols
– It is easy for researchers to deploy QUIC+ apps
– A dozen of open-source implementations exist
• QUIC is easy to extend
– Easy to add new types of frames
– Sample extension : Multipath QUIC, see
http://www.multipath-quic.org

46. Agenda
• Today's Internet
• Transport Layer : evolution or revolution ?
– Multipath TCP
– QUIC
• Network Layer evolution
– IPv6 Segment Routing

47. Segment Routing
• A radical simplification of MPLS networks
• Basic principles
– Data plane is unchanged (32 bits shim header)
– Control plane becomes much simpler
• LDP
• RSVP-TE
• BGP
• OSPF or ISIS Simple extension

48. Segment Routing in one slide
• Each router has a label advertised by the IGP
– Packets follow shortest path to top label
R1
R4
R3
R5
R2 R7
R8 R9
100
3:7
3:7 7
8:4:7:3 4:7:3
7:3 7:3
3

49. Benefits Segment Routing
for network researchers
• Any network path becomes a succession of
shortest paths
• Reconsider algorithms to solve problems like
– Traffic engineering
• SR is different than IGP weights or MPLS tunnels
– Disjoint paths for resilience and fast reroute
– Monitoring
• With SR, a monitoring station can send packets on a
loop so that they return back to itself

50. IPv6 Segment Routing
• Differences with regular Segment Routing
– 128 bits IPv6 addresses are used to encode
intermediate nodes
• Router loopback addresses
• Network interface addresses
• Endhost addresses
– New IPv6 Extension Header inside each packet
– Endhosts can actively participate in the creation of
segmented paths
• D. Lebrun, O. Bonaventure, Implementing IPv6 Segment
Routing in the Linux kernel, ANRW17, https://www.segment-
routing.org

51. IPv6 Segment Routing
Network Programming
• IPv6 SR enables more than non-shortest paths
– Each nodes advertises one or more prefixes
R4 R5
R2 R7
R8 R9
IGP : 2001:…:4/40
FCT1:param
FCT2:param
Locator Function Param
C. Filsfils et al., SRv6 Network Programming, draft-filsfils-spring-srv6-
network-programming-03, Dec. 2017

52. eBPF
bytecode
Realising Network Programming :
the power of eBPF
Application
verifier
K
E
R
N
E
L
bpf syscall
map
eBPF
bytecode
eBPF
VM
M. Xhonneux et al., Leveraging eBPF for programmable network functions
with IPv6 Segment Routing, Proc. Conext 2018

53. Conclusion
• Innovation is back in the core Internet protocols
– Network researchers should participate
• Multipath TCP
– Spatial dimension opens many new challenges
• QUIC
– Truly extensible secure transport, interesting
opportunities
• IPv6 Segment Routing
– Non shortest paths and network programming
http://www.multipath-tcp.org
http://www.segment-routing.org
http://www.segment-routing.net
https://quicwg.github.io