Our job might be to build web applications, but we can't build apps that rely on networking if we don't know how these networks and the big network that connects them all (this thing called the Internet) actually work. I'll walk through the basics of networking, then dive a lot deeper (from TCP/UDP to IPv4/6, source/destination ports, sockets, DNS and even BGP). Prepare for an eye-opener when you realize how much a typical app relies on all of these (and many more) working flawlessly... and how you can prepare your app for failure in the chain.

1. Wim Godden
Cu.be Solutions
@wimgtr
Your app lives on a network
Networking for web developers

2. Who am I ?
Wim Godden (@wimgtr)

3. Where I'm from

4. Where I'm from

5. Where I'm from

6. Where I'm from

7. Where I'm from

8. Where I'm from

9. My town

10. My town

11. Belgium – the traffic

12. Who am I ?
Wim Godden (@wimgtr)
Founder of Cu.be Solutions (http://cu.be)
Open Source developer since 1997
Developer of PHPCompatibility, OpenX, Nginx SLIC, ...
Speaker at PHP and Open Source conferences

13. Who are you ?
Developers ?
System engineers ?
Network engineers ?
Do you know how the Internet works ?

14. We’re web developers, not network engineers !
Know enough to build new stuff
Know enough to maintain existing code
What if...
Customer Support Desk Developers

15. Do you know these ?
TCP
UDP
IP
DNS
BGP
MAC address
IPv4
IPv6
SYN
ACK
Source port
Destination port
Default gateway
Routing table

16. Basics : OSI model
Physical
Layer 1
Data Link
Layer 2
Network
Layer 3
Transport
Layer 4
Session
Layer 5
Presentation
Layer 6
Application
Layer 7
Wires, network card, wireless interface
Data protocol (ethernet, ...)
IP adressing
TCP, UDP, ports, ...
TLS, L2TP, SOCKS, PPTP, ...
Serialization, data translation
HTTP, DNS, SMTP, ...

17. Basics : packets
01011010111010
Physical cable
or wireless
01011010111010

18. Basics : packets
Destination MAC (6 bytes) Source MAC (6 bytes) Type (2 bytes)
Payload (46 – 1500 bytes) CRC (4 bytes)
Part 1 : Ethernet frame
0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31
0 Version Header
length
DSCP ECN Total length
32 Identification Flags Fragment Offset
64 Time To Live Protocol Header Checksum
96 Source IP Address
128 Destination IP Address
160 Options (if required)
< Contents of the packet >
Part 2 : IPv4 header (min. 160 bytes)
Part 3 : TCP/UDP/… header and data

19. Basics : TCP packet
Bit 0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31
0 Source port Destination port
32 Sequence number
64 Acknowledgment number
96 Data
offset
Flags Window size
128 Checksum Urgent pointer
160 Options (if required)
< Contents of the packet >

20. Basics : packets
Destination MAC (6 bytes) Source MAC (6 bytes) Type (2 bytes)
Payload (46 – 1500 bytes) CRC (4 bytes)
Part 1 : Ethernet frame

21. Sending on a local network
Each system has a MAC address
Assigned by manufacturer
Can be overwritten (for VM or failover)
Same physical network → send packet to MAC address
Switch knows MAC address(es) of devices and forwards traffic
Layer 2

22. Sending IP traffic on local network
Requires IP addresses
Where to send ? We need to know MAC address
Uses ARP (Address Resolution Protocol) for lookup
Stores IP ↔ MAC relation in ARP table
What’s “local” ?
→ Same IP subnet
OK, what’s a subnet ?
Layer 3
16:58:56.933019 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.0.15 tell 192.168.0.12, length 28
16:58:56.938019 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.0.15 is-at 00:50:56:8b:6a:b7, length 46

23. IP adressing (IPv4)
IPv4 adressing = CIDR notation
xxx.xxx.xxx.xxx where 0 <= xxx <= 255
0.0.0.0 → 255.255.255.255
Total amount of IP addresses available :
256 * 256 * 256 * 256 = 28
* 28
* 28
* 28
= 232
= 4.3 billion
Subnet = range of IP addresses
192.168.0.0 → 192.168.0.255
Typical notation uses a “mask” :
192.168.0.0 → 192.168.0.255 = 192.168.0.0/24
IPv4 provides 232
addresses
A /24 mask gives 2(32-24)
or 28
addresses = 256 addresses
Local networks :
10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16

24. Sending IP traffic on local network
Client
192.168.0.15/24
Server
192.168.0.2/24
MAC for
192.168.0.2 ?
AA:BB:CC:DD:EE:FF
Let’s talk !

25. Sending IP traffic to remote device
Requires IP addresses
Where to send ?
Can not use ARP : MAC addresses are not shared beyond local network
Uses default gateway
Provided by DHCP or
Set statically
Must be on same subnet → address found in ARP table
Layer 3

26. Sending IP traffic to remote device
Requires IP addresses
Where to send ?
Can not use ARP : MAC addresses are not shared beyond local network
Uses default gateway
Provided by DHCP or
Set statically
Must be on same subnet → address found in ARP table
Layer 3
Client Router ServerInternet
192.168.0.15 192.168.0.1 194.7.1.4
Default gateway
MAC for
192.168.0.1 ?
AA:BB:CC:DD:EE:FF
Destination : AA:BB:CC:DD:EE:FF
Contents : TCP packet to 194.7.1.4
See ARP table : arp -a
See default gateway : route -n (Lin)
route print (Win)

27. Basics : TCP packet
Bit 0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31
0 Source port Destination port
32 Sequence number
64 Acknowledgment number
96 Data
offset
Flags Window size
128 Checksum Urgent pointer
160 Options (if required)
< Contents of the packet >

28. Establishing a TCP connection
Client Server
SYN
SYN ACK
ACK
Data
Sequence no = 1002
Acknowledge no = 9001
Sequence no = 1000
Sequence no = 1001
Acknowledge no = 9000

29. Establishing a TCP connection
Client Server
SYN
SYN ACK
ACK
Data
Sequence no = 1002
Acknowledge no = 9001
Sequence no = 1000
Sequence no = 1001
Acknowledge no = 9000
Brussels Montreal45ms
0
45
90
135

30. Establishing a TCP connection
Client Server
SYN
SYN ACK
ACK
Data
Sequence no = 1002
Acknowledge no = 9001
Sequence no = 1000
Sequence no = 1001
Acknowledge no = 9000
Brussels Montreal45ms
0
45
90
135
London10ms
10
20
30

31. TCP Window Size
Client Server
Brussels Montreal
SYN
SYN ACK
rwnd = 8192
rwnd = 8192
rwnd = 16384
ACK
DATA
sysctl net.ipv4.tcp_window_scaling

32. TCP Slow Start
Client Server
Brussels Montreal45ms
0
45
90
135
180
225

33. TCP Slow Start
Client Server
Brussels Montreal45ms
0
45
90
135
180
225

34. New vs existing connection
Client Server
Brussels Montreal45ms
0
45
90
135
280
325
SYN
SYN ACK
ACK
DATA
(x8)
ACK
(x4)
GET /url
DATA
(x4)
ACK
(x8)
370
415
Processing request
235

35. New vs existing connection
Client Server
GET /url
Brussels Montreal45ms
0
45
145
180
DATA
DATA
(x12)
ACK
(x12)
Processing request
225

36. TCP Performance
Upgrade to latest Linux kernel or OS
Check window size
Reduce latency (move servers closer to client)
Reuse already established connections

37. UDP
User Datagram Protocol
Unreliable Datagram Protocol
Connectionless
→ No 3-way handshake required
Simple packet structure
Packets might not arrive
Packets might arrive out of order
Ideal for streaming, gaming, ...
Bit 0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31
0 Source port Destination port
32 Length Checksum
< Contents of the packet >

38. TCP/UDP ports
Bit 0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31
0 Source port Destination port
32 Sequence number
64 Acknowledgment number
96 Data
offset
Flags Window size
128 Checksum Urgent pointer
160 Options (if required)
< Contents of the packet >
Bit 0-3 4-7 8-11 12-15 16-19 20-23 24-27 28-31
0 Source port Destination port
32 Length Checksum
< Contents of the packet >
TCP
UDP

39. Source and Destination ports
Destination port : defined by service
HTTP : TCP port 80
HTTPS : TCP port 443
DNS : UDP port 53
Source port : for identification of a connection
Client Server
80
80
80
5000
5001
5002
See active connections with
source/destination ports :
netstat -n

40. Fetching a website
Need to fetch http://cu.be
TCP doesn’t know what cu.be is
→ needs an IP address
Looks up IP address through DNS
Open a socket
Connect to IP address on port 80
Send HTTP request over the connection
Get data back
Get images, CSS, javascript over the same connection
Close the connection
Show the webpage

41. DNS lookups
Through a DNS server
Authoritative : in charge of the domain name
Recursive : asks the authoritative server, then caches for a while
→ Cache time is defined by TTL
Usually you will use a recursive server (owned by your provider)
Client Recursive
DNS
Server
IP for
cu.be ?
Root DNS
server
IP for
cu.be ?
.be DNS
server
cu.be DNS
server
Ask the .be
DNS server
IP for
cu.be ?Ask the cu.beDNS server
IP forcu.be ?194.50.97.38
194.50.97.38

42. DNS lookups
Actual lookups depend on type of DNS record
DNS holds lots of things :
A record = IP addresses
CNAME records = aliases for A records
MX records = mail servers
NS records = DNS servers
TXT = various stuff (anti-spam mostly)
2 tools to debug DNS :
dig
nslookup

43. Sockets
The layer between your application and TCP, UDP, ...
Abstracts syntax
Makes it easy to switch between protocols
Provides an easy interface
No need to know implementation
Send a stream of data → split up in packets
Receive lots of data → converted from packets to string

44. Packets over the Internet
Client Router ServerInternet
192.168.0.15 192.168.0.1 194.7.1.4
BGP protocol decides how packets are routed
Each public network has AS (Autonomous System) number
AS3356 = Level3
AS39628 = Cu.be
BGP announces subnets over BGP to its uplink providers :
“AS39628 here… you can reach 194.50.97.0/24 through me”

45. BGP routes

46. BGP routing
Router
AS 1
Router
AS 5
Router
AS 5
Router
AS 52
Router
AS 10
Router
AS 2
Client
Server

47. IPv6
Created to solve lack of IP addresses (4.3 billion in IPv4)
Standard created in 90s
Deployed on most major sites, but small sites behind
Addresses :
IPv4 address : 192.168.0.1
IPv6 address : 2001:0db8:0000:0000:0000:0000:0370:7334
Abbreviated : 2001:0db8::0370:7334
Can’t talk to eachother !
Address space :
2128
= 340,282,366,920,938,463,463,374,607,431,770,000,000
Deployment rates (source : Google) :
Global : 13.12%
US : 29.78%
Canada : 16.58%
Belgium : 48.42%
Should you use it ? YES ! (But don’t forget about firewalling !)

48. TLS
Client Server
0
45
90
135
180
225
SYN
ClientHello
SYN ACK
ACK
ServerHello
Certificate
ServerHelloDone
ClientKeyExchange
ChangeCipherSpec
Finished ChangecipherSpec
Finished
DATA 270
315

49. TLS with Session Resumption
Client Server
0
45
90
135
180
225
SYN
ClientHello
SYN ACK
ACK
ServerHello
ChangecipherSpec
Finished
ChangeCipherSpec
Finished
DATA
270
315

50. TLS → HSTS
HSTS = HTTP Strict Transport Security
Remembers that a site is SSL-only
Prevents users from going to http:// then redirected to https://
Prevents leaking of session cookies over unsecured wifi

51. HTTP
It’s what we use every day ;-)
There’s “new” version : HTTP/2
Developed by Google as SPDY
Designed for speed
Multiple simultaneous requests in 1 connection
Binary format (pro : more efficient – con : harder to debug)
TLS/SSL encryption is standard
Built-in prioritization
Server Push
Header compression
Try it out
Deploy it !

52. Questions ?

53. Questions ?

54. Contact
Twitter @wimgtr
Slides http://www.slideshare.net/wimg
E-mail wim@cu.be
Thanks !