My foray into Go began a few years ago when I started working at DigitalOcean. While building an abstraction layer on top of Kubernetes and familiarizing myself with the language, I began to love it. Syntactically simple, with amazing concurrency primitives and a wonderful community, Go was an excellent choice for a cloud-hosting company with a variety of low-level, server-side microservices.
In the last year, however, I've joined the software-defined networking team and learned of another application of Go; networking services. The networking team at DigitalOcean uses Go for a variety of purposes - from DHCP servers to IP address management services..to even wrappers around virtual switch tooling. Intrigued, I decided to also investigate how Go could be used to build other services such as port scanners and load-balancers.
This session will highlight my networking journey via Go. I will discuss useful packages, key learnings, and even struggles faced while building a variety networking services within and outside of DigitalOcean. I will discuss both relevant packages within the standard library and open source packages used to implement key network protocols. As a result, listeners will gain an understanding of how to specifically leverage Go for their own networking needs.
70. digitalocean.com
Building a layer-7 LB
1. Use HTTP protocol.
2. Accept client-side connections.
3. Pass client-side request to one of the
backends.
4. Return server-response back to the client.
71. digitalocean.com
HTTP:
HyperText Transfer Protocol
An application layer (7) protocol:
● request-response based
● stateless
● Has different methods (GET,
PUT, etc)
● Uses TCP or UDP sockets
under-the-hood
network definition
Source:
https://www.ntu.edu.sg/home/ehchua/programming/webprogramming/HTTP_Basic
s.html
77. digitalocean.com
Building a layer-4 LB:
1. Use TCP protocol (via a streaming socket).
2. Accept client-side connections.
3. Open backend connection.
4. Use one goroutine to shuttle packets from the
client to the backend.
5. Use another goroutine to shuttle packets from
the backend to the client.
78. digitalocean.com
TCP: Transmission
Control Protocol
Connection-oriented protocol
with 3 phases:
● Connection establishment
(TCP handshake)
● Data transfer
● Connection termination
network definition
Source:
https://www.researchgate.net/figure/Three-way-Handshake-in-TCP-Connection-
Establishment-Process_fig7_313951935
84. digitalocean.com
func copy(from net.Conn, to net.Conn) error {
for {
readBytes := make([]byte, 1024)
n, err := from.Read(readBytes)
if err != nil {
...
}
if _, err = to.Write(readBytes[:n]); err != nil {
...
}
}
}
4. Read and copy
85. digitalocean.com
Loadbalancer IRL:
● Exact mechanism of handling connections and/or requests
depends on the layer.
● Layer-4 loadbalancers should be more efficient:
○ Won’t necessarily create a new backend connection per
client conn.
○ Use NAT and munging incoming packets.
● Better loadbalancing algorithms.
● Client stickiness.
87. digitalocean.com
Nmap
Linux utility that scans hosts and
attempts to determine open
UDP/TCP ports
network definition
Source: https://commons.wikimedia.org/wiki/File:Screenshot_Nmap.png
89. digitalocean.com
Building a TCP port-scanner
1. Select range of ports to scan.
2. Try to open and connect a TCP socket to a
remote address (and port)..
3. Print results.
91. digitalocean.com
conSema := make(chan struct{}, 10)
var wg sync.WaitGroup
for i := 1; i <= 65535; i++ {
wg.Add(1)
go func(port int) {
conSema <- struct{}{}
...
wg.Done()
<-conSema
}(i)
}
wg.Wait()
2. Use waitgroup and channels
92. digitalocean.com
Nmap IRL:
● Checks UDP and TCP ports on local or remote hosts.
● Host discovery
● OS detection
● Auditing security of a firewall
96. digitalocean.com
DHCP Protocol
● Dynamic Host Configuration
Protocol is used by routers to
allocate IP addresses to
network interfaces
● DHCPv6 uses NDP and
DHCPv4 uses ARP
network definition
Source: https://study-ccna.com/dhcp-dns/
98. digitalocean.com
Building a DHCP server:
1. Open and bind a raw socket to an interface.
2. Read data from socket into bytes buffer.
3. Unmarshal into DHCP message and retrieve sender
hardware address.
4. Switch between handlers based on message type.
5. Validate DHCP request message and craft response.
6. Unicast response back to sender.
103. digitalocean.com
ifi, err := net.InterfaceByName(iface)
if err != nil {
return nil, err
}
pc, err := raw.ListenPacket(ifi, uint16(ethernet.EtherTypeIPv4),
&raw.Config{
// Don't set any timeouts to avoid syscall busy
// loops, since this server will run forever anyway.
NoTimeouts: true,
})
1. Open and bind raw socket
104. digitalocean.com
b := make([]byte, 1500)
for {
n, from, err := s.pc.ReadFrom(b)
if err != nil {
...
continue
}
buf := make([]byte, n)
copy(buf, b)
workC <- request{
buf: buf,
from: from,
}
}
2. Read data into buffer
106. digitalocean.com
func (h *handler) serveDHCPv4(ctx context.Context, req
*dhcp4.Packet, from *dhcp4conn.Addr) (*dhcp4.Packet, net.Addr)
{
...
if !from.EthernetSource.Equal(reqMAC) {
...
return h.shouldNAK(req.Type, to, broadcast)
}
...
}
4. Validate request
107. digitalocean.com
func (h *handler) serveDHCPv4(ctx context.Context, req
*dhcp4.Packet, from *dhcp4conn.Addr) (*dhcp4.Packet, net.Addr) {
...
res, err := h.buildResponse(ctx, params, req, from)
if err != nil {
topError(span, err, "failed to build response")
return h.shouldNAK(req.Type, to, broadcast)
}
res.Broadcast = broadcast
return res, to
...
}
5. Build and send response
108. digitalocean.com
DHCP Summary:
1. DHCP: dynamic host configuration protocol
used to dynamically assign IP address.
2. Use raw sockets.
3. Jump down to link-layer to do source MAC
validation.