Kubernetes network policies can be bypassed through several methods, including excessive privileges that allow editing labels, creating pods in allowed namespaces, or insecure kubelet configurations. Monitoring for suspicious role bindings, host mounts, and static pod paths as well as enforcing CIS benchmarks can help mitigate these risks. Detection tools like Falco and prevention tools like OPA, admission controllers, and pod security policies can also help prevent network policy bypass.

1. Kubernetes Network
Policy Bypass

2. @mfdii
About me.
Kaizhe Huang
Security Researcher, Sysdig
@kaizhe @kaizhe

3. Agenda
● CNI
● CNI Plugins
● Kubernetes Network Policy
● Bypass Network Policy
● Mitigation Strategy

4. CNI
CNI (Container Network Interface), a Cloud Native Computing Foundation project,
consists of:
1. A specification
2. Libraries for writing plugins to configure network interfaces in Linux
containers,
3. Some supported plugins.

5. CNI
CNI concerns itself only with network connectivity of containers and removing
allocated resources when the container is deleted.
Operation Requirement (container runtime)
● add container to network
● delete container to network
Communication Requirement (k8s cluster network model)
● pods on a node can communicate with all pods on all nodes without NAT
● agents on a node (e.g. system daemons, kubelet) can communicate with all
pods on that node
Note: CNI plugins are one of the flavors of k8s network plugins

6. CNI Plugins
● Executable binaries
● Implement CNI Spec
○ Create network interfaces for
containers
○ Allocate IP address (by calling
other IPAM plugin like host-local)
● Implement Network Policy (maybe)

7. Kubernetes Network Policy
A network policy is a specification of how groups of pods are allowed to
communicate with each other and other network endpoints. It is implemented by
the network plugin.

8. Kubernetes Network Policy
● Namespaced Object - Enforcement applies for pods in the namespace
● Whitelist Policy - Define what network communication is allowed
○ Ingress Rule
○ Egress Rule
Note: There is a deny all (both ingress and egress)

9. Kubernetes Network Policy
● spec.pod Selector - grouping of pods to
which the policy applies
● spec.ingress
○ ip Block - IP CIDR ranges to allow as
ingress sources
○ namespace Selector - namespaces
for which all Pods should be allowed
as ingress sources
○ podSelector - particular Pods in the
same namespace as the
NetworkPolicy which should be
allowed as ingress sources
● spec.egress - same as spec.ingress except
for egress destination

10. Kubernetes Network Policy
Example:
● create an ingress rule allow for a specific
service
Enforcement:
Allow connections from Pods with the label
role=client in namespaces with the label
user=alice.

11. A Basic Example
Namespace: default
Nginx-web
Namespace: good
Good Pod
Label:
From: good
Namespace: bad
Bad Pod
Label:
From: bad
Network Policy Ingress Rules:
1. Namespace label: from:good
2. Pod label: from:good

12. Bypass Network Policies
Question: assume you have network policies properly configured, can they still
be bypassed?

13. Patch Network Policy
Namespace: default
Nginx-web
Namespace: good
Good Pod
Label:
From: good
Namespace: bad
Bad Pod
Label:
From: bad
Network Policy Ingress Rules:
1. Namespace label: from:bad
2. Pod label: from:bad
Privilege to patch network
policy (cluster)

14. Create Pod in Allowed Namespace
Namespace: default
Nginx-web
Namespace: good
Good Pod
Label:
From: good
Namespace: bad
Bad Pod
Label:
From: bad
Network Policy Ingress Rules:
1. Namespace label: from:good
2. Pod label: from:good
bad-pod
Privilege to create pod
(cluster)

15. Exec into Good Pod
Namespace: default
Nginx-web
Namespace: good
Good Pod
Label:
From: good
Namespace: bad
Bad Pod
Label:
From: bad
Network Policy Ingress Rules:
1. Namespace label: from:good
2. Pod label: from:good
Privilege to exec into
Good pod or Nginx-web
(cluster)

16. Edit Labels
Namespace: default
Nginx-web
Namespace: good
Good Pod
Label:
From: good
Namespace: good
Bad Pod
Label:
From: good
Network Policy Ingress Rules:
1. Namespace label: from:good
2. Pod label: from:good
Privilege to edit labels
(namespaced)

17. Via Kubelet
Namespace: default
Nginx-web
Namespace: good
Good Pod
Label:
From: good
Namespace: bad
Bad Pod
Label:
From: bad
Network Policy Ingress Rules:
1. Namespace label: from:good
2. Pod label: from:good
Privilege to create
nodes/proxy (cluster)
kubelet

18. Via Kubelet
Namespace: default
Nginx-web
Namespace: good
Good Pod
Label:
From: good
Namespace: bad
Bad Pod
Label:
From: bad
Network Policy Ingress Rules:
1. Namespace label: from:good
2. Pod label: from:good
Kubelet
misconfiguration
kubelet

19. Insecure Host Mount
Namespace: default
Nginx-web
Namespace:
good
Good Pod
Label:
From: good
Namespace: bad
Bad Pod
Label:
From: bad
Network Policy Ingress Rules:
1. Namespace label: from:good
2. Pod label: from:good
Write to static pod path
Write to /etc/kubernetes/manifest
Bad-static-pod
Label:
From: good

20. Recap
● Risk Types
○ Excessive Privileges Grant
■ Create pod/deployment/[Cron]jobs/statefulset/daemonSet/pods-exec/nodes-proxy
■ Patch networkpolicy/namespace/pod
○ Insecure Host Mount
■ Static pod path
○ Insecure Kubelet Configuration
■ With default authorization (AlwaysAllow)
● Asset Access Path
○ Via network
○ Via kubelet

21. Mitigation Strategy
● Watch [cluster]rolebinding create/patch events where roles including:
○ Create pod/deployment/[Cron]jobs/statefulset/daemonSet/pods-exec/nodes-proxy
○ Patch networkpolicy/namespace/pod
○ Get secrets
○ Impersonate [privileged subjects]
● Watch pods create event with host mounts including
○ Static pod path
○ /var/lib/kubelet/pods or its parent path
● Enforce CIS benchmark in k8s cluster

22. Mitigation Tools
● Detection
○ Falco
○ K8s audit webhook
● Prevention
○ OPA
○ Admission Controller
○ PodSecurityPolicy (kube-psp-advisor can help)

23. ● https://github.com/containernetworking/cni
● https://github.com/containernetworking/cni/blob/master/SPEC.md
● Kubernetes Networking
● Network Policies
● Understand Kubernetes Networking
● Container Networking Interface aka CNI
● Introduction to Container Network Interface (CNI)
● Kubernetes Network Plugins
● Benchmark results of Kubernetes network plugins (CNI)
References