Writing the Container Network Interface(CNI) plugin in golang
•
9 j'aime•2,417 vues
An introduction to Container Network Interface (CNI), including what problems it want solve and how it works. Also contains a example about how to write a simple CNI plugin with golang
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Writing the Container Network Interface(CNI) plugin in golang
Similaire à Writing the Container Network Interface(CNI) plugin in golang (20)
Plus de HungWei Chiu
Plus de HungWei Chiu (20)
Dernier
Dernier (20)
Writing the Container Network Interface(CNI) plugin in golang
- 1. Introduction to CNI (Container Network Interface) Hwchiu (Hung-Wei Chiu) Linkernetworks.com
- 2. Who Am I • Hung-Wei Chiu ( ) • hwchiu@linkernetworks.com • Blog: hwchiu.com • Experience • Software Engineer at Linker Networks • Co-Founder of SDNDS-TW (Software Defined Network) • Co-Found of CUTNG(Cloud Native Taiwan User Group) • Open Source Experience • SDN Related Projects (ONOS/Mininet/Floodlight) • OVS-CNI
- 3. Outline • Introduction to CNI • How to write a CNI in golang
- 4. Before Taking About The CNI
- 5. Do You Heard Before? • Linux network namespace • Kernel function • Docker • Docker Network • Bridge/Host..etc • Kubernetes • Flannel/Calico/Weave..etc
- 6. So, What Is Docker? How It Works ?
- 8. A Simple HTTP Server • docker run --name my-nginx -p 8080:80 nginx • Use the localhost:8080 to communicate with nginx. • How it works ?
- 9. Step By Step 1. Create a Linux Bridge Linux Host br0
- 10. Step By Step 1. Create a Linux Bridge 2. Create a Container Linux Host br0 Container (Nginx)
- 11. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair Linux Host br0 Container (Nginx) veth234 veth123
- 12. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair 4. Attach veth pari to container and bridge (also rename) Linux Host br0 Container (Nginx) veth234 eth0
- 13. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair 4. Attach veth pari to container and bridge (also rename) 5. Assign an IP address to container Linux Host br0 Container (Nginx) veth234 eth0172.16.2.5/24
- 14. Step By Step 1. Create a Linux Bridge 2. Create a Container 3. Create a veth pair 4. Attach veth pari to container and bridge (also rename) 5. Assign an IP address to container 6. Setup a iptablses rule for 8080:80 Linux Host br0 Container (Nginx) veth234 eth0172.16.2.5/24
- 15. In The Previous Example • The networking part is handled by the linux network namepsace (ns) • veth is used to connect two different ns
- 16. Do We Have Any Other Options ? • Docker run –network=… • Bridge (bydefault) • Host • ContainerID • Docker networks (CNM) • Create your network.
- 17. How About Other Container System ? • LXC • rkt • Mesos • Kubernetes • …etc
- 18. We Need To Make It Simple • Develop once, run everywhere • That’s CNI (Container Network Interface) • https://github.com/containernetworking/cni • Developed by go language
- 19. What Is CNI • A CNCF (Cloud Native Computing Foundation) project • For Linux Containers • Consists of a specification and libraries for writing plugins. • Only care about networking connectivity of containers • Create/Remove
- 20. Who Use CNI • rkt - container engine • Kubernetes - a system to simplify container operations • OpenShift - Kubernetes with additional enterprise features • Cloud Foundry - a platform for cloud applications • Apache Mesos - a distributed systems kernel • Amazon ECS - a highly scalable, high performance container management service
- 21. Network Connectivity • Use the previous docker example, The CNI will do • Create the Linux Bridge • Create the veth and attach to the container (ns) • Find a IP address and assign the IP to the Linux Bridge • Other staffs (You can do anything you want)
- 22. Others CNI • SR-IOV (Physical NIC to container) • OVS (Use OpenvSwitch rather than Linux Bridge) • Flannel (Support tunnel via UDP/VXLAN) • MacVlan/IPVlan • PTP • Vlan • …etc
- 23. So, How To Develop a CNI Plugin?
- 24. Let’s See A Example
- 25. First • Assume we have already implemented a CNI called simple-cni • Assume we have create a network namespace (ns) vir the following command • ip netns add ns1 • We have a json config contains the information we need. • { ”name”: “simple-cni” }
- 26. Second • Execute the following command • sudo CNI_COMMAND=ADD CNI_CONTAINERID=ns1 CNI_NETNS=/var/run/netns/ns1 CNI_IFNAME=eth10 CNI_PATH=`pwd` ./simple-cni < config
- 27. Explain • COMMAND • ADD/DELETE/VERSION • CONTAINERID • Just a ID… • NETNS • The location of ns • IFNAME • NIC name in the container • PATH • Where to find the binary • Stdin • Just a json config
- 28. What The Simple-CNI do • Load the information from the config (bridge name, IP address) • Create a Linux Bridge • Create a veth and attach to $NETNS • Rename the NIC to $IFNAME • Set the IP address to the NIC (We call it IPAM )
- 30. Skeleton • We should implement two function (Add/Delete) for CNI_COMMAND • We will get those data via skel.CmdArgs
- 33. First • We should add a special function init
- 34. First(Cont’d) • Decode the StdinData to out structure. • You can define any data you want. • In my example. I get the bridge name and IP address from the config.
- 35. First • Decode the StdinData to out structure.
- 36. Create a Linux Bridge • We have to ways to create a linux bridge • Call the linux command (brctl addbr ….) • Use the netlink to create a linux bridge • We use this method our example.
- 37. Create a Linux Bridge • Prepare a bridge object netlink.Bridge{} • Create a bridge via netlink.LinkAdd • brctl add br • Up the Linux bridge via netlink.LinkSetUp • ifconfig xxx up
- 39. Second • Create a veth pair via netlink.Veth • Setup the veth via netlink.LinkSetUp • Move one side of veth to another ns via netlink.LinkSetNsFd • Setup the NICs of the veth via netlink.LinkSetUp
- 40. Second(cont’d) • We can create a veth on the host ns and move one side into container ns. • Or, we can create a veth on the container ns and move one side into host ns. • Choose any approach you like.
- 41. Second • The better way is to use the function provide by containernetworking/plugins/pkg/ip package.
- 42. The simple way. • Get the NS Object from the ns.GetNs • Call the SetupVeth on the continaer ns.
- 44. Third. • We need to attach the one side of the veth into the Linux bridge • First, get the Link Object via netlink.LinkByName • Second, attach the link to bridge via netlink.LinkSetMaster
- 45. Now • We have created the Linux bridge • We have create a veth and connect the host ns and container ns. • We also attach the veth to the Linux Bridge Linux Host br0 Network Namespace veth234 eth0
- 46. Next • We need to handle the IPAM (IP address management) • In this example, we get the IP address from the config. • We can set the ip address via netlink.AddrAdd
- 48. Let’s Demo Now.
- 49. Other Things About CNI • Build-in IPAM • Host • DHCP • DIY
- 51. By The Way
- 53. Q&A