Who Lives in Our Garden?
•
0 j'aime•1,835 vues
This document discusses containers and their history in Cloud Foundry. It describes the evolution from Warden to Garden container managers, and Garden's modular architecture and Linux-based backends like Aufs and Docker. It also mentions other backends for Garden like Greenhouse (Windows) and Guardian (supporting additional technologies like Docker, LXC, etc). Finally, it discusses the Open Containers Initiative standards and provides some debugging tips.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Who Lives in Our Garden?
Similaire à Who Lives in Our Garden? (20)
Plus de Altoros
Plus de Altoros (20)
Dernier
Dernier (20)
Who Lives in Our Garden?
- 1. Maksim Zhylinski ALTOROS @altoros Who Lives in Our Garden @uzzable
- 5. History of containers in Cloud Foundry
- 7. • namespaces • PID • IPC • Network • UTS (hostname) • cgroups • cpu • memory • devices • pivot_root
- 8. Warden architecture Warden client (DEA) container Warden server Container manager wshd app Warden by Glyn Normington Protocol
- 9. DEA + Warden (2011) Diego + Garden (2014)
- 10. Garden (2014) Implemented in Go (Garden = Go warden) Modular architecture: supports different back ends
- 11. Garden architecture Garden client (Diego) container Garden server wshd app Garden by Glyn Normington Protocol Container manager Back end
- 15. /var/vcap/packages/cflinuxfs2/rootfs /var/vcap/data/garden/aufs_graph/aufs/diff/… Rootfs (readonly) Container file system (read-write) /var/vcap/data/garden/aufs_graph/aufs/mnt/… Union mount point Garden filesystem
- 17. Docker image with garden /var/vcap/data/garden/aufs_graph/aufs/mnt/… Union mount point training/webapp ubuntu:14.04 /var/vcap/data/garden/aufs_graph/aufs/diff/… /var/vcap/data/garden/aufs_graph/aufs/diff/… /var/vcap/data/garden/aufs_graph/aufs/diff/… Container file system Docker image layers
- 18. Caveats Only v2 Docker registry API Does not support private repositories Source registry should be available on instance create
- 21. Files ystem isolation – unique user per container Disk usage – NTFS quotas CPU, Memory usage – Windows Job objects Network isolation – bind apps directly to external IP
- 25. The Open Containers Initiative
- 30. Debugging tips
- 31. cf ssh <app name>
- 34. Security
- 36. Cloud Foundry Concourse CI Garden users BOSH lite your app?..
Notes de l'éditeur
- Hello everyone, I’m happy and proud to be here today. Thank you for coming. My name is Maksim Zhylinski, I work as an engineer in Altoros. My talk is called “Who lives in garden?” and it’s about containers and precisely CloudFoundry containers.
- So what containers actually are? Container is a combination of operating system primitives that allows separate out set of processes and make them pretend like they are in own operating system. Add: 6-7 years?
- Containers offer higher-level isolation, with many applications running under the host operating system, all of them sharing certain operating system libraries and the operating system's kernel.
- Since a container doesn't have to load up an operating system and providers almost no overhead, you can create containers in a split-second, rather than minutes for a virtual machine. That makes containers perfect technology to use in different solutions, like running applications in Platform as a service.
- Let’s talk about history of containers in CloudFoundry. Containers was first-class citizens from the very beginning of the Cloud Foundry existence.
- First implementation of containers in CloudFoundry was called Warden and it was introduced in 2011. It was written mostly in Ruby with bits of C code.
- In order to provide processes isolation from host operating system and other processes was used linux namespaces feature, cgroups for resources limiting and management, pivot_root for filesystem isolation.
- Let’s see on warden architecture. It was quite monolithic – warden server, which was providing service to the warden clients (DEA) talking to warden by protobuf-based protocol was coupled in a single application with the container manager, which was responsible for container lifecycle routines.
- So, in 2014 brand new CloudFoundry runtime was introduced called Diego instead of good old DEA. It has a lot of cool features, but I will not go into details as far as it’s out of scope of my topic. And as a part of new Diego runtime new container manager was introduced – Garden.
- What has changed? Garden was rewritten on Go programming language, as many other components of CloudFoundry. Basically, garden stands for “Go warden”. But, what most important garden was designed in a modular way. What does it mean?
- Let’s see garden’s architecture. Still, garden clients (Diego cells) talks to Garden server by protocol (protocol is based on protobuf, but garden server also provides HTTP protocol for debugging purpose). But in order to provide flexibility garden server was decoupled from container manager (which now lives in component called “backend”), so that garden server provides unified contract that does not depend on underlying containers implementation.
- What backends are available today? It’s a Garden linux – at the moment default linux containers backend, greenhouse – windows containers backend, and one more linux backend called Guardian.
- Garden linux is a successor of warden’s containers implementation, it uses same features to provide process isolation: linux namespaces, cgroups, layered filesystems. But there’s killer-feature was added to garden-linux backend in addition to default buildpack-centric lifecycle is a…
- ability to run Docker containers! How does it work?
- First of all let’s dig a bit how garden works. Garden (as well as Warden was) uses layered filesystems. During the default for CloudFoundry buildpack lifecycle manager takes rootfs – basically linux root filesystem, and mounts it together with the another layer with application bits using aufs (by default) or overlayfs – to create a single filesystem to pivot_root to.
- So how docker images are run with garden? Docker images themselves are consist out of layers each containing diff applied on top of the previous layer.
- In order to run docker image garden just mount docker image layers instead of built-in rootfs keeping the rest pretty much the same. Garden uses libraries from Docker so contents of container exactly match the contents of the associated Docker image.
- So what’s the caveats? You can use only docker registry of version 2 You can use only public repositories Diego does caches pulled image globally, so garden needs access to Docker registry even when scaling up already running application. Using standard buildpack lifecycle during staging process CloudFoundry creates droplet, which can be reused later, so you can scale up your application and get exactly the same container. With Docker images – if registry is down, or someone deleted or modified image – you’re in trouble. Last item actually not strictly related to the topic of my talk, as far as it’s a business of whole runtime, but not a container manager. I took this three items from a pivotal’s official documentation, but I have to argue points 2 & 3.
- There’s a Diego docker cache BOSH release that helps to deal with them. Diego docker cache basically, caches pulled docker images from Docker hub. This helps to break dependency on docker registry availability or images change. But also this cache provides ability to set your credentials during first ‘cf push’. Bad part is that this project is in cloudfoundry-incubator and the feature not yet supported by CloudFoundry CLI, so providing credentials is a bit tricky.
- While the Linux is most popular operating system to deploy applications to nowadays, still, it’s not the only one. Dot NET applications are widely spread in enterprise area and it’s mandatory to support them in modern Platform-as-a-Service. Tricky part is that Windows does not support containers, as Linux does. But CloudFoundry runtime team managed to create Greenhouse backend using couple of tricks and techniques.
- Filesystem isolation – unique user created per container to utilize native Windows Access Control Lists Disk usage – NTFS quotas To manage memory limits Windows Kernel Job Objects are used … and to isolate network applications launched inside a container bind directly to the external IP of the VM
- And the last backend available is called “Guardian”. Why need for another one Linux container backend?
- There are plenty of containers implementation novadays: Famous known Docker LXC, that has been for a years already lmctfy made in google OpenVZ …and of course Garden … and many more Which are…
- …not compatible with each other.
- So in 2015 Open Containers Initiative appeared to create…
- industry standards around containers to build open, portable, platform-, cloud- and hardware-independent containers and runtime format.
- The OCI currently provides two specifications: the container Image Specification (runtime-spec) and the container Runtime Specification (image-spec).
- Main players of containers market has joined this initiative, so I tend to believe in future of unified containers.
- For example less than two weeks ago Kelsey Hightower announced native support for OCI containers in Kubernetes. So back to CloudFoundry. Garden already supports OCI containers for a while via it’s runc backend called Guardian. It’s not yet default backend for CloudFoundry. Current version is 0.9. And when it will hit 1.0 it’ll become a default backend for CloudFoundry.
- Ok, now some debugging tips for those of you who having troubles with your applications in CloudFoundry containers. Sometimes logs aren’t enough – your application works not as it was supposed to do. You want to get into container to see what’s inside. Maybe some mandatory file is missing, or something with permissions or whatever.
- Easiest way is to use CloudFoundry CLI’s built-in SSH command. Each garden container in CF comes with built-in SSH server, which makes it extremely easy.
- And you application bit is under root “app” directory.
- For those of you who wants to debug container manager itself I recommend to take a look at this tool. It’s basically CLI for garden – you can create and delete containers, shell into it and many other things like Diego does.
- And couple of words about security. How secure containers are? If we’re talking about linux-based containers provided by garden-linux and guardian backends… they are based on namespaces and cgroups—core Linux kernel features that have been around for years and used as the basis for many popular projects. So, I believe that you can trust CF containers as much as you can trust the Linux kernel itself. But if we talk about Greenhouse backend, windows containers implementation that it is impossible to create completely isolated filesystem, as linux pivot_root does, so greenhouse containers share Program files folder, so you’d better not to keep any secrets there.
- firewall
- And finally, what projects do use Garden now? Obviously it’s CloudFoundry Concourse CI – it’s a Continuos Integration and Continuos Delivery modern tool that support pipelining And BOSH lite – BOSH deployed in single VM and uses Garden containers to create jobs.