Production FS:
Adapt or Die
hello!
Claudia & Tiago
callisto13 tscolari
Summary
✘ Glossary - What is a Root Filesystem?
✘ What is CloudFoundry?
✘ Warden
✘ Garden Linux
✘ Garden runC
✘ GrootFS + ...
1) Directory hierarchy and structure
1) Type: How the data is organised
What is a FileSystem?
✘ Top of the dir structure
✘ Mount point for other FSes at boot
✘ Contains all critical startup files
✘ Sets state of syst...
What is a RootFS?
Host
What is a RootFS?
Container
Host
✘ Open Source Platform as a Service
✘ Development started in 2009 (VMWare)
✘ First released in 2011
✘ Run application insi...
Provide framework and runtime support for applications
Officially supported:
○ Binary
○ Go
○ Java
○ .Net Core
○ Node.js
Bu...
Buildpack
Buildpack
RootFS
Application Code
RootFS
Droplet
Compilation
Droplet
✘ Multitenancy
✘ Application
Quotas
Isolation
✘ Independent of Host FS
What did CF need?
2011
✘ Kernel 2.6
✘ Containers?
✘ No user namespace available
1.
AUFS + WARDEN
2011
Warden
✘ Developed in Ruby and C
✘ Initially with LXC
✘ Coupled to Linux
Namespaces (exc User) &
cgroups
WHY AUFS?
✘ Mounting the rootfs
was faster than
copying it
✘ No duplicated files
But...
✘ No support for quotas
✘ Not in M...
RootFS Mnt: RootFS
RW Layer
Droplet
Mnt: RootFS
RW Layer
Droplet
Mnt: RootFS
RW Layer
Droplet
...
root root root root
uniq...
2013-2014
✘ User namespaces
✘ Security
✘ Scalability
✘ Containers
2.
BTRFS + GARDEN LINUX
2014/2015
Garden & Diego
✘ Replacement for Warden
✘ Go (w)arden - Garden
✘ Platform Agnostic API -
future support for windows
✘ New ...
+ Docker Images
✘ No control over
RootFS anymore
✘ Security Risks ✘ User namespaces
Why BTRFS?
✘ Dependence on Docker graph
driver
✘ Built in support for quotas
✘ The other options were:
Overlay - not matur...
Everything was changing
✘ New scheduler
✘ New container runtime
✘ New container Filesystem
✘ New IAAS
And...
✘ Huge Performance Hit: BTRFS blamed (eventually)
✘ Theory was BTRFS garbage collection was consuming all IOPS from...
3.
AUFS + GARDEN LINUX
October 2015
Why AUFS, again?
✘ Familiarity
✘ But…
Quotas?
RootFS Mnt: RootFS
mnt:/dev/loop1
Droplet
...
root root
APP1
Mnt: RootFS
mnt:/dev/loop2
Droplet
root
APP2
Mnt: RootFS
mnt:...
2015
✘ runC
4.
AUFS + GARDEN RunC
May 2016
Open Containers Initiative / RunC
✘ Open Standard for containers specification
✘ Implementation of OCI container specs
Garden RUNC
✘ Rewritten to use RunC
✘ More GO, Less C
✘ More Security
AUFS...
✘ More bugs
✘ Distraction
2015/2016
✘ OCI: Image-Spec
✘ GrootFS - new project to replace Garden Linux backend
Dedicated team
✘ Security: Garden runC...
5.
BTRFS + GROOTFS +
GARDEN runC
June 2016
Why BTRFS, again?
✘ Snapshotting: plays well with
container images
✘ Could be (almost) rootless
✘ Quotas
✘ Previous issues...
Container Image
Snapshot: rootfs
Droplet
1001
Buildpack App
Snapshot: layer1
1001
Docker Img App
Snapshot: layer2
Snapshot: layer3
But...
✘ New performance issue: `btrfs enable quota`
OVERLAY/XFS +
GROOTFS + GARDEN
runC
6.
February 2017
Why Overlay+XFS?
✘ Maturity ✘ Overlay for layering ✘ XFS for quotas
XFS folder app1/
Overlay mount
upperdir: app1/diff
lowerdirs: layer1:layer2:layer3
App 1
XFS folder app2/
Overlay mount
up...
The Future
What’s Next?
✘ EXT4
Kernel 4.5 in Stemcell
Match host FS
✘ ShiftFS
On the fly user mappings
No translation layer
Conclusion
✘ Nothing is forever
✘ There are always risks
Agility is key
✘ Focus!
thanks!
Any questions?
callisto13 / cberesford@pivotal.io
tscolari / tscolari@pivotal.io
Slide template by SlidesCarnival
Production FS: Adapt or die - Claudia Beresford & Tiago Scolar
Production FS: Adapt or die - Claudia Beresford & Tiago Scolar
Production FS: Adapt or die - Claudia Beresford & Tiago Scolar
Prochain SlideShare
Chargement dans…5
×

Production FS: Adapt or die - Claudia Beresford & Tiago Scolar

249 vues

Publié le

Retrouvez la présentation de Claudia Beresford & Tiago Scolar de Pivotal lors de la conférence du Paris Container Day

Publié dans : Technologie
0 commentaire
0 j’aime
Statistiques
Remarques
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Aucun téléchargement
Vues
Nombre de vues
249
Sur SlideShare
0
Issues des intégrations
0
Intégrations
2
Actions
Partages
0
Téléchargements
5
Commentaires
0
J’aime
0
Intégrations 0
Aucune incorporation

Aucune remarque pour cette diapositive

Production FS: Adapt or die - Claudia Beresford & Tiago Scolar

  1. 1. Production FS: Adapt or Die
  2. 2. hello! Claudia & Tiago callisto13 tscolari
  3. 3. Summary ✘ Glossary - What is a Root Filesystem? ✘ What is CloudFoundry? ✘ Warden ✘ Garden Linux ✘ Garden runC ✘ GrootFS + Garden runC ✘ The Future ✘ Questions?
  4. 4. 1) Directory hierarchy and structure 1) Type: How the data is organised What is a FileSystem?
  5. 5. ✘ Top of the dir structure ✘ Mount point for other FSes at boot ✘ Contains all critical startup files ✘ Sets state of system ✘ Has tools for recovery of broken system and data What is a Root FileSystem?
  6. 6. What is a RootFS? Host
  7. 7. What is a RootFS? Container Host
  8. 8. ✘ Open Source Platform as a Service ✘ Development started in 2009 (VMWare) ✘ First released in 2011 ✘ Run application inside containers ✘ Supports buildpacks and Docker images What is CloudFoundry?
  9. 9. Provide framework and runtime support for applications Officially supported: ○ Binary ○ Go ○ Java ○ .Net Core ○ Node.js Buildpacks? ○ PHP ○ Python ○ Ruby ○ Static File
  10. 10. Buildpack Buildpack RootFS Application Code RootFS Droplet Compilation
  11. 11. Droplet
  12. 12. ✘ Multitenancy ✘ Application Quotas Isolation ✘ Independent of Host FS What did CF need?
  13. 13. 2011 ✘ Kernel 2.6 ✘ Containers? ✘ No user namespace available
  14. 14. 1. AUFS + WARDEN 2011
  15. 15. Warden ✘ Developed in Ruby and C ✘ Initially with LXC ✘ Coupled to Linux Namespaces (exc User) & cgroups
  16. 16. WHY AUFS? ✘ Mounting the rootfs was faster than copying it ✘ No duplicated files But... ✘ No support for quotas ✘ Not in Mainline Kernel
  17. 17. RootFS Mnt: RootFS RW Layer Droplet Mnt: RootFS RW Layer Droplet Mnt: RootFS RW Layer Droplet ... root root root root unique uid APP1 APP2 APP3
  18. 18. 2013-2014 ✘ User namespaces ✘ Security ✘ Scalability ✘ Containers
  19. 19. 2. BTRFS + GARDEN LINUX 2014/2015
  20. 20. Garden & Diego ✘ Replacement for Warden ✘ Go (w)arden - Garden ✘ Platform Agnostic API - future support for windows ✘ New scheduler ✘ DEA(Go) - Diego
  21. 21. + Docker Images ✘ No control over RootFS anymore ✘ Security Risks ✘ User namespaces
  22. 22. Why BTRFS? ✘ Dependence on Docker graph driver ✘ Built in support for quotas ✘ The other options were: Overlay - not mature DeviceMapper - required LVM ZFS - proprietary
  23. 23. Everything was changing ✘ New scheduler ✘ New container runtime ✘ New container Filesystem ✘ New IAAS
  24. 24. And... ✘ Huge Performance Hit: BTRFS blamed (eventually) ✘ Theory was BTRFS garbage collection was consuming all IOPS from the cells ✘ BTRFS new and didn’t have enough support at the time
  25. 25. 3. AUFS + GARDEN LINUX October 2015
  26. 26. Why AUFS, again? ✘ Familiarity ✘ But… Quotas?
  27. 27. RootFS Mnt: RootFS mnt:/dev/loop1 Droplet ... root root APP1 Mnt: RootFS mnt:/dev/loop2 Droplet root APP2 Mnt: RootFS mnt:/dev/loop3 Droplet root APP3 sparse
  28. 28. 2015 ✘ runC
  29. 29. 4. AUFS + GARDEN RunC May 2016
  30. 30. Open Containers Initiative / RunC ✘ Open Standard for containers specification ✘ Implementation of OCI container specs
  31. 31. Garden RUNC ✘ Rewritten to use RunC ✘ More GO, Less C ✘ More Security
  32. 32. AUFS... ✘ More bugs ✘ Distraction
  33. 33. 2015/2016 ✘ OCI: Image-Spec ✘ GrootFS - new project to replace Garden Linux backend Dedicated team ✘ Security: Garden runC Rootless
  34. 34. 5. BTRFS + GROOTFS + GARDEN runC June 2016
  35. 35. Why BTRFS, again? ✘ Snapshotting: plays well with container images ✘ Could be (almost) rootless ✘ Quotas ✘ Previous issues fixed in kernel 4.4 ✘ Big companies investing ✘ Support from Canonical
  36. 36. Container Image
  37. 37. Snapshot: rootfs Droplet 1001 Buildpack App Snapshot: layer1 1001 Docker Img App Snapshot: layer2 Snapshot: layer3
  38. 38. But... ✘ New performance issue: `btrfs enable quota`
  39. 39. OVERLAY/XFS + GROOTFS + GARDEN runC 6. February 2017
  40. 40. Why Overlay+XFS? ✘ Maturity ✘ Overlay for layering ✘ XFS for quotas
  41. 41. XFS folder app1/ Overlay mount upperdir: app1/diff lowerdirs: layer1:layer2:layer3 App 1 XFS folder app2/ Overlay mount upperdir: app2/diff lowerdirs: layer1:layer2:layer3 App 2
  42. 42. The Future
  43. 43. What’s Next? ✘ EXT4 Kernel 4.5 in Stemcell Match host FS ✘ ShiftFS On the fly user mappings No translation layer
  44. 44. Conclusion ✘ Nothing is forever ✘ There are always risks Agility is key ✘ Focus!
  45. 45. thanks! Any questions? callisto13 / cberesford@pivotal.io tscolari / tscolari@pivotal.io Slide template by SlidesCarnival

×