SlideShare une entreprise Scribd logo
1  sur  22
Télécharger pour lire hors ligne
Linux Plumbers Conference 2014, Düsseldorf 
Docker and the Linux kernel 
Cristian S., Docker Inc. 
cristian@docker.com
What is Docker ?
The Matrix From Hell
Another Matrix From Hell
Solution: 
the intermodal shipping container
Solved!
Solution to the deployment problem: 
the Linux container
Solved!
High level overview 
• Uses namespaces & cgroups 
• Runs on mainline kernels 
• Lower overhead than VMs 
• Can run a full system with an init or a 
single service/process 
• Snapshot based approach to build 
one image on top of another
Docker's code 
• Licensed under the Apache 2 license 
• There's no paid/premium/commercial 
version 
• Docker and its code are free and will 
be free 
• Lives at: 
https://github.com/docker/docker/
Use cases 
• Application development, 
testing, packaging & deployment 
• PaaS/SaaS/cloud infrastructure 
• Application & service isolation 
• Stress testing & benchmarking 
(including the Linux kernel)
Under the hood 
• Exec drivers provide the execution environment 
(virtualization/container tech) 
– native (libcontainer based) and LXC 
– are platform dependent 
– native is the default 
• Graph drivers are the storage providers 
– aufs, devicemapper, btrfs and vfs 
– vfs shouldn't be used (only used by the tests by 
default) 
– PRs open on GitHub for ZFS and OverlayFS 
• Existing graph & exec drivers only support Linux
Kernel requirements 
• Kernel 3.8 is the absolute minimum 
(except RHEL's 2.6.32) 
• Stable & supported kernels >= 3.10 are 
recommended 
• BTRFS has special requirements 
• Kernel configuration can be checked using 
https://raw.githubusercontent.com/docker/ 
docker/master/contrib/check-config.sh
AUFS graph driver stability & 
performance 
• Operations with containers are faster than devicemapper & 
(sometimes) btrfs 
• Known problems 
– Stale NFS file handle, cap_set_file, invalid argument on mount 
– Causes troubles on btrfs, remote file systems and many file 
systems which aren't ext3/ext4 
– Direct IO problems & poor performance for IO intensive workloads 
• Limitations 
– Requires aufs-tools for auplink to dereference hard links 
– No support for hard links across layers 
– Limit of 127 layers 
– Can't be used on Fedora/RHEL/all systems which don't apply AUFS 
patches and don't ship aufs-tools 
• AUFS is developed outside of the mainline kernel tree 
• Update the kernel using distro updates
Devicemapper graph driver stability 
& performance 
• Uses loopback mounted block devices by default 
• Allows EXT4 or XFS to be used 
• Known problems 
– EBUSY errors (fix to be tested) 
– Potential file system corruption bug with ext4 
● Might be caused by loopback mounted block devices 
– Older kernels: space not freed when freed on the file 
system, various kernel errors and problems 
• Limitations 
– Devicemapper requires configuration of the storage to avoid 
the use of loopback mounted block devices 
• Update the kernel using distro updates
btrfs graph driver stability & 
performance 
• BTRFS bugs are a problem for Docker 
• Using the RAID like features of BTRFS is likely to cause data loss 
• Known problems 
– Data can be corrupted if exotic mount options are used 
– Kernel 3.8 & older than the latest stable kernels can cause data loss & 
corruption 
– The file system becomes slower the more data is written & stored on it 
– Performance degrades quickly & fragmentation is a problem 
– Balancing the file system to fix fragmentation could trigger some bugs 
• BTRFS is used automatically if Docker's root folder is on BTRFS 
• Update the kernel using distro updates 
• Using the latest minor version of supported kernels is recommended
Performance & stability 
• Generally speaking, the kernel has been stable 
– Fixes have been made to file systems, namespaces, cgroups, 
netfilter, aufs, btrfs, devicemapper and other kernel components 
– Recent kernels and the newest minor versions of LTS kernels have 
fixed a lot of problems, including devicemapper and btrfs problems 
– Some PID 1 issues are still being discussed 
– Changes around the cgroups and the namespaces have been 
made; this has also helped improve stability for containers 
• Performance is actively being studied 
– Docker's code has been improved to use less memory & be faster 
– Performance needs to be studied on the kernel side to achieve 
better scalability by a) making fewer syscalls in Docker where 
possible, b) making those syscalls faster in the kernel
Kernel recommendations 
• Avoid running kernels no longer supported 
by your Linux distribution (kernel 3.8 lts-raring 
from Ubuntu 12.04.x) 
• Open Docker issues on github; post errors 
(full kernel panics, btrfs check output, 
Docker daemon logs, `docker info`, 
`docker version` and `uname -a` output) 
when encountering errors 
• File bug reports with your distribution
What can Docker do for the kernel? 
• Can be used to test the kernel's stability and 
evaluate its performance 
• Changes made to the kernel can be tested 
easily against Docker 
• Can be used to test the running kernel to 
avoid breaking user space compatibility 
• Makes it simple to mix workloads for stress 
testing and performance testing 
• Exposes some hard to trigger kernel problems
What can a kernel developer do 
with Docker? 
• Find bottlenecks in the kernel's code 
• Run a KVM VM in a container 
• Stress testing for file systems, network, 
namespaces and the kernel in general 
• Large scale testing with containers 
• Hardware testing in containers 
• Network testing with advanced 
topologies
How can someone contribute to Docker? 
What about kernel developers? 
• You don't necessarily have to write code 
• Providing feedback, doing code review or 
telling the right kernel developer to do 
that are also useful helpful 
• Suggestions on how to debug specific 
kernel bugs and bugs in general are 
welcome 
• Contributing to Docker is also helping the 
kernel because Docker relies on the kernel
Thank you! Questions? 
http://docker.com 
@docker

Contenu connexe

Tendances

Why Docker
Why DockerWhy Docker
Why DockerdotCloud
 
Introduction to Docker - 2017
Introduction to Docker - 2017Introduction to Docker - 2017
Introduction to Docker - 2017Docker, Inc.
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to DockerAditya Konarde
 
Docker Networking Overview
Docker Networking OverviewDocker Networking Overview
Docker Networking OverviewSreenivas Makam
 
Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes VMware Tanzu
 
Kubernetes
KubernetesKubernetes
Kuberneteserialc_w
 
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...Simplilearn
 
Docker introduction & benefits
Docker introduction & benefitsDocker introduction & benefits
Docker introduction & benefitsAmit Manwade
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker IntroductionRobert Reiz
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetesRishabh Indoria
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with DockerRavindu Fernando
 
Introduction to docker
Introduction to dockerIntroduction to docker
Introduction to dockerInstruqt
 
Everything You Need To Know About Persistent Storage in Kubernetes
Everything You Need To Know About Persistent Storage in KubernetesEverything You Need To Know About Persistent Storage in Kubernetes
Everything You Need To Know About Persistent Storage in KubernetesThe {code} Team
 
Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad Docker, Inc.
 

Tendances (20)

Why Docker
Why DockerWhy Docker
Why Docker
 
Docker, LinuX Container
Docker, LinuX ContainerDocker, LinuX Container
Docker, LinuX Container
 
Introduction to Docker - 2017
Introduction to Docker - 2017Introduction to Docker - 2017
Introduction to Docker - 2017
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
Docker Networking Overview
Docker Networking OverviewDocker Networking Overview
Docker Networking Overview
 
Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes
 
Kubernetes
KubernetesKubernetes
Kubernetes
 
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
Docker introduction & benefits
Docker introduction & benefitsDocker introduction & benefits
Docker introduction & benefits
 
Docker Networking
Docker NetworkingDocker Networking
Docker Networking
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Docker
DockerDocker
Docker
 
DevOps with Kubernetes
DevOps with KubernetesDevOps with Kubernetes
DevOps with Kubernetes
 
Maven tutorial
Maven tutorialMaven tutorial
Maven tutorial
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with Docker
 
Introduction to docker
Introduction to dockerIntroduction to docker
Introduction to docker
 
Everything You Need To Know About Persistent Storage in Kubernetes
Everything You Need To Know About Persistent Storage in KubernetesEverything You Need To Know About Persistent Storage in Kubernetes
Everything You Need To Know About Persistent Storage in Kubernetes
 
Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad
 

Similaire à Docker and the Linux Kernel

Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetesDongwon Kim
 
Docking postgres
Docking postgresDocking postgres
Docking postgresrycamor
 
Linux: Everyting-as-a-service
Linux: Everyting-as-a-serviceLinux: Everyting-as-a-service
Linux: Everyting-as-a-serviceRohit Sansiya
 
Hands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestrationHands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestrationAmir Hossein Sorouri
 
Lessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersLessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersBlueData, Inc.
 
Docker introduction
Docker introductionDocker introduction
Docker introductionWalter Liu
 
Central Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdownCentral Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdownAndrew Denner
 
Gentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile EverythingGentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile EverythingDonnie Berkholz
 
Building community with CentOS Stream
Building community with CentOS StreamBuilding community with CentOS Stream
Building community with CentOS StreamDavide Cavalca
 
Learning of docker storage driver (container file system)
Learning of docker storage driver  (container file system)Learning of docker storage driver  (container file system)
Learning of docker storage driver (container file system)Deepak Kumar
 
Getting Started with Docker
Getting Started with DockerGetting Started with Docker
Getting Started with DockerGeeta Vinnakota
 
State of the Container Ecosystem
State of the Container EcosystemState of the Container Ecosystem
State of the Container EcosystemVinay Rao
 
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...Andrew Denner
 
Unraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudUnraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudSalman Baset
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityPhil Estes
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on DockerDataWorks Summit
 
Java in containers
Java in containersJava in containers
Java in containersMartin Baez
 
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...The Linux Foundation
 

Similaire à Docker and the Linux Kernel (20)

Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
 
Docking postgres
Docking postgresDocking postgres
Docking postgres
 
Linux: Everyting-as-a-service
Linux: Everyting-as-a-serviceLinux: Everyting-as-a-service
Linux: Everyting-as-a-service
 
Hands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestrationHands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestration
 
Docker Dojo
Docker DojoDocker Dojo
Docker Dojo
 
Lessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersLessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker Containers
 
First steps on CentOs7
First steps on CentOs7First steps on CentOs7
First steps on CentOs7
 
Docker introduction
Docker introductionDocker introduction
Docker introduction
 
Central Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdownCentral Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdown
 
Gentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile EverythingGentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile Everything
 
Building community with CentOS Stream
Building community with CentOS StreamBuilding community with CentOS Stream
Building community with CentOS Stream
 
Learning of docker storage driver (container file system)
Learning of docker storage driver  (container file system)Learning of docker storage driver  (container file system)
Learning of docker storage driver (container file system)
 
Getting Started with Docker
Getting Started with DockerGetting Started with Docker
Getting Started with Docker
 
State of the Container Ecosystem
State of the Container EcosystemState of the Container Ecosystem
State of the Container Ecosystem
 
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
 
Unraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudUnraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production Cloud
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker Security
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
 
Java in containers
Java in containersJava in containers
Java in containers
 
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
 

Plus de Docker, Inc.

Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Docker, Inc.
 
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildDocker, Inc.
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSDocker, Inc.
 
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXSecuring Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXDocker, Inc.
 
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeHow To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeDocker, Inc.
 
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDistributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDocker, Inc.
 
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubThe First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubDocker, Inc.
 
Monitoring in a Microservices World
Monitoring in a Microservices WorldMonitoring in a Microservices World
Monitoring in a Microservices WorldDocker, Inc.
 
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...Docker, Inc.
 
Predicting Space Weather with Docker
Predicting Space Weather with DockerPredicting Space Weather with Docker
Predicting Space Weather with DockerDocker, Inc.
 
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeBecome a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeDocker, Inc.
 
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryHow to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryDocker, Inc.
 
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Docker, Inc.
 
Kubernetes at Datadog Scale
Kubernetes at Datadog ScaleKubernetes at Datadog Scale
Kubernetes at Datadog ScaleDocker, Inc.
 
Labels, Labels, Labels
Labels, Labels, Labels Labels, Labels, Labels
Labels, Labels, Labels Docker, Inc.
 
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelDocker, Inc.
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSDocker, Inc.
 
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...Docker, Inc.
 
Developing with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDeveloping with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDocker, Inc.
 

Plus de Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience
 
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXSecuring Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINX
 
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeHow To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and Compose
 
Hands-on Helm
Hands-on Helm Hands-on Helm
Hands-on Helm
 
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDistributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at Salesforce
 
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubThe First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker Hub
 
Monitoring in a Microservices World
Monitoring in a Microservices WorldMonitoring in a Microservices World
Monitoring in a Microservices World
 
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
 
Predicting Space Weather with Docker
Predicting Space Weather with DockerPredicting Space Weather with Docker
Predicting Space Weather with Docker
 
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeBecome a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio Code
 
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryHow to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container Registry
 
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!
 
Kubernetes at Datadog Scale
Kubernetes at Datadog ScaleKubernetes at Datadog Scale
Kubernetes at Datadog Scale
 
Labels, Labels, Labels
Labels, Labels, Labels Labels, Labels, Labels
Labels, Labels, Labels
 
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
 
Developing with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDeveloping with Docker for the Arm Architecture
Developing with Docker for the Arm Architecture
 

Dernier

Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.IPLOOK Networks
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1DianaGray10
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingMAGNIntelligence
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInThousandEyes
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024Brian Pichman
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Muhammad Tiham Siddiqui
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTxtailishbaloch
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch TuesdayIvanti
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxNeo4j
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTopCSSGallery
 

Dernier (20)

Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced Computing
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch Tuesday
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development Companies
 

Docker and the Linux Kernel

  • 1. Linux Plumbers Conference 2014, Düsseldorf Docker and the Linux kernel Cristian S., Docker Inc. cristian@docker.com
  • 5. Solution: the intermodal shipping container
  • 7. Solution to the deployment problem: the Linux container
  • 9. High level overview • Uses namespaces & cgroups • Runs on mainline kernels • Lower overhead than VMs • Can run a full system with an init or a single service/process • Snapshot based approach to build one image on top of another
  • 10. Docker's code • Licensed under the Apache 2 license • There's no paid/premium/commercial version • Docker and its code are free and will be free • Lives at: https://github.com/docker/docker/
  • 11. Use cases • Application development, testing, packaging & deployment • PaaS/SaaS/cloud infrastructure • Application & service isolation • Stress testing & benchmarking (including the Linux kernel)
  • 12. Under the hood • Exec drivers provide the execution environment (virtualization/container tech) – native (libcontainer based) and LXC – are platform dependent – native is the default • Graph drivers are the storage providers – aufs, devicemapper, btrfs and vfs – vfs shouldn't be used (only used by the tests by default) – PRs open on GitHub for ZFS and OverlayFS • Existing graph & exec drivers only support Linux
  • 13. Kernel requirements • Kernel 3.8 is the absolute minimum (except RHEL's 2.6.32) • Stable & supported kernels >= 3.10 are recommended • BTRFS has special requirements • Kernel configuration can be checked using https://raw.githubusercontent.com/docker/ docker/master/contrib/check-config.sh
  • 14. AUFS graph driver stability & performance • Operations with containers are faster than devicemapper & (sometimes) btrfs • Known problems – Stale NFS file handle, cap_set_file, invalid argument on mount – Causes troubles on btrfs, remote file systems and many file systems which aren't ext3/ext4 – Direct IO problems & poor performance for IO intensive workloads • Limitations – Requires aufs-tools for auplink to dereference hard links – No support for hard links across layers – Limit of 127 layers – Can't be used on Fedora/RHEL/all systems which don't apply AUFS patches and don't ship aufs-tools • AUFS is developed outside of the mainline kernel tree • Update the kernel using distro updates
  • 15. Devicemapper graph driver stability & performance • Uses loopback mounted block devices by default • Allows EXT4 or XFS to be used • Known problems – EBUSY errors (fix to be tested) – Potential file system corruption bug with ext4 ● Might be caused by loopback mounted block devices – Older kernels: space not freed when freed on the file system, various kernel errors and problems • Limitations – Devicemapper requires configuration of the storage to avoid the use of loopback mounted block devices • Update the kernel using distro updates
  • 16. btrfs graph driver stability & performance • BTRFS bugs are a problem for Docker • Using the RAID like features of BTRFS is likely to cause data loss • Known problems – Data can be corrupted if exotic mount options are used – Kernel 3.8 & older than the latest stable kernels can cause data loss & corruption – The file system becomes slower the more data is written & stored on it – Performance degrades quickly & fragmentation is a problem – Balancing the file system to fix fragmentation could trigger some bugs • BTRFS is used automatically if Docker's root folder is on BTRFS • Update the kernel using distro updates • Using the latest minor version of supported kernels is recommended
  • 17. Performance & stability • Generally speaking, the kernel has been stable – Fixes have been made to file systems, namespaces, cgroups, netfilter, aufs, btrfs, devicemapper and other kernel components – Recent kernels and the newest minor versions of LTS kernels have fixed a lot of problems, including devicemapper and btrfs problems – Some PID 1 issues are still being discussed – Changes around the cgroups and the namespaces have been made; this has also helped improve stability for containers • Performance is actively being studied – Docker's code has been improved to use less memory & be faster – Performance needs to be studied on the kernel side to achieve better scalability by a) making fewer syscalls in Docker where possible, b) making those syscalls faster in the kernel
  • 18. Kernel recommendations • Avoid running kernels no longer supported by your Linux distribution (kernel 3.8 lts-raring from Ubuntu 12.04.x) • Open Docker issues on github; post errors (full kernel panics, btrfs check output, Docker daemon logs, `docker info`, `docker version` and `uname -a` output) when encountering errors • File bug reports with your distribution
  • 19. What can Docker do for the kernel? • Can be used to test the kernel's stability and evaluate its performance • Changes made to the kernel can be tested easily against Docker • Can be used to test the running kernel to avoid breaking user space compatibility • Makes it simple to mix workloads for stress testing and performance testing • Exposes some hard to trigger kernel problems
  • 20. What can a kernel developer do with Docker? • Find bottlenecks in the kernel's code • Run a KVM VM in a container • Stress testing for file systems, network, namespaces and the kernel in general • Large scale testing with containers • Hardware testing in containers • Network testing with advanced topologies
  • 21. How can someone contribute to Docker? What about kernel developers? • You don't necessarily have to write code • Providing feedback, doing code review or telling the right kernel developer to do that are also useful helpful • Suggestions on how to debug specific kernel bugs and bugs in general are welcome • Contributing to Docker is also helping the kernel because Docker relies on the kernel
  • 22. Thank you! Questions? http://docker.com @docker