The challenge of application 
distribution 
Introduction to Docker
Credits & references 
Intros slides borrowed from 
• dotCloud’s « Docker introduction » slideshow 
• AWS re:Invent 2014 se...
Agenda 
• The application distribution challenge 
• The current solutions 
• Introduction to Docker, Containers, and the M...
The challenge
The challenge: apps have fundamentally changed 
~2000 2014 
Long lived Development is iterative and 
constant 
Monolithic ...
Everybody already heard/said it:
The challenge: apps have fundamentally changed 
Static website 
User DB 
Redis + redis-sentinel 
Web frontend 
Queue Analy...
The Matrix From Hell 
Static website 
Web frontend 
Background workers 
User DB 
Analytics DB 
Queue 
? ? ? ? ? ? ? 
? ? ?...
The current solutions
Current solutions 
• ‘Configuration’ tools like Ansible, Chef, Puppet, … 
• VM tools : Vagrant 
• Container tools: Google’...
Goods 
Multiplicity of 
transporting/storing 
Multipilicity of 
methods for 
Do I worry about 
how goods 
interact (e.g. 
...
Also a matrix from hell 
? ? ? ? ? ? ? 
? ? ? ? ? ? ? 
? ? ? ? ? ? ? 
? ? ? ? ? ? ? 
? ? ? ? ? ? ? 
? ? ? ? ? ? ?
Multiplicity of 
Goods 
Multiplicity of 
methods for 
transporting/storin 
g 
Do I worry about 
how goods 
interact (e.g. ...
Containers applied to software
Static website User DB Web frontend Queue Analytics DB 
Dev VM 
QA server Public Cloud 
Contributor’s 
laptop 
Docker is a...
Static website 
Web frontend 
Background workers 
User DB 
Analytics DB 
Queue 
Dev VM QA Server 
Single Prod 
Server 
Ons...
Good for developers 
• Build once…(finally) run anywhere* 
• A clean, safe, hygienic and portable runtime environment for ...
Good for (Dev)Ops 
• Configure once…run anything 
• Make the entire lifecycle more efficient, consistent, and repeatable 
...
Why it works—separation of concerns 
• Dan the Developer 
– Worries about what’s “inside” the 
container 
• His code 
• Hi...
Under the hood 
• To learn more about how it works, some background: 
–Linux namespaces 
• PID 
• IPTables 
• UTS (~hostna...
Containers vs. VMs 
App 
A 
Bins 
/ 
Libs 
Bins 
/ 
Libs 
Bins 
/ 
Libs 
Hypervisor (Type 2) 
Host OS 
Server 
Guest 
OS 
...
Why are Docker containers lightweight? 
VMs Containers 
App 
A 
Bins 
/ 
Libs 
Original App 
(No OS to take 
up space, res...
Containers, images, …
Terminology 
Docker concept OOP analogy 
Dockerfile Source code 
Image Class 
Container Instance
What are the basics of the Docker system? 
Dockerfile 
For 
A 
Source 
Code 
Repositor 
y 
Docker Engine 
Docker 
Containe...
Changes and Updates 
App 
A’’ 
Bins 
/ 
Libs 
Docker Engine 
Docker 
Container 
Image 
Registry 
Push 
Update 
Docker Engi...
A new open source project… or the next big thing? 
• Compatible with Jenkins, Travis, Chef, Puppet, Vagrant and OpenStack…...
Ecosystem Support 
• Operating systems 
– Virtually any distribution with a 2.6.32+ kernel 
– Red Hat/Docker collaboration...
Use Cases 
• Ted Dziuba on the Use of Docker for Continuous Integration at Ebay Now 
– https://speakerdeck.com/teddziuba/d...
Use Cases—From the Community 
Use Case Examples Link 
Clusters Building a MongoDB cluster using docker http://bit.ly/1acbj...
How to build an image? 
Two ways to create an image: 
• You work in a container, then commit it for exploratory work 
• Yo...
1 container ?= (1process|1task|other) ? 
Best practices appear 
• Do not force you to 1 process per container 
• Ideal cas...
Containers are isolated… 
then how to make them talk to each other? 
• Private network between containers 
• Linking to ma...
How to handle data persistence 
• Data only containers, sharing data volumes 
• Mounting this data volumes in the running ...
How to handle data persistence 
How data volume works
How to handle data persistence 
How data volume works
Demo time! 
• Quick look at the Docker commands 
• Isolation demo 
• Multi containers demo: installing and launching Go Ma...
Warning 
•Docker is great 
–It reduces dependencies, provides isolated and 
controlled context 
•But it brings new tools, ...
Some references 
• Start by the online tutorial and documentation, they are 
great! 
• Here’s some links kept while learni...
More references 
First steps 
• dotScale 2013 - Solomon Hykes - Why we built Docker (video) 
• Twitter University – Solomo...
Tools for dev, day-to-day 
• For everyone 
• On Mac 
– Kitematic UI to manage your images/containers 
• On Windows 
– Try ...
Thank you
The challenge of application distribution - Introduction to Docker (2014 dec 03)
Prochain SlideShare
Chargement dans…5
×

The challenge of application distribution - Introduction to Docker (2014 dec 03)

984 vues

Publié le

Live recording with the demos: https://www.youtube.com/watch?v=0XRcmJEiZOM

Contents
- The application distribution challenge
- The current solutions
- Introduction to Docker, Containers, and the Matrix from Hell
- Why people care: Separation of Concerns
- Technical Discussion
- Ecosystem, momentum
- How to build Docker images
- How to make containers talk to each other, how to handle data persistence
- Demo 1: isolation
- Demo 2: real case - installing Go Math! Academy, tail –f containers, unit tests

Publié dans : Logiciels
0 commentaire
4 j’aime
Statistiques
Remarques
  • Soyez le premier à commenter

Aucun téléchargement
Vues
Nombre de vues
984
Sur SlideShare
0
Issues des intégrations
0
Intégrations
5
Actions
Partages
0
Téléchargements
36
Commentaires
0
J’aime
4
Intégrations 0
Aucune incorporation

Aucune remarque pour cette diapositive
  • Placeholder for demos, has I always click next! Switch to shell now
  • The challenge of application distribution - Introduction to Docker (2014 dec 03)

    1. 1. The challenge of application distribution Introduction to Docker
    2. 2. Credits & references Intros slides borrowed from • dotCloud’s « Docker introduction » slideshow • AWS re:Invent 2014 session APP303, « Lighting Fast Deploys with Docker Containers and AWS »
    3. 3. Agenda • The application distribution challenge • The current solutions • Introduction to Docker, Containers, and the Matrix from Hell • Why people care: Separation of Concerns • Technical Discussion • Ecosystem, momentum • How to build Docker images • How to make containers talk to each other, how to handle data persistence • Demo 1: isolation • Demo 2: real case - installing Go Math!, tail –f containers, unit tests
    4. 4. The challenge
    5. 5. The challenge: apps have fundamentally changed ~2000 2014 Long lived Development is iterative and constant Monolithic and built on a single stack Built from loosely coupled components Deployed to a single server Deployed to a multitude of servers
    6. 6. Everybody already heard/said it:
    7. 7. The challenge: apps have fundamentally changed Static website User DB Redis + redis-sentinel Web frontend Queue Analytics DB Background workers API endpoint nginx 1.5 + modsecurity + openssl + bootstrap 2 postgresql + pgv8 + v8 hadoop + hive + thrift + OpenJDK Ruby + Rails + sass + Unicorn Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client Development VM QA server Public Cloud Disaster recovery Contributor’s laptop Production Servers Multiplicity of Stacks Multiplicity of hardware environments Production Cluster Customer Data Center Do services and apps interact appropriately? Can I migrate smoothly and quickly?
    8. 8. The Matrix From Hell Static website Web frontend Background workers User DB Analytics DB Queue ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Dev VM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’s laptop Customer Servers
    9. 9. The current solutions
    10. 10. Current solutions • ‘Configuration’ tools like Ansible, Chef, Puppet, … • VM tools : Vagrant • Container tools: Google’s LMCTFY
    11. 11. Goods Multiplicity of transporting/storing Multipilicity of methods for Do I worry about how goods interact (e.g. coffee beans next to spices) Can I transport quickly and smoothly (e.g. from boat to train to truck) Cargo Transport Pre-1960
    12. 12. Also a matrix from hell ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
    13. 13. Multiplicity of Goods Multiplicity of methods for transporting/storin g Do I worry about how goods interact (e.g. coffee beans next to spices) Can I transport quickly and smoothly (e.g. from boat to train to truck) Solution: Intermodal Shipping Container …in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.
    14. 14. Containers applied to software
    15. 15. Static website User DB Web frontend Queue Analytics DB Dev VM QA server Public Cloud Contributor’s laptop Docker is a shipping container system for code Multiplicity of Stacks Multiplicity of hardware environments Production Cluster Customer Data Center Do services and apps interact appropriately? Can I migrate smoothly and quickly …that can be manipulated using standard operations and run consistently on virtually any hardware platform An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
    16. 16. Static website Web frontend Background workers User DB Analytics DB Queue Dev VM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’s laptop Customer Servers Docker eliminates the matrix from Hell
    17. 17. Good for developers • Build once…(finally) run anywhere* • A clean, safe, hygienic and portable runtime environment for your app. • No worries about missing dependencies, packages and other pain points during subsequent deployments. • Run each app in its own isolated container, so you can run various versions of libraries and other dependencies for each app without worrying • Automate testing, integration, packaging…anything you can script • Reduce/eliminate concerns about compatibility on different platforms, either your own or your customers. • Cheap, zero-penalty containers to deploy services? A VM without the overhead of a VM? Instant replay and reset of image snapshots? That’s the power of Docker * Anywhere = any x86 server running a modern Linux kernel (3.2+ generally. 2.6.32+ for RHEL 6.5+, Fedora, & related)
    18. 18. Good for (Dev)Ops • Configure once…run anything • Make the entire lifecycle more efficient, consistent, and repeatable • Increase the quality of code produced by developers. • Eliminate inconsistencies between development, test, production, and customer environments • Support segregation of duties • Significantly improves the speed and reliability of continuous deployment and continuous integration systems • Because the containers are so lightweight, address significant performance, costs, deployment, and portability issues normally associated with VMs
    19. 19. Why it works—separation of concerns • Dan the Developer – Worries about what’s “inside” the container • His code • His Libraries • His Package Manager • His Apps • His Data – All Linux servers look the same • Oscar the Ops Guy • Worries about what’s “outside” the container • Logging • Remote access • Monitoring • Network config • All containers start, stop, copy, attach, migrate, etc. the same way
    20. 20. Under the hood • To learn more about how it works, some background: –Linux namespaces • PID • IPTables • UTS (~hostname) • Mount • IPC (inter-process) –CGroups –LXC Containers
    21. 21. Containers vs. VMs App A Bins / Libs Bins / Libs Bins / Libs Hypervisor (Type 2) Host OS Server Guest OS App A’ Gues t OS App B Gues t OS App A’ Docker Host OS Server App A Bins/Libs Bins/Libs App B App B’ App B’ App B’ VM Container Containers are isolated, but share OS and, where appropriate, bins/libraries Guest OS Guest OS …result is significantly faster deployment, much less overhead, easier migration, faster restart
    22. 22. Why are Docker containers lightweight? VMs Containers App A Bins / Libs Original App (No OS to take up space, resources, or require restart) App Δ Bins / App A Bins / Libs App A’ Bins / Libs Gues t OS Modified App Copy on write capabilities allow us to only save the diffs between container A and container A’ VMs App A Bins / Libs Guest OS Every app, every copy of an app, and every slight modification of the app requires a new virtual server App A Copy of App No OS. Can Share bins/libs Guest OS Guest OS
    23. 23. Containers, images, …
    24. 24. Terminology Docker concept OOP analogy Dockerfile Source code Image Class Container Instance
    25. 25. What are the basics of the Docker system? Dockerfile For A Source Code Repositor y Docker Engine Docker Container Image Registry Build Docker Host 2 OS (Linux) Container A Container B Container C Container A Push Search Pull Run Host 1 OS (Linux)
    26. 26. Changes and Updates App A’’ Bins / Libs Docker Engine Docker Container Image Registry Push Update Docker Engine App A Bins / Libs App Δ Bins / Base Container Image Host is now running A’’ Container Mod A’’ App Δ Bins / App A Bins / Libs Bins / Host running A wants to upgrade to A’’. Requests update. Gets only diffs Container Mod A’
    27. 27. A new open source project… or the next big thing? • Compatible with Jenkins, Travis, Chef, Puppet, Vagrant and OpenStack… • Ecosystem: lots of tools, even by serious contributors (Google – Kubernetes, cAdvisor - NetFlix …), new IAAS provider emerge (like Tutum), • Supported by Amazon, Google, Microsoft, Rackspace, Digital Ocean, Joyent, RedHat, … • Momentum... See pull requests numbers and frequency to get an idea • Used in production… even by big players (HP, Ebay, New Relic, Etsy, Yelp, Spotify, …)
    28. 28. Ecosystem Support • Operating systems – Virtually any distribution with a 2.6.32+ kernel – Red Hat/Docker collaboration to make work across RHEL 6.4+, Fedora, and other members of the family (2.6.32 +) – CoreOS—Small core OS purpose built with Docker • OpenStack – Docker integration into NOVA (& compatibility with Glance, Horizon, etc.) accepted for Havana release • Private PaaS – OpenShift – Solum (Rackspace, OpenStack) – Other TBA • Public PaaS – Deis, Voxoz, Cocaine (Yandex), Baidu PaaS • Public IaaS – Native support in Rackspace, Digital Ocean,+++ – AMI (or equivalent) available for AWS & other • DevOps Tools – Integrations with Chef, Puppet, Jenkins, Travis, Salt, Ansible +++ • Orchestration tools – Mesos, Heat, ++ – Shipyard & others purpose built for Docker • Applications – 1000’s of Dockerized applications available at index.docker.io
    29. 29. Use Cases • Ted Dziuba on the Use of Docker for Continuous Integration at Ebay Now – https://speakerdeck.com/teddziuba/docker-at-ebay – http://www.youtube.com/watch?feature=player_embedded&v=0Hi0W4gX--4 • Sasha Klizhentas on use of Docker at Mailgun/Rackspace • http://www.youtube.com/watch?feature=player_embedded&v=CMC3xdAo9RI • Sebastien Pahl on use of Docker at CloudFlare • http://www.youtube.com/watch?feature=player_embedded&v=-Lj3jt_-3r0 • Cambridge HealthCare • http://blog.howareyou.com/post/62157486858/continuous-delivery-with-docker-and- jenkins-part-i • Red Hat Openshift and Docker • https://www.openshift.com/blogs/technical-thoughts-on-openshift-and-docker
    30. 30. Use Cases—From the Community Use Case Examples Link Clusters Building a MongoDB cluster using docker http://bit.ly/1acbjZf Production Quality MongoDB Setup with Docker http://bit.ly/15CaiHb Wildfly cluster using Docker on Fedora http://bit.ly/1bClX0O Build your own PaaS OpenSource PaaS built on Docker, Chef, and Heroku Buildpacks http://deis.io Web Based JiffyLab – web based environment for the instruction, or lightweight use of, Environment for Python and UNIX shell Instruction http://bit.ly/12oaj2K Easy Application Deployment Deploy Java Apps With Docker = Awesome http://bit.ly/11BCvvu How to put your development environment on docker http://bit.ly/1b4XtJ3 Running Drupal on Docker http://bit.ly/15MJS6B Installing Redis on Docker http://bit.ly/16EWOKh Create Secure Sandboxes Docker makes creating secure sandboxes easier than ever http://bit.ly/13mZGJH Create your own SaaS Memcached as a Service http://bit.ly/11nL8vh Automated Application Deployment Multi-cloud Deployment with Docker http://bit.ly/1bF3CN6 Continuous Integration and Deployment Next Generation Continuous Integration & Deployment with dotCloud’s Docker and Strider http://bit.ly/ZwTfoy Testing Salt States Rapidly With Docker http://bit.ly/1eFBtcm Lightweight Desktop Virtualization Docker Desktop: Your Desktop Over SSH Running Inside Of A Docker Container http://bit.ly/14RYL6x
    31. 31. How to build an image? Two ways to create an image: • You work in a container, then commit it for exploratory work • You build a new image from a Dockerfile for automated, repeatable process
    32. 32. 1 container ?= (1process|1task|other) ? Best practices appear • Do not force you to 1 process per container • Ideal case: 1 ‘task’ per container, with separation of concerns. • The task might require 1 single process • Or 0 (for data-only containers) • Or several (ex: nginx+php-fpm, monitoring, …)
    33. 33. Containers are isolated… then how to make them talk to each other? • Private network between containers • Linking to make them easy to talk to a friend container • Expose ports • Mount volumes (rw or ro) • Advanced/not so trivial other solutions –Service discovery –Enter into the container’s namespace
    34. 34. How to handle data persistence • Data only containers, sharing data volumes • Mounting this data volumes in the running containers
    35. 35. How to handle data persistence How data volume works
    36. 36. How to handle data persistence How data volume works
    37. 37. Demo time! • Quick look at the Docker commands • Isolation demo • Multi containers demo: installing and launching Go Math! Academy in seconds. • Running unit tests in a container
    38. 38. Warning •Docker is great –It reduces dependencies, provides isolated and controlled context •But it brings new tools, and new technologies, hence a potential complexity increase –libswarm, systemd, etcd, ambassadord, fleet, etc
    39. 39. Some references • Start by the online tutorial and documentation, they are great! • Here’s some links kept while learning… but so much available! –Eight Docker Development Patterns –Panamax and Docker and Some Best Practices –Tutum blog –The 5 Most Important Things I’ve Learned From Using Docker –Understanding Docker Volumes –Docker's Youtube videos
    40. 40. More references First steps • dotScale 2013 - Solomon Hykes - Why we built Docker (video) • Twitter University – Solomon Hykes – Introduction to Docker (video) Going further • Docker layers cost (blog), on how to optimize your images layer size. • Twitter University - Docker at Spotify (video) • Top 10 open-source Docker developer tools (blog, incomplete but gives ideas) • Docker In-depth: Volumes (blog) Food for thought / Stimulate your imagination • AWS re:Invent 2014 - Lightning fast deploys with Docker containers and AWS (video, slides) • FutureStack14 - Docker and the Future of Modern Software (New Relic blog, video) • AWS re:Invent 2014 - Infrastructure as Code (video, more related to Cloudformation than to Docker but very inspiring if you consider the two technologies) • Alternative? Rocket by CoreOS… started a debate (like Joyent answer)
    41. 41. Tools for dev, day-to-day • For everyone • On Mac – Kitematic UI to manage your images/containers • On Windows – Try the new XShell5 beta to easily deal with all your current containers • So many others… and new ones every week • Fig (Fast isolated environments, by Docker) repo, site • A Better Development Environment with Docker and Fig (blog) • Lots of tools to manage multi containers apps: Kubernetes, Shipyard, Panamax,
    42. 42. Thank you

    ×