In this deck from the Stanford HPC Conference, Christian Kniep from Docker, Inc. gives a tutorial on linux containers. "This tutorial provides a detailed overview of the components needed to run containerized applications and explores how distributed HPC applications can be tackled. We’ll explain the concept of Linux Containers and describe the bits and pieces participants will explore following step-by-step examples. The workshop will introduce the predominant forms of orchestration in the industry; what problems they solve and how to approach the problem. Attendees will explore the benefits and drawbacks of orchestrators first hand with their own small exemplary stack deployments. Finally the workshop will introduce how HPC and Big Data workloads can be tackled on-top of these service-oriented clusters." Watch the video: https://youtu.be/LJinZpCTyk0 Learn more: http://www.docker.com/ and http://hpcadvisorycouncil.com Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. DevOps Workflow
From 0 to kube in 60 min
Christian Kniep, v2018-02-20
Technical Account Manager, Docker Inc.http://qnib.org/devops-workflow

2. Motivation
Iteration barriers

3. ClusterLaptop
Works on my Laptop!
Why is DevOps not enough.
ProdDevelopment
Dev
Artifact: code
Env: interactive
Ops
Artifact: pkg/img
Env: automated
DevOps
Release/Integration
Release
Artifact: pkg/img
Env: automated
DevRelOps
Prod
Ops
Artifact: pkg/img
Env: automated
Development
Dev
Artifact: code
Env: interactive
DevRel*Ops
Prod
Ops
Artifact: pkg/img
Env: automated
Development
Dev
Artifact: code
Env: interactive
Q&A
Performance
Release/Integration
Release
Artifact: pkg/img
Env: automated
Experiment
Experiment
Experiment

4. ● Ship the image using a (private) Docker Registry
● Orchestrate the Image: docker-compose.yml
Build, Ship & Run
Where ‘Run’ == ‘Orchestrate’
● Build the image: Dockerfile + context -> Docker Image
For the developer the orchestration should be transparent and out of scope.
Artifacts are:
1. Dockerfile + context (git repository)
2. docker-compose.yml (git repository)

5. Application Containerization
Portability, Reproducibility
As a developer I want to work on a
webapp for DockerCon EU, which
represents a simple three tier stack.
● web: JS app I am working on
● words: API serving business logic
● db: Database backend
web
api
db
api

6. Prerequisite for Workshop
http://qnib.org/devops-workflow
Docker Desktop
- Docker4Mac / Docker4Win edge-channel (provides kubernetes)
- DockerCE for Linux
Git Repository
git clone https://github.com/ChristianKniep/k8s-wordsmith-demo.git
cd k8s-wordsmith-demo.git
git checkout update/container_to_kube
docker-compose pull
cd swarmprom ; docker-compose pull

7. #1 Building the Image(s)
First Step as a Developer

8. docker-compose build
$ export TAG=$(date +%s)
$ docker-compose build
WARNING: Some services (api) use the 'deploy' key, which will be ignored.
Building web
Step 1/12 : FROM golang:1.9.1-alpine3.6 as builder
*snip*
Successfully tagged qnib/k8s-wordsmith-web:1518955537
Building api
*snip*
Successfully tagged qnib/k8s-wordsmith-api:1518955537
Building db
*snip*
Successfully tagged qnib/k8s-wordsmith-db:1518955537

9. #2 Simple Container
One-shot Orchestration

10. docker-compose build
$ docker-compose up
WARNING: Some services (api) use the 'deploy' key, which will be ignored.
Creating network "k8swordsmithdemo_default" with the default driver
Creating k8swordsmithdemo_web_1 ... done
Creating k8swordsmithdemo_api_1 ... done
Creating k8swordsmithdemo_db_1 ... done
Attaching to k8swordsmithdemo_db_1, k8swordsmithdemo_api_1,
k8swordsmithdemo_web_1
db_1 | The files belonging to this database system will be owned by user
"postgres".
db_1 | This user must also own the server process.

11. #3 Deploying Stack in SWARM
Simple yet Powerful Orchestration

12. SWARM at a Glance
Easy to understand and reason about

13. $ cd swarmprom
$ docker stack deploy -c docker-compose.yml prom
Creating network prom_net
Creating config prom_service_rules
Creating config prom_node_name
Creating config prom_caddy_config
Creating config prom_dockerd_config
Creating config prom_node_rules
Creating config prom_task_rules
Creating service prom_caddy
Creating service prom_dockerd-exporter
Creating service prom_cadvisor
Creating service prom_grafana
Creating service prom_node-exporter
Creating service prom_prometheus
Deploy Auxiliary Stack to Monitor SWARM
Fork of: https://github.com/stefanprodan/swarmprom

14. Docker stack deploy
Using the Docker SWARM API
$ docker stack deploy -c docker-compose.yml words
Ignoring unsupported options: build
Creating network words_default
Creating service words_web
Creating service words_db
Creating service words_api
$ docker service ls --format="{{.Name}}t{{.Replicas}}t{{.Image}}t{{.Ports}}"
words_api 5/5 qnib/k8s-wordsmith-api:1518955537
words_db 1/1 qnib/k8s-wordsmith-db:1518955537
words_web 1/1 qnib/k8s-wordsmith-web:1518955537 *:8080->80/tcp
$
$ docker stack deploy -c docker-compose.yml words #2>dev/null
Updating service words_web (id: fv7z1lryqmysxhzobdjrbgecl)
image qnib/k8s-wordsmith-web:1518955537 could not be accessed on a
registry to record its digest. Each node will access
qnib/k8s-wordsmith-web:1518955537 independently, possibly leading to
different nodes running different versions of the image.

15. #4 Deploying Stack in Kubernetes
Using CustomResourceDefinitions

16. Docker stack deploy #2
Using the docker-CRD within Kube
$ docker stack rm words
Removing service words_api
Removing service words_db
Removing service words_web
Removing network words_default
$ export DOCKER_ORCHESTRATOR=kubernetes
$ docker stack deploy -c docker-compose.yml words
Stack words was created
Waiting for the stack to be stable and running...
- Service api has one container running
- Service db has one container running
- Service web has one container running
Stack words is stable and running
$

17. #5 Deploying Stack in Kubernetes
Using Kubernetes Object Files

18. Kubectl deploy #2
Using the kubectl CLI
$ export
DOCKER_ORCHESTRATOR=kubernetes
$ kubectl apply -f .
deployment "kube-api" created
service "kube-api" created
deployment "kube-db" created
service "kube-db" created
deployment "kube-web" created
service "kube-web" created
$

19. THANK YOU :)
christian.kniep@docker.com