Docker, Atomic Host and Kubernetes.

2. Docker, Atomic Host and
Kubernetes.
Red Hat
Jooho Lee
Technical Support Engineer

3. IT Trend - Cloud
• Cloud Computing Trends (Right Scale)

4. What is DevOps?
• DevOps (Developers + Operations)
• Software development methodology
• Culture
• Emphasize on communication, collaboration and integration.
• Achieve rapid release.
Developer Operations
Rapid Change StabilitySeparate workflow
Different object
Different situation

5. Key points in DevOps
• Worth & Object
• Respect
• Share value
• Share ownership
• Agreement
• Process
• Share same workflow
• Synchronize focus
• Decrease cycle time
• Tool
• Automation
• Package
• Build
• Test
Operating System
Infrastructure
Application
Platform
QA
DevOps
DevOps is not only development culture but also collaboration process
,which could develop and operate service independently in a organization

6. DevOps
• Benefit
• Innovate faster
• More responsive to business needs
• Better collaboration
• Better quality
• More frequent releases
• Necessary
• New Mindset (Open mind)
• New Tools (Provisioning, Monitoring, etc)
• New Skills (From platform to QA)

7. Docker Concept
• Share kernel (Lightweight)
• Share resources (Fast)
• Share image (Portable)
• Layered file system
(Easy commit)
• …….

8. Why DevOps prefer docker?
• Container-based Platform
• Easy/Fast deployment, build, provisioning
• Similar performance compared to bare-metal
• Low learning curve (Dockerfile use bash shell)
• Ansible(Python)/ Chef(Ruby and Erlang) / Puppet(Ruby)
• RedHat, IBM, Microsoft, CoreOS... Many companies support.

9. DevOps needs more…
• Reliable system management. (Atomic Host, etc)
• Handle network complexity (OVS, flanneld, etc)
• Management module for bigger infrastructure. (Kubernetes, etc)
Docker in real
Minimal Docker

10. What is Atomic Host?
• Trusted operating system platform
• Container-based application
• Service deployment
• End-to-End hosting architecture that’s modern, reliable and secure.
Reliable distribution OS High Secure Module
SELinux
Container Management

11. Atomic Host Feature
• Support packages
• Docker
• Flannel
• Kubernetes
• rpm-ostree
• Support various file system for Docker
• vfs
• devicemapper
• btrfs
• aufs (not recommend for production, not supported)
• Networking
• Single-host networking : Docker
• Multi-host networking : Kubernetes & Flannel

12. How to debug Atomic Host ?
• Atomic Host is minimal of RHEL system.
• Does NOT use “YUM” on Atomic Host.
• How debug or install packages?
• Red hat provide “Red Hat Enterprise Linux Atomic Tools Container Image”
• It’s Big (1G)
• Contains man pages
• Opens privileges
• May behave differently
• How?
# docker pull rhel7/rhel-tools
# atomic run rhel7/rhel-tools
[root@localhost /]#

13. What is Kubernetes?
• Greek for “pilot” or “helmsman of a ship”
• Kubernetes is an open source Container Cluster orchestration framework that
was started by Google in 2014.

14. Kubernetes do WHAT?
• Manage docker containers centrally.
• Manage nodes.
• Handle complex networking.
…..
Manage a cluster of Linux containers as a single system
to accelerate Dev and simplify Ops

15. Kubernetes Architecture
Kubectl(user commands)
Scheduling
actuator
REST
(pods,services,
rep.controllers)
Replication
Controller
Storage
(etcd)
kubelet Proxy
……
Auth
APIServer
Scheduler
Node
cadvisor
Pod Pod Pod
kubelet Proxy
……
Node
cadvisor
Pod Pod Pod

16. Main Terms
• Master
• Managing machine, which oversees one or more minions.
• Minion
• A slave that runs tasks as delegated by the user and Kubernetes master.
• Pod
• An application (or part of an application) that runs on a minion.
• Replication Controller
• Ensures that the requested number of pods are running on minions at all times.
• Label
• An arbitrary key/value pair that the Replication Controller uses for service discovery
• Service
• An endpoint that provides load balancing across a replicated group of pods
• kubectl
• The command line config tool

17. Main Components
• API Server
• REST server
• Controller Tower
• Controller Manager
• Replication Controller Management (Watches etcd)
• Scheduler
• Communicate with minions
• Decide a minion to distribute workload
• Check if the task happen.
• Kubelet
• Manage container deployments
• Ensure the state of containers (which is supposed to be in)
• Kube-proxy
• Route and forward traffic to and from containers
• ETCD
• Distributed, consistent key value store for shared configuration and service discovery

18. How to configure Kubernetes?
# Comma seperated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd_servers=http://192.168.20.10:4001"
configure
# The address on the local server to listen to.
KUBE_API_ADDRESS="--address=0.0.0.0"...
# How the replication controller and scheduler find the kube-apiserver
KUBE_MASTER="--master=192.168.20.10:8080"
apiserver
# Comma seperated list of minions
KUBELET_ADDRESSES="--machines=minion1,minion2,minion3,minion4"
controller-manager
Service Register
# systemctl enable etcd kube-apiserver kube-controller-manager kube-scheduler
Service Start
# systemctl start etcd kube-apiserver kube-controller-manager kube-scheduler
Service Check
# systemctl status etcd kube-apiserver kube-controller-manager kube-scheduler
# systemctl status etcd kube-apiserver kube-controller-manager kube-scheduler | grep active |wc –l
4
Master :
/etc/kubernetes/

19. How to configure Kubernetes?
Master
{
"Network": "172.16.0.0/12",
"SubnetLen": 24,
"Backend": { "Type": "vxlan" }
}
Create Flanneld-conf.json
Flanneld Configuration
curl -L http://localhost:4001/v2/keys/atomic01/network/config
-XPUT --data-urlencode value@flanneld-conf.json
Create key/value on ETCD

20. How to configure Kubernetes?
Minion
Create /etc/sysconfig/flanneld
# etcd url location. Point this to the server where etcd runs
FLANNEL_ETCD="http://192.168.122.10:4001"
# etcd config key. This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_KEY="/atomic01/network"
[Unit]
After=flanneld.service
Requires=flanneld.service
[Service]
EnvironmentFile=/run/flannel/subnet.env
ExecStartPre=-/usr/sbin/ip link del docker0
ExecStart=/usr/bin/docker -d --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU}
$OPTIONS $DOCKER_STORAGE_OPTIONS
Create /etc/systemd/system/docker.service.d/10-flanneld-network.conf
Flanneld Configuration

21. How to configure Kubernetes?
# Comma seperated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd_servers=http://192.168.20.10:4001“
config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all
interfaces)
KUBELET_ADDRESS="--address=192.168.20.11"..
.# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname_override=minion1"
kubelet
Minion

22. What does Flanneld do?
ens3
192.168.10.11
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod A Pod B
Apache container Apache container
Apache container in Pod A
Apache container in Pod B
Communication ??

23. What does Flanneld do?
ens3
192.168.10.11
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod A Pod B
Apache container Apache container
Apache container in Pod A
Apache container in Pod B
Communication OK
through docker0 bridge

24. What does Flanneld do?
ens3
192.168.10.11
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod A Pod B
Apache container Apache container
ens3
192.168.10.12
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod C Pod D
Apache container Apache container
br0
192.168.10.1

25. What does Flanneld do?
ens3
192.168.10.11
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod A Pod B
Apache container Apache container
ens3
192.168.10.12
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod C Pod D
Apache container Apache container
br0
192.168.10.1
Apache container in Pod A Apache container in Pod C
Communication ??

26. What does Flanneld do?
ens3
192.168.10.11
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod A Pod B
Apache container Apache container
ens3
192.168.10.12
docker0
172.17.42.1/16
veth0
172.17.0.1/24
veth0
172.17.0.2/24
Atomic Host
Pod C Pod D
Apache container Apache container
br0
192.168.10.1
Apache container in Pod A Apache container in Pod C
Communication NO

27. What does Flanneld do?
ens3
192.168.10.11
docker0
172.16.32.1/24
veth0
172.16.32.2/24
veth0
172.16.32.3/24
Atomic Host
Pod A Pod B
Apache container Apache container
br0
192.168.10.1
Flannel.1
172.16.32.0/16
flanneld
ens3
192.168.10.12
docker0
172.16.10.1/24
veth0
172.16.10.2/24
veth0
172.16.10.3/24
Atomic Host
Pod C Pod D
Apache container Apache container
Flannel.1
172.16.10.0/16
flanneld
Apache container in Pod A Apache container in Pod C
Communication YES

28. Label
Service Pod
Pod Pod
Replication
Controller
Replication
Controller
Replication
Controller
Service
Pod
Replication
Controller
Replication
Controller
Service

29. Label
Service Pod
Pod Pod
Replication
Controller
Replication
Controller
Replication
Controller
Service
Pod
Replication
Controller
Replication
Controller
Service
labels :
name : apache

30. Label
Service Pod
Pod Pod
Replication
Controller
Replication
Controller
Replication
Controller
Service
Pod
Replication
Controller
Replication
Controller
Service
labels :
name : apache
port : 80

31. Pod
id: "apache"
kind: "Pod"
apiVersion: "v1beta1"
desiredState:
manifest:
version: "v1beta1"
containers:
- name: "apache80"
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
cpu: 100
labels:
name: "apache"
port: "80"

32. Pod
id: "apache"
kind: "Pod"
apiVersion: "v1beta1"
desiredState:
manifest:
version: "v1beta1"
containers:
- name: "apache80"
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
cpu: 100
labels:
name: "apache"
port: "80"
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
apache 172.16.69.238 apache80 local-registry:5000/rhel7-ews21:demo minion4/ name=apache,port=80 Running

33. Pod
id: "apache"
kind: "Pod"
apiVersion: "v1beta1"
desiredState:
manifest:
version: "v1beta1"
containers:
- name: "apache80"
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
cpu: 100
labels:
name: "apache"
port: "80"
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
apache 172.16.69.238 apache80 local-registry:5000/rhel7-ews21:demo minion4/ name=apache,port=80 Running
minion1 minion2 minion3 minion4
Pod : apache
apache80
Container
172.16.69.238

34. Pod
id: "apache"
kind: "Pod"
apiVersion: "v1beta1"
desiredState:
manifest:
version: "v1beta1"
containers:
- name: "apache80"
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
cpu: 100
labels:
name: "apache"
port: "80"
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
apache 172.16.69.238 apache80 local-registry:5000/rhel7-ews21:demo minion4/ name=apache,port=80 Running
minion1 minion2 minion3 minion4
Pod : apache
apache80
Container
172.16.69.238
172.16.69.28
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a0e11d8363e local-registry:5000/rhel7-ews21:demo "/bin/sh -c '$EWS_HO 14 minutes ago Up 14 minutes k8s_apache80.afee6806

35. Pod
id: "apache"
…..
containers:
- name: "apache80"
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
cpu: 100
- name: "apache8080"
image: local-registry:5000/rhel7-ews21:8080
imagePullPolicy: PullAlways
ports:
- containerPort: 8080
hostPort: 8080
cpu: 100
labels:
name: "apache"
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
apache 172.16.15.9 apache80 local-registry:5000/rhel7-ews21:demo minion3/ name=apache Running
apache8080 jonlangemak/docker:web_container_8080
minion3
Pod : apache
apache80
172.16.15.9
apache8080
pause

36. Pod
id: "apache"
…..
containers:
- name: "apache80"
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
cpu: 100
- name: "apache8080"
image: jonlangemak/docker:web_container_8080
imagePullPolicy: PullAlways
ports:
- containerPort: 8080
hostPort: 8080
cpu: 100
labels:
name: "apache"
minion3
Pod : apache
apache80
172.16.15.9
apache8080
pause
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
af1e66870aa1 jonlangemak/docker:web_container_8080 "/usr/bin/supervisor 7 minutes ago Up 7 minutes k8s_apache8080…..
f3c0c5009a1b local-registry:5000/rhel7-ews21:demo "/bin/sh -c '$EWS_HO 10 minutes ago Up 10 minutes k8s_apache80.32….
605fccc3e06f kubernetes/pause:latest "/pause" 10 minutes ago Up 10 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp k8s_net.13b8d9d0_...

37. Replication Controller
id: apache-rc-controller
apiVersion: v1beta1
kind: ReplicationController
desiredState:
replicas: 3
replicaSelector:
name: apache
podTemplate:
desiredState:
manifest:
version: v1beta1
id: apache-pod
containers:
- name: apache-pod
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
labels:
name: apache
port : 80

38. Replication Controller
id: apache-rc-controller
apiVersion: v1beta1
kind: ReplicationController
desiredState:
replicas: 3
replicaSelector:
name: apache
.…
id: apache-pod
containers:
- name: apache-pod
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
labels:
name: apache
port : 80
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
apache-rc-controller apache-pod local-registry:5000/rhel7-ews21:demo name=apache 3
minion1 minion2 minion3 minion4
Pod : apache
apache80
Container
172.16.69.239
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
ff433d37-…-52540098677f 172.16.31.28 apache-pod loal-registry:5000/rhel7-ews21:demo minion1/ name=apache Running
26a89e0e-…-52540098677f 172.16.20.15 apache-pod local-registry:5000/rhel7-ews21:demo minion2/ name=apache Running
d7deb52e-…-52540098677f 172.16.69.239 apache-pod local-registry:5000/rhel7-ews21:demo minion4/ name=apache Running
Pod : apache
apache80
Container
172.16.31.28
Pod : apache
apache80
Container
172.16.20.15
Replication Controller

39. Replication Controller
id: apache-rc-controller
apiVersion: v1beta1
kind: ReplicationController
desiredState:
replicas: 1
replicaSelector:
name: apache
.…
id: apache-pod
containers:
- name: apache-pod
image: local-registry:5000/rhel7-ews21:demo
imagePullPolicy: PullAlways
ports:
- containerPort: 80
hostPort: 80
labels:
name: apache
port : 80
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
apache-rc-controller apache-pod local-registry:5000/rhel7-ews21:demo name=apache 1
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
d7deb52e-…-52540098677f 172.16.69.239 apache-pod local-registry:5000/rhel7-ews21:demo minion4/ name=apache Running
minion1 minion2 minion3 minion4
Pod : apache
apache80
Container
172.16.69.239
Pod : apache
apache80
Container
172.16.31.28
Pod : apache
apache80
Container
172.16.20.15
Replication Controller

40. Service
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
ff433d37-…-52540098677f 172.16.31.28 apache-pod local-registry:5000/rhel7-ews21:demo minion1/ name=apache Running
26a89e0e-…-52540098677f 172.16.20.15 apache-pod local-registry:5000/rhel7-ews21:demo minion2/ name=apache Running
d7deb52e-…-52540098677f 172.16.69.239 apache-pod local-registry:5000/rhel7-ews21:demo minion4/ name=apache Running
NAME LABELS SELECTOR IP PORT
apache-service name=apache-service name=apache 10.254.220.71 80
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
apache-rc-controller apache-pod local-registry:5000/rhel7-ews21:demo name=apache 3
Service
Replication Controller
Pod
id: apache-service
kind: Service
apiVersion: v1beta1
port: 80
containerPort: 80
selector:
name: apache
labels:
name: apache-service

41. Service #iptables –L –n –t nat
….
Chain KUBE-PORTALS-CONTAINER (1 references)
target prot opt source destination
REDIRECT tcp -- 0.0.0.0/0 10.254.57.4 /* apache-service */ tcp dpt:80 redir ports 42927
REDIRECT tcp -- 0.0.0.0/0 10.254.12.160 /* kubernetes */ tcp dpt:443 redir ports 43044
REDIRECT tcp -- 0.0.0.0/0 10.254.34.217 /* kubernetes-ro */ tcp dpt:80 redir ports 49932
Chain KUBE-PORTALS-HOST (1 references)
target prot opt source destination
DNAT tcp -- 0.0.0.0/0 10.254.57.4 /* apache-service */ tcp dpt:80 to:192.168.20.12:42927
DNAT tcp -- 0.0.0.0/0 10.254.12.160 /* kubernetes */ tcp dpt:443 to:192.168.20.12:43044
DNAT tcp -- 0.0.0.0/0 10.254.34.217 /* kubernetes-ro */ tcp dpt:80 to:192.168.20.12:49932
Mininon 2
#iptables –L –n –t nat
Chain KUBE-PORTALS-CONTAINER (1 references)
target prot opt source destination
REDIRECT tcp -- 0.0.0.0/0 10.254.57.4 /* apache-service */ tcp dpt:80 redir ports 40144
REDIRECT tcp -- 0.0.0.0/0 10.254.12.160 /* kubernetes */ tcp dpt:443 redir ports 42578
REDIRECT tcp -- 0.0.0.0/0 10.254.34.217 /* kubernetes-ro */ tcp dpt:80 redir ports 36642
Chain KUBE-PORTALS-HOST (1 references)
target prot opt source destination
DNAT tcp -- 0.0.0.0/0 10.254.57.4 /* apache-service */ tcp dpt:80 to:192.168.20.14:40144
DNAT tcp -- 0.0.0.0/0 10.254.12.160 /* kubernetes */ tcp dpt:443 to:192.168.20.14:42578
DNAT tcp -- 0.0.0.0/0 10.254.34.217 /* kubernetes-ro */ tcp dpt:80 to:192.168.20.14:36642
Mininon 4

42. Service
minion1 minion2 minion3 minion4
Pod : apache
apache80
Container
172.16.69.239
Pod : apache
apache80
Container
172.16.31.28
Pod : apache
apache80
Container
172.16.20.15
Service
10.254.220.71:80

43. Service
minion1 minion2 minion3 minion4
Pod : apache
apache80
Container
172.16.69.239
Pod : apache
apache80
Container
172.16.31.28
Pod : apache
apache80
Container
172.16.20.15
Kube-Proxy Kube-ProxyKube-Proxy Kube-Proxy
docker0
flannel.1
12 14
docker0
flannel.1
docker0
flannel.1
docker0
flannel.1
1311
On 172.16.20.15(minion2)
request to 10.254.220.71:80
# Curl 10.254.220.71

44. Kubectl commands
• Create
• kubectl create pod/service/rc –f *.yaml/json
• Log
• kubectl log pod <pod_id>
• Get
• kubectl get pod/service/rc
• Describe
• kubectl describe pod <pod_id>
• Update
• kubectl update –f *.yaml/json
• Delete
• kubectl delete pod/service/rc <id> or –l <label>
• Resize replicas
• kubectl resize --current-replicas=3 --replicas=4 rc apache-rc-controller
• Rollingupdate
• kubectl rollingupdate apache-rc-controller -f apache-rc-2.yaml --update-period="5s"

45. Live Demo
•EAP test environment using Docker
•Rolling Update

46. Q&A
•Questions?

47. Thank you.
한국레드햇
이주호 과장 TSE
jlee@redhat.com