SlideShare une entreprise Scribd logo
1  sur  73
Télécharger pour lire hors ligne
INTRODUCTION TO KUBERNETES
OpenNMS DevJam 2019
by Alejandro Galue
WHY CONTAINERS
• Build, Ship, and Run any App, Anywhere.
• No more dependency problems; e.x. no RPM/DEB or
dynamic libraries incompatibility issues.
• No more: "but it works on my machine", as the same
image used in dev is deployed to prod.
• Serverless implementations are mainly backed by
containers.
WHY KUBERNETES
• Be able to manage hundreds or thousands of containers,
on a fleet of hundred or thousands of nodes.
• Be able to deploy an application without worrying about
the hardware infrastructure.
• Be able to guarantee that applications will stay running
according with the specifications.
• Be able to facilitate scaling and upgrades not only for the
workloads but for the k8s cluster itself.
WHAT IS KUBERNETES
• A distributed cluster technology that manages container-based systems
in a declarative manner using an API (a container orchestrator).
• Designed from the ground-up as a loosely coupled collection of
components centered around deploying, maintaining and scaling
workloads.
• Abstracts away the underlying hardware of the nodes and provides a
uniform interface for workloads to be both deployed and consume the
shared pool of resources.
• Works as an engine for resolving state by converging actual and the
desired state of the system (self-healing).
A COUPLE OF KEY CONCEPTS…
PODS
• Atomic unit or smallest “unit of
work”of Kubernetes.
• Pods are one or MORE
containers that share volumes, a
network namespace, and are a
part of a single context.
Host A
Pod
ExternalVolume
container container
Network Namespace 10.255.16.3
Pod Network
Ephemeral
SERVICES
• Unified method of
accessing the exposed
workloads of Pods.
• Durable resource
- static cluster IP
- static namespaced DNS
name
Host A
Labels:
app=nginx
env=prod
Labels:
app=nginx
env=prod
Labels:
app=mysql
env=dev
Host B
Labels:
app=nginx
env=prod
Labels:
app=mysql
env=prod
Labels:
app=nginx
env=dev
Service
app=nginx
env=prod
NOT
Ephemeral
ARCHITECTURE OVERVIEW
Control Plane - 1,2,…n
kube
controller manager
kube
scheduler
kube
apiserver
kubectl
Node 1
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
Node 2
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
cloud
controller manager
Cloud
Provider
API
Load
Balancer
End Users
Cloud Provider
Network Edge
Control Plane - 1,2,…n
Containers
kube-control-manager
kube-scheduler
kube
apiserver
kubectl
Node 1
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
Node 2
Containers
System Services
kubelet
kube-proxy
Container Runtime
Pod Pod
cloud-controller-manager
Cloud
Provider
API
Load
Balancer
End Users
Cloud Provider
Network Edge
System Services
kubelet
Container Runtime
additional services
kube-proxy
KUBE-APISERVER
• Provides a forward facing REST interface into the
kubernetes control plane and datastore.
• All clients and other applications interact with
kubernetes strictly through the API Server.
• Acts as the gatekeeper to the cluster by handling
authentication and authorization, request validation,
mutation, and admission control in addition to being
the front-end to the backing datastore.
master
• etcd acts as the cluster datastore.
• Purpose in relation to Kubernetes is to provide a
strong, consistent and highly available key-value store
for persisting cluster state.
• Stores objects and config information.
• Uses “Raft Consensus” among a quorum of systems to
create a fault-tolerant consistent “view” of the cluster.
master
KUBE-CONTROLLER-MANAGER
• Serves as the primary daemon that manages
all core component control loops.
• Monitors the cluster state via the apiserver
and steers the cluster towards the desired
state.
master
KUBE-SCHEDULER
• Component on the master that watches newly
created pods that have no node assigned, and
selects a node for them to run on.
• Factors taken into account for scheduling decisions
include individual and collective resource
requirements, hardware/software/policy
constraints, affinity and anti-affinity specifications,
data locality, inter-workload interference and
deadlines.
master
CLOUD-CONTROLLER-MANAGER
• Daemon that provides cloud-provider specific
knowledge and integration capability into the
core control loop of Kubernetes.
• The controllers include Node, Route, Service,
and add an additional controller to handle
things such as PersistentVolume Labels.
master
Optional
KUBE-PROXY
• Manages the network rules on each node.
• Performs connection forwarding or load
balancing for Kubernetes cluster services.
worker
KUBELET
• An agent that runs on each node in the cluster.
It makes sure that containers are running in a
pod.
• The kubelet takes a set of PodSpecs that are
provided through various mechanisms and
ensures that the containers described in those
PodSpecs are running and healthy.
worker
CONTAINER RUNTIME ENGINE
A container runtime is a CRI (Container Runtime
Interface) compatible application that executes and
manages containers.
• containerd (Docker)
• cri-o
• rkt
• kata (formerly clear and hyper)
• virtlet (VM CRI compatible runtime)
worker
KUBERNETES NETWORKING
• Pod Network
- Cluster-wide network used for pod-to-pod
communication managed by a CNI
(Container Network Interface) plugin.
• Service Network
- Cluster-wide range of Virtual IPs managed by
kube-proxy for service discovery.
FUNDAMENTAL NETWORKING RULES
• All containers within a pod can communicate
with each other unimpeded.
• All Pods can communicate with all other Pods
without NAT.
• All nodes can communicate with all Pods (and
vice-versa) without NAT.
• The IP that a Pod sees itself as is the same IP
that others see it as.
What if I want to limit
communication within
Pods ?
Learn about Network
Policies (and make sure
the chosen CNI supports
it)
API
API OVERVIEW
• The REST API is the true
keystone of Kubernetes.
• Everything within Kubernetes
is as an API Object.
• Referenced within an object as
the apiVersion and kind.
Format:
/apis/<group>/<version>/<resource>
Examples:
/apis/apps/v1/deployments
/apis/batch/v1beta1/cronjobs
OBJECT MODEL
• Objects are a “record of intent” or a
persistent entity that represent the desired
state of the object within the cluster.
• All objects MUST have  apiVersion, kind, and
poses the nested fields metadata.name,
metadata.namespace, and metadata.uid.
OBJECT EXPRESSION -YAML
• Files or other representations
of Kubernetes Objects are
generally represented in YAML.
• Three basic data types:
- mappings  - hash or
dictionary,
- sequences - array or list
- scalars - string, number,
boolean etc
apiVersion: v1
kind: Pod
metadata:
name: sample
namespace: test
spec:
containers:
- name: container1

image: nginx
- name: container2

image: alpine
YAMLVS JSON
{
  "apiVersion": "v1",
  "kind": "Pod",
  "metadata": {
    "name": "pod-example"
  },
  "spec": {
    "containers": [
      {
        "name": "nginx",
        "image": "nginx:stable-alpine",
        "ports": [
{ "containerPort": 80 }
]
      }
    ]
  }
}
apiVersion: v1
kind: Pod
metadata:
  name: pod-example
spec:
  containers:
  - name: nginx
    image: nginx:stable-alpine
    ports:
    - containerPort: 80
Are you wondering about theYAML schema ?
kubectl explain is your friend ;)
What about kinds or versions ?
kubectl api-versions is your friend ;)
CORE CONCEPTS
NAMESPACES
Namespaces are a logical cluster or
environment, and are the primary method of
partitioning a cluster or scoping access.
apiVersion: v1
kind: Namespace
metadata:
  name: prod
  labels:
    app: MyBigWebApp
$ kubectl get ns --show-labels
NAME          STATUS    AGE       LABELS
default       Active    11h       <none>
kube-public   Active    11h       <none>
kube-system   Active    11h       <none>
prod          Active    6s        app=MyBigWebApp
Known as
Projects in
OpenShift
PODS
• Atomic unit or smallest
“unit of work”of
Kubernetes.
• Foundational building
block of Kubernetes
Workloads.
• Pods are one or MORE
containers that share
volumes, a network
namespace, and are a
part of a single context.
Host A
Pod
ExternalVolume
container container
Network Namespace 10.255.16.3
Pod Network
apiVersion: v1
kind: Pod
metadata:
name: sample
spec:
containers:
- name: nginx
image: nginx:stable-alpine
ports:
  - containerPort: 80
    name: http
    protocol: TCP
env:
- name: MYVAR
value: isAwesome
command: [“/bin/sh”, “-c”]
args: [“echo ${MYVAR}”]
readinessProbe:
tcpSocket:
port: http
initialDelaySeconds: 10
periodSeconds: 10
livenessProbe:
httpGet:
path: /
port: http
initialDelaySeconds: 30
periodSeconds: 60
Array of
ports to expose
Name of the
container
Container
ImageArray of
environment
variables
Entrypoint
Array
~ Docker
ENTRYPOINT
Arguments
to pass to the
command
~ Docker CMD
Tells
when the
application is ready
to receive
requests
Checks if the
container is still
alive (running)
LABELS
• key-value pairs that
are used to identify,
describe and group
together related sets of
objects or resources.
• NOT characteristic of
uniqueness.
• Have a strict syntax
with a slightly limited
character set*.
https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set
Host A
Pod
container
Network Namespace 10.255.16.3
Pod Network
Labels:
gpu=nvidia
Labels:
app=nginx
env=prod
apiVersion: v1
kind: Pod
metadata:
  name: pod-label-example
  labels:

    app: nginx

    env: prod
spec:
  containers:
  - name: nginx
    image: nginx:stable-alpine
    ports:
    - containerPort: 80
SELECTORS
Selectors use
labels to filter or
select objects, and
are used
throughout
Kubernetes.
Host A
Pod
mlapp
Network Namespace 10.255.16.3
Pod Network
Labels:
gpu=nvidia
Labels:
app=mlapp
env=prodapiVersion: v1
kind: Pod
metadata:
  name: mlapp
  labels:

    app: mlapp

    env: prod
spec:
  nodeSelector:
  - gpu: nvidia
  containers:
  - name: nginx
    image: tensorflow/tensorflow
Host B
Labels:
gpu=amd
SERVICES
• Unified method of accessing the
exposed workloads of Pods.
• Durable resource (unlike Pods)
- static cluster-unique IP
- static namespaced DNS name
<service name>.<namespace>.svc.cluster.local
There are 4 major service types:
• ClusterIP (default)
• NodePort
• LoadBalancer
• ExternalName
CLUSTER IP SERVICE
Labels:
app=nginx
env=prod
kube-proxy
Host A
Labels:
app=nginx
env=prod
Host B
Pod
Host C
Pod
Network
kube-proxy
kube-proxy
Name:         example-prod
Selector:     app=nginx,env=prod
Type:         ClusterIP
IP:           10.96.28.176
Port:         <unset>  80/TCP
TargetPort:   80/TCP
Endpoints:    10.255.16.3:80,

              10.255.16.4:80
• The Pod on host C requests the service.
• Hits host iptables and it load-balances
the connection between the endpoints
residing on Hosts A, B
/ # nslookup example-prod.default.svc.cluster.local
Name:      example-prod.default.svc.cluster.local
Address 1: 10.96.28.176 example-prod.default.svc.cluster.local
NODE PORT SERVICE
Labels:
app=nginx
env=prod
kube-proxy
Host A
Labels:
app=nginx
env=prod
Host B
Labels:
app=nginx
env=dev
Host C
Name:         example-prod
Selector:     app=nginx,env=prod
Type:         NodePort
IP:           10.96.28.176
Port:         <unset>  80/TCP
TargetPort:   80/TCP
NodePort:     <unset>  32410/TCP
Endpoints:    10.255.16.3:80,

              10.255.16.4:80
• User can hit any host in cluster on
NodePort IP and get to service.
• Does introduce extra hop if hitting a
host without instance of the pod.
32410
32410kube-proxy
32410kube-proxy
LOAD BALANCER SERVICE
• LoadBalancer services extend NodePort.
• Works in conjunction with an external
system to map a cluster external IP to the
exposed service.
Labels:
app=nginx
env=prod
kube-proxy
Host A
Labels:
app=nginx
env=prod
Host B
Labels:
app=nginx
env=dev
Host C
32410
32410kube-proxy
32410kube-proxy
Load
Balancer
Cloud Provider
Edge Network
Name:         example-prod
Selector:     app=nginx,env=prod
Type:         LoadBalancer
IP:           10.96.28.176
LoadBalancer

Ingress:      172.17.18.43
Port:         <unset>  80/TCP
TargetPort:   80/TCP
NodePort:     <unset>  32410/TCP
Endpoints:    10.255.16.3:80,

              10.255.16.4:80
INGRESSKnown as
Routes in
OpenShift
• An API object that manages external access
to the services in a cluster, through a single
LoadBalancer.
• Provides load balancing, SSL termination and
name/path-based virtual hosting
• Gives services externally-reachable URLs.
• Multiple implementations to choose from.
service:
grafana
port=3000
Host A
service:
kibana
port=5601
Host B
service:
opennms
port=8980
Host C
grafana.example.com
kibana.example.com
Cloud Provider
Edge Network
Ingress
Controller
Host D
Load
Balancer
onms.example.com
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-rules
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: onms.example.com
http:
paths:
- path: /
backend:
serviceName: opennms
servicePort: 8980
…
EXTERNAL NAME SERVICE
• ExternalName is used to
reference endpoints
OUTSIDE the cluster.
• Creates an internal CNAME
DNS entry that aliases
another.
apiVersion: v1
kind: Service
metadata:
  name: example-prod
spec:
  type: ExternalName
spec:
  externalName: example.com
WORKLOADS
WORKLOADS
• Workloads within Kubernetes
are higher level objects that
manage Pods or other higher
level objects.
• In ALL CASES a Pod Template
is included, and acts the base
tier of management.
•ReplicaSet
•Deployment
•DaemonSet
•StatefulSet
•Job
•CronJob
PODTEMPLATE
• Workload Controllers
manage instances of Pods
based off a provided
template.
• Pod Templates are Pod
specs with limited metadata.
• Controllers use Pod
Templates to make actual
pods
apiVersion: v1
kind: Pod
metadata:
  name: pod-example
  labels:

    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
template:
  metadata:
    labels:

      app: nginx
  spec:
    containers:
    - name: nginx
      image: nginx
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-example
  labels:

    app: nginx
spec:
replicas: 3
  selector:
    matchLabels:

      app: nginx
template:
  metadata:
    labels:

      app: nginx
  spec:
    containers:
    - name: nginx
      image: nginx
RESOURCE MODEL
• Request: amount of a resource
allowed to be used, with a strong
guarantee of availability.
- CPU (seconds/second), RAM
(bytes)
- Scheduler will not over-commit
requests
• Limit: max amount of a resource that
can be used, regardless of guarantees
- Scheduler ignores limits
apiVersion: v1
kind: Pod
metadata:
  name: pod-example
  labels:

    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
  resources:
  requests:
  memory: "64Mi"
    cpu: "250m"
  limits:
  memory: "128Mi"
    cpu: "500m"
INIT CONTAINERS
If we compare a container template with a Class
definition in Java, an initContainer would be the
constructor of a class; while a running Pod, would be
an instance of that class.
Can be used for everything that should happen
before the containers within a Pod start running. For
example: wait for dependencies, initialize volumes or
databases, verify requirements, etc.
REPLICA SET
• Primary method of managing
pod replicas and their lifecycle.
• Includes their scheduling,
scaling, and deletion.
• Their job is simple: Always
ensure the desired number
of pods are running.
ReplicaSet Pod
• replicas: The desired number of instances of
the Pod.
• selector: The label selector for the
ReplicaSet will manage ALL Pod instances
that it targets; whether it’s desired or not.
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: rs-example
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx

      env: prod
  template:
    <pod template>
DEPLOYMENT
• Declarative method of managing Pods via
ReplicaSets.
• Provide rollback functionality and update
control.
• Updates are managed through the pod-
template-hash label.
• Each iteration creates a unique label that is
assigned to both the ReplicaSet and
subsequent Pods.
PodReplicaSetDeployment
• revisionHistoryLimit: The number of previous iterations of the
Deployment to retain.
• strategy: Describes the method of  updating the Pods based on
the type.Valid options are:
- Recreate: All existing Pods are killed before the new ones
are created.
- RollingUpdate: Cycles through updating the Pods according
to the parameters: maxSurge and maxUnavailable.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-example
spec:
  replicas: 3

  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: nginx

      env: prod
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    <pod template>
DAEMON SET
• Ensure that all nodes matching certain
criteria will run an instance of the
supplied Pod.
• They bypass default scheduling
mechanisms.
• Are ideal for cluster wide services such
as log forwarding, or health monitoring.
• Revisions are managed via a
controller-revision-hash label.
DaemonSet Pod
• spec.template.spec.nodeSelector: The
primary selector used to target nodes.
• Default Host Labels:
- kubernetes.io/hostname
- beta.kubernetes.io/os
- beta.kubernetes.io/arch
• Cloud Host Labels:
- failure-domain.beta.kubernetes.io/zone
- failure-domain.beta.kubernetes.io/region
- beta.kubernetes.io/instance-type
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: ds-example
spec:
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: nginx
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  template:
    spec:
      nodeSelector:
        nodeType: edge
     <pod template>
STATEFUL SET
• Tailored to managing Pods that must persist or maintain state.
• Pod identity including hostname, network, and storage
WILL be persisted.
• Assigned a unique ordinal name following the convention of
‘<statefulset name>-<ordinal index>’.
• Naming convention is also used in Pod’s network Identity and
Volumes.
• Pod lifecycle will be ordered and follow consistent patterns.
• Revisions are managed via a controller-revision-hash label.
Pod
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: sts-cassandra
spec:
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  serviceName: cassandra
  updateStrategy:
   type: RollingUpdate
   rollingUpdate:
     partition: 0
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      containers:
      - name: cassandra-node
        image: cassandra:3.11.4
        env:
        - name: CASSANDRA_SEEDS
        value: sts-cassandra-0.cassandra
        - name: CASSANDRA_CLUSTER_NAME
        value: my-cluster
        ports:
        - containerPort: 7000
        - containerPort: 7199
        - containerPort: 9042
        volumeMounts:
        - name: data
          mountPath: /cassandra_data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: standard
      resources:
        requests:
          storage: 100Gi
The name of the
associated headless service;
or a service without a
ClusterIP
Pods with an
ordinal greater than the
partition value will be
updated one-by-one in
reverse order
StatefulSet
Template of the
persistent volume(s)
request to use for each
instance of the
StatefulSet.
HEADLESS SERVICE
<StatefulSet Name>-<ordinal>.<service name>.<namespace>.svc.cluster.local
apiVersion: v1
kind: Service
metadata:
  name: cassandra
spec:
  clusterIP: None
  selector:
    app: cassandra
  ports:
  - name: intra-node
    port: 80
  - name: jmx
    port: 7199
  - name: cql
    port: 9042
/ # dig cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> cassandra.default.svc.cluster.local +noall +answer
;; global options: +cmd
cassandra.default.svc.cluster.local. 5 IN A 172.17.0.5
cassandra.default.svc.cluster.local. 5 IN A 172.17.0.4
cassandra.default.svc.cluster.local. 5 IN A 172.17.0.6
/ # dig sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer
/ # dig sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer
/ # dig sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer
$ kubectl get pods
NAME            READY     STATUS    RESTARTS   AGE
sts-cassandra-0   1/1       Running   0          11m
sts-cassandra-1   1/1       Running   0          11m
sts-cassandra-2   1/1       Running   0          11m
VOLUME CLAIMTEMPLATE
<Volume Name>.<StatefulSet Name>-<ordinal>
 volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: standard
      resources:
        requests:
          storage: 1Gi
Persistent Volumes associated with a
StatefulSet will NOT be automatically
garbage collected when its associated
StatefulSet is deleted. They must
manually be removed.
$ kubectl get pvc
NAME                STATUS    VOLUME                                    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-cassandra-sts-0 Bound pvc-8baec2a5-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
data-cassandra-sts-1 Bound pvc-95463714-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
data-cassandra-sts-2 Bound pvc-9807c42a-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
JOB
• Job controller ensures one or more
pods are executed and successfully
terminate.
• Will continue to try and execute the
job until it satisfies the completion
and/or parallelism condition.
• Pods are NOT cleaned up until the
job itself is deleted.
PodJob
apiVersion: batch/v1
kind: Job
metadata:
  name: job-example
spec:
  backoffLimit: 4
  completions: 4
  parallelism: 2
  template:
    spec:
      restartPolicy: Never
    template:
    <pod-template>
• backoffLimit: The number of failures before
the job itself is considered failed.
• completions: The total number of successful
completions desired.
• parallelism: How many instances of the pod
can be run concurrently.
• spec.template.spec.restartPolicy:Jobs only
support a restartPolicy of type Never or
OnFailure.
CRONJOB
• An extension of the Job Controller,
it provides a method of executing
jobs on a cron-like schedule.
• CronJobs within Kubernetes

 use UTC ONLY.
PodJobCronJob
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: cronjob-example
spec:
  schedule: "*/1 * * * *"
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      completions: 4
      parallelism: 2
      template:
        <pod template>
The cron
schedule for
the job
The number of
successful jobs to
retain
The number of
failed jobs to
retain
OPERATORS
• Might be the most powerful feature of
K8s.
• When none of the workloads fits the
needs of your app, you can create your
own controllers with your own
specifications (CRD).
• Should be implemented in Go, but there
are alternatives.
• A given controller usually requires a
ServiceAccount with proper RBAC Role
to be able to manage workloads (e.x.
create and maintain Pods, Services, etc.)
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata:
name: opennms-database
spec:
teamId: OpenNMS
volume:
size: 100Gi
numberOfInstances: 3
enableMasterLoadBalancer: false
enableReplicaLoadBalancer: false
users:
opennms:
- superuser
- createdb
databases:
opennms: opennms
postgresql:
version: "10"
STORAGE
VOLUMES
• Storage that is tied to the Pod’s Lifecycle.
• A pod can have one or more types of volumes
attached to it.
• Can be consumed by any of the containers
within the pod.
• Survive Pod restarts; however their durability
beyond that is dependent on the Volume Type.
PERSISTENTVOLUMES
• A PersistentVolume (PV) represents a storage resource. 
• PVs are a cluster wide resource linked to a backing
storage provider: NFS, EBS, GCEPersistentDisk, etc.
• Generally provisioned by an administrator.
• Their lifecycle is handled independently from a pod
• CANNOT be attached to a Pod directly. Relies on a 
PersistentVolumeClaim (PVC).
• A PVC is a namespaced request for storage.
STORAGE CLASS
• Satisfies a set of requirements instead of mapping
to a storage resource directly.
• Ensures that an application’s claim for storage is
portable across numerous backends or
providers.
CONFIGURATION
CONFIG MAPS
• Externalized data stored within
kubernetes.
• Can be referenced through several
different means:
- environment variable
- a command line argument (via
environment variable)
- injected as a file into a volume mount
• Can be created from a manifest, literals,
directories, or files directly.
apiVersion: v1
kind: ConfigMap
metadata:
  name: manifest-example
data:
  state: Minnesota
  city: Minneapolis
  content: |
    Look at this,
    its multiline!
SECRETS
• Functionally identical to a ConfigMap.
• Stored as base64 encoded content.
• Encrypted at rest within etcd (if
configured!).
• Ideal for username/passwords,
certificates or other sensitive information
that should not be stored in a container.
• Can be created from a manifest, literals,
directories, or from files directly.
apiVersion: v1
kind: Secret
metadata:
  name: manifest-secret
type: Opaque
data:
  username: ZXhhbXBsZQ==
  password: bXlwYXNzd29yZA==
• type: There are three different types of
secrets within Kubernetes:
- docker-registry - credentials used to
authenticate to a container registry
- generic/Opaque - literal values from
different sources
- tls - a certificate based secret
• data: Contains key-value pairs of base64
encoded content.
INSTALLATION
BARE-METAL, CLOUD, OR

LOCAL INSTALLATION
Install
OpenShift
is hard ;)
Local PC Installation

For development and testing
• minikube
• minishift (for OKD)
• oc cluster up (for OKD)
• Docker for Mac or Windows
• microk8s (for Linux via Snap)
Cloud Installation

Single Command Experience
• gcloud container clusters create
• kops create cluster
• eksctl create cluster
• az aks create
• az openshift create
OpenShift 4 promises this for the first
time since its conception.
Bare-Metal / On-Premise
• kubeadm (single command
to spin up a master or a
worker)
• OpenShift/OKD Ansible
Tools
On-Premise vs Cloud Managed
• Masters are managed for you, and the only additional resources are the instances for worker nodes
• Complete integration with cloud services (volumes, network, load balancers, etc.)
SECURITY
Here is where
OpenShift is
Excellent!!!
https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/
THINGS TO KEEP IN MIND
• Users (to access cluster through the API). Usually implemented through certificates.
• Service Accounts (for controllers and operators, to make changes from within K8s).
• RBAC: Roles and RoleBinding, ClusterRole and ClusterRoleBinding (for users, groups
and service accounts).
• Never run containers as root, and block privilege escalations (OpenShift enforce this by
default using SecurityContextConstraint; known I k8s as PodSecurityPolicy).
• Limit resources per namespace (total CPUs and memory, number of Pods, …).
• Create your own images and keep a private registry (OpenShift offers this by default).
• Research about NetworkPolicy (which requires a CNI that supports this feature).
• Research about applications like vault (to enhance protection for your secrets).
• Research about clair (a tool for vulnerability scans on containers).
• Research about kube-bench (a tool that analyzes your cluster for security best practices).
• Research about gvisor.dev (a container sandbox focused on security and efficiency).
Some inspired
by OpenShift
ACCESS
On user the certificate (signed by the CA
used on the API server), the Common
Name (CN) will be interpreted as the
username, and the Organization (O) as
the group where this user belongs to.
Use the kubectl command to
create the kubeconfig based
on the user’s certificate, to
access the cluster resources.
Define an RBAC Role to
connect resources
(deployments, pods,
ingress, …) with operations
(create, delete, list, …) on a
given namespace.
Define an RBAC
RoleBinding to connect a
role with subjects (i.e.
group, user, service-
account)
Use a
ClusterRole for
cluster wide access
For a
ClusterRole use
ClusterRoleBinding
Now you
have
access!
cluster-admin
O=system:masters
MONITORING
HOWTO MONITOR K8S?
• Based on Google’s Four Golden Signals: latency, errors, traffic, saturation, use
Brendan Gregg’s USE method (Utilization, Saturation, Errors) for Resources,
and Tom Wilkie’s RED method (Rate, Errors, Duration) for Services.
• Metric Server collects metrics such as CPU and Memory by each pod and
node from the Summary API, exposed by Kubelet (via cAdvisor) on each
node.This allows the usage of K8s features like the HorizontalPodAutoscaler.
• The usage of Prometheus is widely adopted.This solution is based on their
own node_exporter (deployed as DaemonSet) which exposes lots of Linux
metrics in Prometheus format.
MONITOR K8S WITH OPENNMS
• OpenNMS has been used to monitor network infrastructure (or
network devices).
• OpenNMS can monitor servers at some degree, and can do some
level of application monitoring; but traditionally other applications
are used in this context.
• For Kubernetes, Prometheus is wildly adopted, on a level that it is
considered the standard.
• OpenNMS can be used to expand the functionality of the already
existing observability/monitoring tools in Kubernetes world.
MONITOR K8S WITH OPENNMS
• Have an operator forwarding important K8s events to OpenNMS, so you
can use all the event processing infrastructure to react upon changes on
your K8s cluster.
• There is no such thing as fixed IP addresses in Kubernetes, so forget
about Pods. But, if providing support FQDN support in OpenNMS is
possible, that can make monitoring K8s services a reality in the future.
• Build a solution based on the Metrics API / cAdvisor, or Prometheus
(directly or indirectly through the node_exporter), to get data from the
running Nodes, Pods and Services.
• SNMP could run on Kubernetes nodes; but that’s not an option for Pods.
HOWTO DEPLOY OPENNMS IN K8S?
• Use a StatefulSet for OpenNMS (1 replica), to take advantage of auto-deployed PVCs, as
the dynamic nature of the configuration directory makes OpenNMS a stateful application.
Keep in mind "kubectl cp" for transferring files.
• Use a StatefulSet for Minions, to take advantage of the persistent network identity, and use
Pod name as the minion-ID.This is recommended, even if Minions are stateless by nature.
• Use a Deployment for Sentinels, as they are stateless in nature and having a unique
identity is not relevant (for now). Although a StatefulSet won’t hurt.
• A combination of initContainers, config-maps, and secrets can be used to pre-configure
the mandatory settings on all the applications.The PersistentVolume will keep the custom
settings added by the user during the lifetime of the product.
OPENNMS DEPENDENCIES
It depends… a given team should decide if it
it worth running PostgreSQL, Kafka, Cassandra
and Elasticsearch within K8s.
Keep in mind that all databases are stateful in
nature, and managing the life-cycle is
different on each case. Solutions based on
operators when available are preferred.
INTEGRATE OPENNMS WITH K8S
• OpenNMS can forward everything to Kafka, and
even consume events from Kafka. From there,
Serverless technologies or Kafka Streams apps
deployed into K8s, can be used to create your own
horizontally scalable integrations triggered by
events, alarms or collected metrics.
• What about using OpenNMS as the source of metrics
for Horizontal Pod Autoscaling or Cluster Workers
auto-scaling ?
DEMO
OVERWHELMED?
Do not forget we haven’t talked about …
• Several k8s features not discussed here ;)
• Helm (the k8s package manager; the yum or apt-get for k8s)
• Service Meshes (Envoy, Istio, Linkerd, …)
• Serverless (Kubeless, Fission, Knative, …)
• Machine Learning (kubeflow, …)
• VMs and Kubernetes (KubeVirt)
• Operators In Depth:

https://operatorhub.io

https://github.com/operator-framework
And more…

Not a big enough lis ? Check https://landscape.cncf.io
QUESTIONS?
THANKYOU!
This work has been inspired by a few presentations from:
https://github.com/cncf/presentations/tree/master/kubernetes
Demo:
https://github.com/OpenNMS/opennms-drift-kubernetes

Contenu connexe

Tendances

Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes VMware Tanzu
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetesrajdeep
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewBob Killen
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetesMichal Cwienczek
 
Kubernetes Architecture
 Kubernetes Architecture Kubernetes Architecture
Kubernetes ArchitectureKnoldus Inc.
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Megan O'Keefe
 
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...Edureka!
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetesRishabh Indoria
 
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideBytemark
 
An Introduction to Kubernetes
An Introduction to KubernetesAn Introduction to Kubernetes
An Introduction to KubernetesImesh Gunaratne
 
(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive Overview(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive OverviewBob Killen
 
Deploying your first application with Kubernetes
Deploying your first application with KubernetesDeploying your first application with Kubernetes
Deploying your first application with KubernetesOVHcloud
 
Kubernetes Architecture and Introduction
Kubernetes Architecture and IntroductionKubernetes Architecture and Introduction
Kubernetes Architecture and IntroductionStefan Schimanski
 
Monitoring kubernetes with prometheus
Monitoring kubernetes with prometheusMonitoring kubernetes with prometheus
Monitoring kubernetes with prometheusBrice Fernandes
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopBob Killen
 

Tendances (20)

Getting Started with Kubernetes
Getting Started with Kubernetes Getting Started with Kubernetes
Getting Started with Kubernetes
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 
Introduction to Kubernetes
Introduction to KubernetesIntroduction to Kubernetes
Introduction to Kubernetes
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive Overview
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Kubernetes PPT.pptx
Kubernetes PPT.pptxKubernetes PPT.pptx
Kubernetes PPT.pptx
 
Kubernetes Architecture
 Kubernetes Architecture Kubernetes Architecture
Kubernetes Architecture
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)
 
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Kubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide
 
DevOps with Kubernetes
DevOps with KubernetesDevOps with Kubernetes
DevOps with Kubernetes
 
An Introduction to Kubernetes
An Introduction to KubernetesAn Introduction to Kubernetes
An Introduction to Kubernetes
 
(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive Overview(Draft) Kubernetes - A Comprehensive Overview
(Draft) Kubernetes - A Comprehensive Overview
 
Deploying your first application with Kubernetes
Deploying your first application with KubernetesDeploying your first application with Kubernetes
Deploying your first application with Kubernetes
 
Kubernetes Architecture and Introduction
Kubernetes Architecture and IntroductionKubernetes Architecture and Introduction
Kubernetes Architecture and Introduction
 
Monitoring kubernetes with prometheus
Monitoring kubernetes with prometheusMonitoring kubernetes with prometheus
Monitoring kubernetes with prometheus
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes Workshop
 

Similaire à DevJam 2019 - Introduction to Kubernetes

Kubernetes presentation
Kubernetes presentationKubernetes presentation
Kubernetes presentationGauranG Bajpai
 
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMwareVMUG IT
 
DevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes IntegrationDevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes IntegrationHank Preston
 
Building Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and KubelessBuilding Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and KubelessBitnami
 
Kubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CDKubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CDStfalcon Meetups
 
Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307Inhye Park
 
Kubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-HassanKubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-HassanSyed Murtaza Hassan
 
Kubernetes Internals
Kubernetes InternalsKubernetes Internals
Kubernetes InternalsShimi Bandiel
 
Containers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes LeoContainers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes LeoLéopold Gault
 
Kubernetes fundamentals
Kubernetes fundamentalsKubernetes fundamentals
Kubernetes fundamentalsVictor Morales
 
An Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery FundamentalsAn Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery FundamentalsAll Things Open
 
Kubernetes for Java developers
Kubernetes for Java developersKubernetes for Java developers
Kubernetes for Java developersRobert Barr
 
Kubernetes 101
Kubernetes 101Kubernetes 101
Kubernetes 101Huy Vo
 
Kubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewKubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewAnkit Shukla
 
Getting started with google kubernetes engine
Getting started with google kubernetes engineGetting started with google kubernetes engine
Getting started with google kubernetes engineShreya Pohekar
 
What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...
What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...
What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...InfluxData
 

Similaire à DevJam 2019 - Introduction to Kubernetes (20)

Kubernetes presentation
Kubernetes presentationKubernetes presentation
Kubernetes presentation
 
Container Orchestration using kubernetes
Container Orchestration using kubernetesContainer Orchestration using kubernetes
Container Orchestration using kubernetes
 
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
01 - VMUGIT - Lecce 2018 - Fabio Rapposelli, VMware
 
DevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes IntegrationDevNetCreate - ACI and Kubernetes Integration
DevNetCreate - ACI and Kubernetes Integration
 
Building Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and KubelessBuilding Cloud-Native Applications with Kubernetes, Helm and Kubeless
Building Cloud-Native Applications with Kubernetes, Helm and Kubeless
 
01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx01. Kubernetes-PPT.pptx
01. Kubernetes-PPT.pptx
 
Kubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CDKubernetes: від знайомства до використання у CI/CD
Kubernetes: від знайомства до використання у CI/CD
 
Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307Docker kubernetes fundamental(pod_service)_190307
Docker kubernetes fundamental(pod_service)_190307
 
Kubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-HassanKubernetes-Presentation-Syed-Murtaza-Hassan
Kubernetes-Presentation-Syed-Murtaza-Hassan
 
Kubernetes Internals
Kubernetes InternalsKubernetes Internals
Kubernetes Internals
 
Containers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes LeoContainers and Kubernetes -Notes Leo
Containers and Kubernetes -Notes Leo
 
Kubernetes fundamentals
Kubernetes fundamentalsKubernetes fundamentals
Kubernetes fundamentals
 
An Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery FundamentalsAn Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery Fundamentals
 
Kubernetes for Java developers
Kubernetes for Java developersKubernetes for Java developers
Kubernetes for Java developers
 
Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
 
Kubernetes 101
Kubernetes 101Kubernetes 101
Kubernetes 101
 
Kubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverviewKubernetes acomprehensiveoverview
Kubernetes acomprehensiveoverview
 
Getting started with google kubernetes engine
Getting started with google kubernetes engineGetting started with google kubernetes engine
Getting started with google kubernetes engine
 
Kubernetes-Meetup
Kubernetes-MeetupKubernetes-Meetup
Kubernetes-Meetup
 
What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...
What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...
What Does Kubernetes Look Like?: Performance Monitoring & Visualization with ...
 

Plus de Ronny Trommer

DevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration APIDevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration APIRonny Trommer
 
Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018Ronny Trommer
 
Dev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMSDev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMSRonny Trommer
 
Dev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache KarafDev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache KarafRonny Trommer
 
Dev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMSDev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMSRonny Trommer
 
DevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time EngineDevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time EngineRonny Trommer
 
OUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMSOUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMSRonny Trommer
 
Who Pulls the Strings?
Who Pulls the Strings?Who Pulls the Strings?
Who Pulls the Strings?Ronny Trommer
 
OSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMSOSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMSRonny Trommer
 

Plus de Ronny Trommer (9)

DevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration APIDevJam 2019 - OpenNMS Integration API
DevJam 2019 - OpenNMS Integration API
 
Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018Dev-Jam 2019 - New since Dev-Jam 2018
Dev-Jam 2019 - New since Dev-Jam 2018
 
Dev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMSDev-Jam 2019 - Container & OpenNMS
Dev-Jam 2019 - Container & OpenNMS
 
Dev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache KarafDev-Jam 2019 - Developing OpenNMS on Apache Karaf
Dev-Jam 2019 - Developing OpenNMS on Apache Karaf
 
Dev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMSDev-Jam 2019 - Developing & Contributing to OpenNMS
Dev-Jam 2019 - Developing & Contributing to OpenNMS
 
DevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time EngineDevJam 2019 - Building an ALEC Time Engine
DevJam 2019 - Building an ALEC Time Engine
 
OUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMSOUCE Reporting Enhancements in OpenNMS
OUCE Reporting Enhancements in OpenNMS
 
Who Pulls the Strings?
Who Pulls the Strings?Who Pulls the Strings?
Who Pulls the Strings?
 
OSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMSOSMC 2011 - Introduction to OpenNMS
OSMC 2011 - Introduction to OpenNMS
 

Dernier

Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 

Dernier (20)

Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 

DevJam 2019 - Introduction to Kubernetes

  • 1. INTRODUCTION TO KUBERNETES OpenNMS DevJam 2019 by Alejandro Galue
  • 2. WHY CONTAINERS • Build, Ship, and Run any App, Anywhere. • No more dependency problems; e.x. no RPM/DEB or dynamic libraries incompatibility issues. • No more: "but it works on my machine", as the same image used in dev is deployed to prod. • Serverless implementations are mainly backed by containers.
  • 3. WHY KUBERNETES • Be able to manage hundreds or thousands of containers, on a fleet of hundred or thousands of nodes. • Be able to deploy an application without worrying about the hardware infrastructure. • Be able to guarantee that applications will stay running according with the specifications. • Be able to facilitate scaling and upgrades not only for the workloads but for the k8s cluster itself.
  • 4. WHAT IS KUBERNETES • A distributed cluster technology that manages container-based systems in a declarative manner using an API (a container orchestrator). • Designed from the ground-up as a loosely coupled collection of components centered around deploying, maintaining and scaling workloads. • Abstracts away the underlying hardware of the nodes and provides a uniform interface for workloads to be both deployed and consume the shared pool of resources. • Works as an engine for resolving state by converging actual and the desired state of the system (self-healing).
  • 5. A COUPLE OF KEY CONCEPTS…
  • 6. PODS • Atomic unit or smallest “unit of work”of Kubernetes. • Pods are one or MORE containers that share volumes, a network namespace, and are a part of a single context. Host A Pod ExternalVolume container container Network Namespace 10.255.16.3 Pod Network Ephemeral
  • 7. SERVICES • Unified method of accessing the exposed workloads of Pods. • Durable resource - static cluster IP - static namespaced DNS name Host A Labels: app=nginx env=prod Labels: app=nginx env=prod Labels: app=mysql env=dev Host B Labels: app=nginx env=prod Labels: app=mysql env=prod Labels: app=nginx env=dev Service app=nginx env=prod NOT Ephemeral
  • 9. Control Plane - 1,2,…n kube controller manager kube scheduler kube apiserver kubectl Node 1 Containers System Services kubelet kube-proxy Container Runtime Pod Pod Node 2 Containers System Services kubelet kube-proxy Container Runtime Pod Pod cloud controller manager Cloud Provider API Load Balancer End Users Cloud Provider Network Edge
  • 10. Control Plane - 1,2,…n Containers kube-control-manager kube-scheduler kube apiserver kubectl Node 1 Containers System Services kubelet kube-proxy Container Runtime Pod Pod Node 2 Containers System Services kubelet kube-proxy Container Runtime Pod Pod cloud-controller-manager Cloud Provider API Load Balancer End Users Cloud Provider Network Edge System Services kubelet Container Runtime additional services kube-proxy
  • 11. KUBE-APISERVER • Provides a forward facing REST interface into the kubernetes control plane and datastore. • All clients and other applications interact with kubernetes strictly through the API Server. • Acts as the gatekeeper to the cluster by handling authentication and authorization, request validation, mutation, and admission control in addition to being the front-end to the backing datastore. master
  • 12. • etcd acts as the cluster datastore. • Purpose in relation to Kubernetes is to provide a strong, consistent and highly available key-value store for persisting cluster state. • Stores objects and config information. • Uses “Raft Consensus” among a quorum of systems to create a fault-tolerant consistent “view” of the cluster. master
  • 13. KUBE-CONTROLLER-MANAGER • Serves as the primary daemon that manages all core component control loops. • Monitors the cluster state via the apiserver and steers the cluster towards the desired state. master
  • 14. KUBE-SCHEDULER • Component on the master that watches newly created pods that have no node assigned, and selects a node for them to run on. • Factors taken into account for scheduling decisions include individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference and deadlines. master
  • 15. CLOUD-CONTROLLER-MANAGER • Daemon that provides cloud-provider specific knowledge and integration capability into the core control loop of Kubernetes. • The controllers include Node, Route, Service, and add an additional controller to handle things such as PersistentVolume Labels. master Optional
  • 16. KUBE-PROXY • Manages the network rules on each node. • Performs connection forwarding or load balancing for Kubernetes cluster services. worker
  • 17. KUBELET • An agent that runs on each node in the cluster. It makes sure that containers are running in a pod. • The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. worker
  • 18. CONTAINER RUNTIME ENGINE A container runtime is a CRI (Container Runtime Interface) compatible application that executes and manages containers. • containerd (Docker) • cri-o • rkt • kata (formerly clear and hyper) • virtlet (VM CRI compatible runtime) worker
  • 19. KUBERNETES NETWORKING • Pod Network - Cluster-wide network used for pod-to-pod communication managed by a CNI (Container Network Interface) plugin. • Service Network - Cluster-wide range of Virtual IPs managed by kube-proxy for service discovery.
  • 20. FUNDAMENTAL NETWORKING RULES • All containers within a pod can communicate with each other unimpeded. • All Pods can communicate with all other Pods without NAT. • All nodes can communicate with all Pods (and vice-versa) without NAT. • The IP that a Pod sees itself as is the same IP that others see it as. What if I want to limit communication within Pods ? Learn about Network Policies (and make sure the chosen CNI supports it)
  • 21. API
  • 22. API OVERVIEW • The REST API is the true keystone of Kubernetes. • Everything within Kubernetes is as an API Object. • Referenced within an object as the apiVersion and kind. Format: /apis/<group>/<version>/<resource> Examples: /apis/apps/v1/deployments /apis/batch/v1beta1/cronjobs
  • 23. OBJECT MODEL • Objects are a “record of intent” or a persistent entity that represent the desired state of the object within the cluster. • All objects MUST have  apiVersion, kind, and poses the nested fields metadata.name, metadata.namespace, and metadata.uid.
  • 24. OBJECT EXPRESSION -YAML • Files or other representations of Kubernetes Objects are generally represented in YAML. • Three basic data types: - mappings  - hash or dictionary, - sequences - array or list - scalars - string, number, boolean etc apiVersion: v1 kind: Pod metadata: name: sample namespace: test spec: containers: - name: container1
 image: nginx - name: container2
 image: alpine
  • 25. YAMLVS JSON {   "apiVersion": "v1",   "kind": "Pod",   "metadata": {     "name": "pod-example"   },   "spec": {     "containers": [       {         "name": "nginx",         "image": "nginx:stable-alpine",         "ports": [ { "containerPort": 80 } ]       }     ]   } } apiVersion: v1 kind: Pod metadata:   name: pod-example spec:   containers:   - name: nginx     image: nginx:stable-alpine     ports:     - containerPort: 80 Are you wondering about theYAML schema ? kubectl explain is your friend ;) What about kinds or versions ? kubectl api-versions is your friend ;)
  • 27. NAMESPACES Namespaces are a logical cluster or environment, and are the primary method of partitioning a cluster or scoping access. apiVersion: v1 kind: Namespace metadata:   name: prod   labels:     app: MyBigWebApp $ kubectl get ns --show-labels NAME          STATUS    AGE       LABELS default       Active    11h       <none> kube-public   Active    11h       <none> kube-system   Active    11h       <none> prod          Active    6s        app=MyBigWebApp Known as Projects in OpenShift
  • 28. PODS • Atomic unit or smallest “unit of work”of Kubernetes. • Foundational building block of Kubernetes Workloads. • Pods are one or MORE containers that share volumes, a network namespace, and are a part of a single context. Host A Pod ExternalVolume container container Network Namespace 10.255.16.3 Pod Network apiVersion: v1 kind: Pod metadata: name: sample spec: containers: - name: nginx image: nginx:stable-alpine ports:   - containerPort: 80     name: http     protocol: TCP env: - name: MYVAR value: isAwesome command: [“/bin/sh”, “-c”] args: [“echo ${MYVAR}”] readinessProbe: tcpSocket: port: http initialDelaySeconds: 10 periodSeconds: 10 livenessProbe: httpGet: path: / port: http initialDelaySeconds: 30 periodSeconds: 60 Array of ports to expose Name of the container Container ImageArray of environment variables Entrypoint Array ~ Docker ENTRYPOINT Arguments to pass to the command ~ Docker CMD Tells when the application is ready to receive requests Checks if the container is still alive (running)
  • 29. LABELS • key-value pairs that are used to identify, describe and group together related sets of objects or resources. • NOT characteristic of uniqueness. • Have a strict syntax with a slightly limited character set*. https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set Host A Pod container Network Namespace 10.255.16.3 Pod Network Labels: gpu=nvidia Labels: app=nginx env=prod apiVersion: v1 kind: Pod metadata:   name: pod-label-example   labels:
     app: nginx
     env: prod spec:   containers:   - name: nginx     image: nginx:stable-alpine     ports:     - containerPort: 80
  • 30. SELECTORS Selectors use labels to filter or select objects, and are used throughout Kubernetes. Host A Pod mlapp Network Namespace 10.255.16.3 Pod Network Labels: gpu=nvidia Labels: app=mlapp env=prodapiVersion: v1 kind: Pod metadata:   name: mlapp   labels:
     app: mlapp
     env: prod spec:   nodeSelector:   - gpu: nvidia   containers:   - name: nginx     image: tensorflow/tensorflow Host B Labels: gpu=amd
  • 31. SERVICES • Unified method of accessing the exposed workloads of Pods. • Durable resource (unlike Pods) - static cluster-unique IP - static namespaced DNS name <service name>.<namespace>.svc.cluster.local There are 4 major service types: • ClusterIP (default) • NodePort • LoadBalancer • ExternalName
  • 32. CLUSTER IP SERVICE Labels: app=nginx env=prod kube-proxy Host A Labels: app=nginx env=prod Host B Pod Host C Pod Network kube-proxy kube-proxy Name:         example-prod Selector:     app=nginx,env=prod Type:         ClusterIP IP:           10.96.28.176 Port:         <unset>  80/TCP TargetPort:   80/TCP Endpoints:    10.255.16.3:80,
               10.255.16.4:80 • The Pod on host C requests the service. • Hits host iptables and it load-balances the connection between the endpoints residing on Hosts A, B / # nslookup example-prod.default.svc.cluster.local Name:      example-prod.default.svc.cluster.local Address 1: 10.96.28.176 example-prod.default.svc.cluster.local
  • 33. NODE PORT SERVICE Labels: app=nginx env=prod kube-proxy Host A Labels: app=nginx env=prod Host B Labels: app=nginx env=dev Host C Name:         example-prod Selector:     app=nginx,env=prod Type:         NodePort IP:           10.96.28.176 Port:         <unset>  80/TCP TargetPort:   80/TCP NodePort:     <unset>  32410/TCP Endpoints:    10.255.16.3:80,
               10.255.16.4:80 • User can hit any host in cluster on NodePort IP and get to service. • Does introduce extra hop if hitting a host without instance of the pod. 32410 32410kube-proxy 32410kube-proxy
  • 34. LOAD BALANCER SERVICE • LoadBalancer services extend NodePort. • Works in conjunction with an external system to map a cluster external IP to the exposed service. Labels: app=nginx env=prod kube-proxy Host A Labels: app=nginx env=prod Host B Labels: app=nginx env=dev Host C 32410 32410kube-proxy 32410kube-proxy Load Balancer Cloud Provider Edge Network Name:         example-prod Selector:     app=nginx,env=prod Type:         LoadBalancer IP:           10.96.28.176 LoadBalancer
 Ingress:      172.17.18.43 Port:         <unset>  80/TCP TargetPort:   80/TCP NodePort:     <unset>  32410/TCP Endpoints:    10.255.16.3:80,
               10.255.16.4:80
  • 35. INGRESSKnown as Routes in OpenShift • An API object that manages external access to the services in a cluster, through a single LoadBalancer. • Provides load balancing, SSL termination and name/path-based virtual hosting • Gives services externally-reachable URLs. • Multiple implementations to choose from. service: grafana port=3000 Host A service: kibana port=5601 Host B service: opennms port=8980 Host C grafana.example.com kibana.example.com Cloud Provider Edge Network Ingress Controller Host D Load Balancer onms.example.com apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-rules annotations: kubernetes.io/ingress.class: nginx spec: rules: - host: onms.example.com http: paths: - path: / backend: serviceName: opennms servicePort: 8980 …
  • 36. EXTERNAL NAME SERVICE • ExternalName is used to reference endpoints OUTSIDE the cluster. • Creates an internal CNAME DNS entry that aliases another. apiVersion: v1 kind: Service metadata:   name: example-prod spec:   type: ExternalName spec:   externalName: example.com
  • 38. WORKLOADS • Workloads within Kubernetes are higher level objects that manage Pods or other higher level objects. • In ALL CASES a Pod Template is included, and acts the base tier of management. •ReplicaSet •Deployment •DaemonSet •StatefulSet •Job •CronJob
  • 39. PODTEMPLATE • Workload Controllers manage instances of Pods based off a provided template. • Pod Templates are Pod specs with limited metadata. • Controllers use Pod Templates to make actual pods apiVersion: v1 kind: Pod metadata:   name: pod-example   labels:
     app: nginx spec:   containers:   - name: nginx     image: nginx template:   metadata:     labels:
       app: nginx   spec:     containers:     - name: nginx       image: nginx apiVersion: apps/v1 kind: Deployment metadata:   name: deployment-example   labels:
     app: nginx spec: replicas: 3   selector:     matchLabels:
       app: nginx template:   metadata:     labels:
       app: nginx   spec:     containers:     - name: nginx       image: nginx
  • 40. RESOURCE MODEL • Request: amount of a resource allowed to be used, with a strong guarantee of availability. - CPU (seconds/second), RAM (bytes) - Scheduler will not over-commit requests • Limit: max amount of a resource that can be used, regardless of guarantees - Scheduler ignores limits apiVersion: v1 kind: Pod metadata:   name: pod-example   labels:
     app: nginx spec:   containers:   - name: nginx     image: nginx   resources:   requests:   memory: "64Mi"     cpu: "250m"   limits:   memory: "128Mi"     cpu: "500m"
  • 41. INIT CONTAINERS If we compare a container template with a Class definition in Java, an initContainer would be the constructor of a class; while a running Pod, would be an instance of that class. Can be used for everything that should happen before the containers within a Pod start running. For example: wait for dependencies, initialize volumes or databases, verify requirements, etc.
  • 42. REPLICA SET • Primary method of managing pod replicas and their lifecycle. • Includes their scheduling, scaling, and deletion. • Their job is simple: Always ensure the desired number of pods are running. ReplicaSet Pod • replicas: The desired number of instances of the Pod. • selector: The label selector for the ReplicaSet will manage ALL Pod instances that it targets; whether it’s desired or not. apiVersion: apps/v1 kind: ReplicaSet metadata:   name: rs-example spec:   replicas: 3   selector:     matchLabels:       app: nginx
       env: prod   template:     <pod template>
  • 43. DEPLOYMENT • Declarative method of managing Pods via ReplicaSets. • Provide rollback functionality and update control. • Updates are managed through the pod- template-hash label. • Each iteration creates a unique label that is assigned to both the ReplicaSet and subsequent Pods. PodReplicaSetDeployment • revisionHistoryLimit: The number of previous iterations of the Deployment to retain. • strategy: Describes the method of  updating the Pods based on the type.Valid options are: - Recreate: All existing Pods are killed before the new ones are created. - RollingUpdate: Cycles through updating the Pods according to the parameters: maxSurge and maxUnavailable. apiVersion: apps/v1 kind: Deployment metadata:   name: deployment-example spec:   replicas: 3
   revisionHistoryLimit: 3   selector:     matchLabels:       app: nginx
       env: prod   strategy:     type: RollingUpdate     rollingUpdate:       maxSurge: 1       maxUnavailable: 0   template:     <pod template>
  • 44. DAEMON SET • Ensure that all nodes matching certain criteria will run an instance of the supplied Pod. • They bypass default scheduling mechanisms. • Are ideal for cluster wide services such as log forwarding, or health monitoring. • Revisions are managed via a controller-revision-hash label. DaemonSet Pod • spec.template.spec.nodeSelector: The primary selector used to target nodes. • Default Host Labels: - kubernetes.io/hostname - beta.kubernetes.io/os - beta.kubernetes.io/arch • Cloud Host Labels: - failure-domain.beta.kubernetes.io/zone - failure-domain.beta.kubernetes.io/region - beta.kubernetes.io/instance-type apiVersion: apps/v1 kind: DaemonSet metadata:   name: ds-example spec:   revisionHistoryLimit: 3   selector:     matchLabels:       app: nginx   updateStrategy:     type: RollingUpdate     rollingUpdate:       maxUnavailable: 1   template:     spec:       nodeSelector:         nodeType: edge      <pod template>
  • 45. STATEFUL SET • Tailored to managing Pods that must persist or maintain state. • Pod identity including hostname, network, and storage WILL be persisted. • Assigned a unique ordinal name following the convention of ‘<statefulset name>-<ordinal index>’. • Naming convention is also used in Pod’s network Identity and Volumes. • Pod lifecycle will be ordered and follow consistent patterns. • Revisions are managed via a controller-revision-hash label. Pod apiVersion: apps/v1 kind: StatefulSet metadata:   name: sts-cassandra spec:   replicas: 3   selector:     matchLabels:       app: cassandra   serviceName: cassandra   updateStrategy:    type: RollingUpdate    rollingUpdate:      partition: 0   template:     metadata:       labels:         app: cassandra     spec:       containers:       - name: cassandra-node         image: cassandra:3.11.4         env:         - name: CASSANDRA_SEEDS         value: sts-cassandra-0.cassandra         - name: CASSANDRA_CLUSTER_NAME         value: my-cluster         ports:         - containerPort: 7000         - containerPort: 7199         - containerPort: 9042         volumeMounts:         - name: data           mountPath: /cassandra_data   volumeClaimTemplates:   - metadata:       name: data     spec:       accessModes: [ "ReadWriteOnce" ]       storageClassName: standard       resources:         requests:           storage: 100Gi The name of the associated headless service; or a service without a ClusterIP Pods with an ordinal greater than the partition value will be updated one-by-one in reverse order StatefulSet Template of the persistent volume(s) request to use for each instance of the StatefulSet.
  • 46. HEADLESS SERVICE <StatefulSet Name>-<ordinal>.<service name>.<namespace>.svc.cluster.local apiVersion: v1 kind: Service metadata:   name: cassandra spec:   clusterIP: None   selector:     app: cassandra   ports:   - name: intra-node     port: 80   - name: jmx     port: 7199   - name: cql     port: 9042 / # dig cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> cassandra.default.svc.cluster.local +noall +answer ;; global options: +cmd cassandra.default.svc.cluster.local. 5 IN A 172.17.0.5 cassandra.default.svc.cluster.local. 5 IN A 172.17.0.4 cassandra.default.svc.cluster.local. 5 IN A 172.17.0.6 / # dig sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-0.cassandra.default.svc.cluster.local +noall +answer / # dig sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-1.cassandra.default.svc.cluster.local +noall +answer / # dig sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer ; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> sts-cassandra-2.cassandra.default.svc.cluster.local +noall +answer $ kubectl get pods NAME            READY     STATUS    RESTARTS   AGE sts-cassandra-0   1/1       Running   0          11m sts-cassandra-1   1/1       Running   0          11m sts-cassandra-2   1/1       Running   0          11m
  • 47. VOLUME CLAIMTEMPLATE <Volume Name>.<StatefulSet Name>-<ordinal>  volumeClaimTemplates:   - metadata:       name: data     spec:       accessModes: [ "ReadWriteOnce" ]       storageClassName: standard       resources:         requests:           storage: 1Gi Persistent Volumes associated with a StatefulSet will NOT be automatically garbage collected when its associated StatefulSet is deleted. They must manually be removed. $ kubectl get pvc NAME                STATUS    VOLUME                                    CAPACITY   ACCESS MODES   STORAGECLASS   AGE data-cassandra-sts-0 Bound pvc-8baec2a5-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m data-cassandra-sts-1 Bound pvc-95463714-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m data-cassandra-sts-2 Bound pvc-9807c42a-8c77-11e9-a6be-0800275ddaf0 100Gi RWO standard 17m
  • 48. JOB • Job controller ensures one or more pods are executed and successfully terminate. • Will continue to try and execute the job until it satisfies the completion and/or parallelism condition. • Pods are NOT cleaned up until the job itself is deleted. PodJob apiVersion: batch/v1 kind: Job metadata:   name: job-example spec:   backoffLimit: 4   completions: 4   parallelism: 2   template:     spec:       restartPolicy: Never     template:     <pod-template> • backoffLimit: The number of failures before the job itself is considered failed. • completions: The total number of successful completions desired. • parallelism: How many instances of the pod can be run concurrently. • spec.template.spec.restartPolicy:Jobs only support a restartPolicy of type Never or OnFailure.
  • 49. CRONJOB • An extension of the Job Controller, it provides a method of executing jobs on a cron-like schedule. • CronJobs within Kubernetes
  use UTC ONLY. PodJobCronJob apiVersion: batch/v1beta1 kind: CronJob metadata:   name: cronjob-example spec:   schedule: "*/1 * * * *"   successfulJobsHistoryLimit: 3   failedJobsHistoryLimit: 1   jobTemplate:     spec:       completions: 4       parallelism: 2       template:         <pod template> The cron schedule for the job The number of successful jobs to retain The number of failed jobs to retain
  • 50. OPERATORS • Might be the most powerful feature of K8s. • When none of the workloads fits the needs of your app, you can create your own controllers with your own specifications (CRD). • Should be implemented in Go, but there are alternatives. • A given controller usually requires a ServiceAccount with proper RBAC Role to be able to manage workloads (e.x. create and maintain Pods, Services, etc.) apiVersion: acid.zalan.do/v1 kind: postgresql metadata: name: opennms-database spec: teamId: OpenNMS volume: size: 100Gi numberOfInstances: 3 enableMasterLoadBalancer: false enableReplicaLoadBalancer: false users: opennms: - superuser - createdb databases: opennms: opennms postgresql: version: "10"
  • 52. VOLUMES • Storage that is tied to the Pod’s Lifecycle. • A pod can have one or more types of volumes attached to it. • Can be consumed by any of the containers within the pod. • Survive Pod restarts; however their durability beyond that is dependent on the Volume Type.
  • 53. PERSISTENTVOLUMES • A PersistentVolume (PV) represents a storage resource.  • PVs are a cluster wide resource linked to a backing storage provider: NFS, EBS, GCEPersistentDisk, etc. • Generally provisioned by an administrator. • Their lifecycle is handled independently from a pod • CANNOT be attached to a Pod directly. Relies on a  PersistentVolumeClaim (PVC). • A PVC is a namespaced request for storage.
  • 54. STORAGE CLASS • Satisfies a set of requirements instead of mapping to a storage resource directly. • Ensures that an application’s claim for storage is portable across numerous backends or providers.
  • 56. CONFIG MAPS • Externalized data stored within kubernetes. • Can be referenced through several different means: - environment variable - a command line argument (via environment variable) - injected as a file into a volume mount • Can be created from a manifest, literals, directories, or files directly. apiVersion: v1 kind: ConfigMap metadata:   name: manifest-example data:   state: Minnesota   city: Minneapolis   content: |     Look at this,     its multiline!
  • 57. SECRETS • Functionally identical to a ConfigMap. • Stored as base64 encoded content. • Encrypted at rest within etcd (if configured!). • Ideal for username/passwords, certificates or other sensitive information that should not be stored in a container. • Can be created from a manifest, literals, directories, or from files directly. apiVersion: v1 kind: Secret metadata:   name: manifest-secret type: Opaque data:   username: ZXhhbXBsZQ==   password: bXlwYXNzd29yZA== • type: There are three different types of secrets within Kubernetes: - docker-registry - credentials used to authenticate to a container registry - generic/Opaque - literal values from different sources - tls - a certificate based secret • data: Contains key-value pairs of base64 encoded content.
  • 59. BARE-METAL, CLOUD, OR
 LOCAL INSTALLATION Install OpenShift is hard ;) Local PC Installation
 For development and testing • minikube • minishift (for OKD) • oc cluster up (for OKD) • Docker for Mac or Windows • microk8s (for Linux via Snap) Cloud Installation
 Single Command Experience • gcloud container clusters create • kops create cluster • eksctl create cluster • az aks create • az openshift create OpenShift 4 promises this for the first time since its conception. Bare-Metal / On-Premise • kubeadm (single command to spin up a master or a worker) • OpenShift/OKD Ansible Tools On-Premise vs Cloud Managed • Masters are managed for you, and the only additional resources are the instances for worker nodes • Complete integration with cloud services (volumes, network, load balancers, etc.)
  • 60. SECURITY Here is where OpenShift is Excellent!!! https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/
  • 61. THINGS TO KEEP IN MIND • Users (to access cluster through the API). Usually implemented through certificates. • Service Accounts (for controllers and operators, to make changes from within K8s). • RBAC: Roles and RoleBinding, ClusterRole and ClusterRoleBinding (for users, groups and service accounts). • Never run containers as root, and block privilege escalations (OpenShift enforce this by default using SecurityContextConstraint; known I k8s as PodSecurityPolicy). • Limit resources per namespace (total CPUs and memory, number of Pods, …). • Create your own images and keep a private registry (OpenShift offers this by default). • Research about NetworkPolicy (which requires a CNI that supports this feature). • Research about applications like vault (to enhance protection for your secrets). • Research about clair (a tool for vulnerability scans on containers). • Research about kube-bench (a tool that analyzes your cluster for security best practices). • Research about gvisor.dev (a container sandbox focused on security and efficiency). Some inspired by OpenShift
  • 62. ACCESS On user the certificate (signed by the CA used on the API server), the Common Name (CN) will be interpreted as the username, and the Organization (O) as the group where this user belongs to. Use the kubectl command to create the kubeconfig based on the user’s certificate, to access the cluster resources. Define an RBAC Role to connect resources (deployments, pods, ingress, …) with operations (create, delete, list, …) on a given namespace. Define an RBAC RoleBinding to connect a role with subjects (i.e. group, user, service- account) Use a ClusterRole for cluster wide access For a ClusterRole use ClusterRoleBinding Now you have access! cluster-admin O=system:masters
  • 64. HOWTO MONITOR K8S? • Based on Google’s Four Golden Signals: latency, errors, traffic, saturation, use Brendan Gregg’s USE method (Utilization, Saturation, Errors) for Resources, and Tom Wilkie’s RED method (Rate, Errors, Duration) for Services. • Metric Server collects metrics such as CPU and Memory by each pod and node from the Summary API, exposed by Kubelet (via cAdvisor) on each node.This allows the usage of K8s features like the HorizontalPodAutoscaler. • The usage of Prometheus is widely adopted.This solution is based on their own node_exporter (deployed as DaemonSet) which exposes lots of Linux metrics in Prometheus format.
  • 65. MONITOR K8S WITH OPENNMS • OpenNMS has been used to monitor network infrastructure (or network devices). • OpenNMS can monitor servers at some degree, and can do some level of application monitoring; but traditionally other applications are used in this context. • For Kubernetes, Prometheus is wildly adopted, on a level that it is considered the standard. • OpenNMS can be used to expand the functionality of the already existing observability/monitoring tools in Kubernetes world.
  • 66. MONITOR K8S WITH OPENNMS • Have an operator forwarding important K8s events to OpenNMS, so you can use all the event processing infrastructure to react upon changes on your K8s cluster. • There is no such thing as fixed IP addresses in Kubernetes, so forget about Pods. But, if providing support FQDN support in OpenNMS is possible, that can make monitoring K8s services a reality in the future. • Build a solution based on the Metrics API / cAdvisor, or Prometheus (directly or indirectly through the node_exporter), to get data from the running Nodes, Pods and Services. • SNMP could run on Kubernetes nodes; but that’s not an option for Pods.
  • 67. HOWTO DEPLOY OPENNMS IN K8S? • Use a StatefulSet for OpenNMS (1 replica), to take advantage of auto-deployed PVCs, as the dynamic nature of the configuration directory makes OpenNMS a stateful application. Keep in mind "kubectl cp" for transferring files. • Use a StatefulSet for Minions, to take advantage of the persistent network identity, and use Pod name as the minion-ID.This is recommended, even if Minions are stateless by nature. • Use a Deployment for Sentinels, as they are stateless in nature and having a unique identity is not relevant (for now). Although a StatefulSet won’t hurt. • A combination of initContainers, config-maps, and secrets can be used to pre-configure the mandatory settings on all the applications.The PersistentVolume will keep the custom settings added by the user during the lifetime of the product.
  • 68. OPENNMS DEPENDENCIES It depends… a given team should decide if it it worth running PostgreSQL, Kafka, Cassandra and Elasticsearch within K8s. Keep in mind that all databases are stateful in nature, and managing the life-cycle is different on each case. Solutions based on operators when available are preferred.
  • 69. INTEGRATE OPENNMS WITH K8S • OpenNMS can forward everything to Kafka, and even consume events from Kafka. From there, Serverless technologies or Kafka Streams apps deployed into K8s, can be used to create your own horizontally scalable integrations triggered by events, alarms or collected metrics. • What about using OpenNMS as the source of metrics for Horizontal Pod Autoscaling or Cluster Workers auto-scaling ?
  • 70. DEMO
  • 71. OVERWHELMED? Do not forget we haven’t talked about … • Several k8s features not discussed here ;) • Helm (the k8s package manager; the yum or apt-get for k8s) • Service Meshes (Envoy, Istio, Linkerd, …) • Serverless (Kubeless, Fission, Knative, …) • Machine Learning (kubeflow, …) • VMs and Kubernetes (KubeVirt) • Operators In Depth:
 https://operatorhub.io
 https://github.com/operator-framework And more…
 Not a big enough lis ? Check https://landscape.cncf.io
  • 73. THANKYOU! This work has been inspired by a few presentations from: https://github.com/cncf/presentations/tree/master/kubernetes Demo: https://github.com/OpenNMS/opennms-drift-kubernetes