SlideShare une entreprise Scribd logo
1  sur  71
Télécharger pour lire hors ligne
OpenShift Virtualization - VM and Image Lifecycle
Building and packaging OS Images with KVM, qemu-img and podman
Mihai Criveti, CTO Cloud Native and Red Hat Solutions IBM, STSM, RHCE, OpenShift Certified
November 23, 2020
1
OpenShift Virtualization Overview
Container Build Tools: Podman, Skopeo and Buildah
Container Registries
Prerequisities and Build Tools
Create Virtual Machine Wizard
Building OS Images
Automating image builds with Kickstart and Ansible
IBM Cloud: VPC Gen 2 Custom Images
Continuous Image Build
Layered Image Compliance with OpenSCAP 2
OpenShift Virtualization Overview
Overview
• Prerequisites
• KVM and qemu tools for building OS images
• qemu, cloud-init and virtio drivers
• podman and skopeo for building container images
• Virtual Machine Options
• OS Support (RHEl 6-8, Fedora, Windows 10, 2012R2,
2016, 2019)
• Type: Ephemeral or Persistent
• Source: PXE, URL, Container or Disk
• Virtual Machine Disks
• Volume Mode: Filesystem or Block
• Storage volume types: persistentVolumeClaim,
dataVolume, ephemeral, containerDisk, emptyDisk,
cloudInitNoCloud
• CDI - Containerized Data Importer
• CDI supported operations matrix
• CDI operations that require scratch space
• Building container images with KVM
virt-install
• Building a Fedora image
• Installing VirtIO, qemu and cloud-init
• Using qemu-img to compress the image
• Packaging the image as a container using podman
• Enable the OpenShift Registry for external access
• Upload images with podman
• Copy images with skopeo
• Testing OS Images
• QCOW2 images on HTTP Servers or S3 Storage
• Container images from the container registry
• Using the IBM Cloud OS Images!
3
Provision OpenShift Virtualization on IBM Cloud
Deploy ROKS
• Create a ROKS cluster with at least 1 Bare Metal worker node. Make sure you select version 4.5.x or higher.
Install the OpenShift Virtualization Operator and create a cluster
• Install the OpenShift Virtualization Operator (2.4 or higher)
• Create a OpenShift Virtualization Operator Deployment: kubevirt-hyperconverged cluster on the Bare
Metal node(s).
Create a new project namespace
oc new-project virtual-machines
You can now create Virtual Machines by clicking on Workloads > Virtualization.
4
Red Hat Portfolio integration
RHV 4.4 can manage OpenShift Virtualization VMs
OpenShift virtualization is a feature of OpenShift
Container Platform and is delivered, integrated and
managed via the OpenShift Operator framework.
OpenShift Virtualization integrates with Red Hat
Virtualization.
Figure 1: RHV Integration
RHV Console
With RHV 4.4, the RHV Manager allows you to easily
add an existing OpenShift virtualization cluster as an
external provider and then to perform basic
management of underlying VMs side-by-side with
RHV VMs.
Figure 2: RHV Integration
5
Migration
VMware Migration
You can import Virtual Machines from VMware.
RHV Migration
You can import Virtual Machines from RHV.
6
Virtual Machine Types
Ephemeral
• When using a ephemeral storage volume type, or containerDisk.
• The ephemeral image is created when the virtual machine starts and stores all writes locally. The
ephemeral image is discarded when the virtual machine is stopped, restarted, or deleted. The backing
volume (PVC) is not mutated in any way.
Persistent (persistentVolumeClaim)
• When using a persistentVolumeClaim
• Attaches an available PV to a virtual machine. Attaching a PV allows for the virtual machine data to persist
between sessions.
• Importing an existing virtual machine disk into a PVC by using CDI and attaching the PVC to a virtual
machine instance is the recommended method for importing existing virtual machines into OpenShift
Container Platform.
Note CDI: Containerized Data Importer.
7
Storing VMI Disks in the Container Registry Use Cases
Immutable VMIs booting from ephemeral disk
• launch VMI workloads backed by local ephemeral storage.
• VMI workload does not need to remain persistent across VMI restarts and the workload does not require
live migration support.
Questions:
• I this emptyDir?
• What happens when it fills up?
• Can we setup a quota?
8
Storage Features
Features Live Migration Host-assisted VM disk cloning
OpenShift Container Storage: RBD block-mode volumes Yes Yes
OpenShift Virtualization hostpath provisioner No Yes
Other multi-node writable storage Yes (1) Yes (1)
Other single-node writable storage No Yes (2)
1. PVCs must request a ReadWriteMany access mode.
2. PVCs must request a ReadWriteOnce access mode.
You cannot live migrate virtual machines that use: - A storage class with ReadWriteOnce (RWO) access mode -
Passthrough features such as SRI-OV and GPU
9
CDI: Containerized Data Importer
10
CDI: Manual Deployment
11
CDI: Containerized Data Importer Overview
CDI Function
• persistent storage management add-on for Kubernetes.
• provides a declarative way to build Virtual Machine Disks on PVCs for Kubevirt VMs
• provides a way to populate PVCs with VM images or other data upon creation.
• data can come from different sources: a URL, a container registry, another PVC (clone), or an upload from a
client.
Import from URL
This method is selected when you create a DataVolume with an http source. Supports basic authentication
(secret) and custom TLS certificates (ConfigMap).
Import from container registry
When a DataVolume has a registry source CDI will populate the volume with a Container Disk downloaded
from the given image URL.
12
CDI supported operations matrix
DataVolumes
DataVolume objects are custom resources that are provided by the Containerized Data Importer (CDI) project.
DataVolumes orchestrate import, clone, and upload operations that are associated with an underlying
PersistentVolumeClaim (PVC). DataVolumes are integrated with KubeVirt, and they prevent a virtual machine
from being started before the PVC has been prepared.
Understanding scratch space
The Containerized Data Importer (CDI) requires scratch space (temporary storage) to complete some
operations, such as importing and uploading virtual machine images. During this process, the CDI provisions a
scratch space PVC equal to the size of the PVC backing the destination DataVolume (DV). The scratch space
PVC is deleted after the operation completes or aborts.
Note: The CDI requires requesting scratch space with a file volume mode, regardless of the PVC backing the
origin DataVolume. If the origin PVC is backed by block volume mode, you must define a StorageClass capable
of provisioning file volume mode PVCs.
13
CDI operations that require scratch space
Registry imports
The CDI must download the image to a scratch space and extract the layers to find the image file. The image
file is then passed to QEMU-IMG for conversion to a raw disk.
Upload image
QEMU-IMG does not accept input from STDIN. Instead, the image to upload is saved in scratch space before it
can be passed to QEMU-IMG for conversion.
HTTP imports of archived images
QEMU-IMG does not know how to handle the archive formats CDI supports. Instead, the image is unarchived
and saved into scratch space before it is passed to QEMU-IMG.
HTTP imports of authenticated images
QEMU-IMG inadequately handles authentication. Instead, the image is saved to scratch space and
authenticated before it is passed to QEMU-IMG.
HTTP imports of custom certificates
QEMU-IMG inadequately handles custom certificates of HTTPS endpoints. Instead, the CDI downloads the
image to scratch space before passing the file to QEMU-IMG.
14
Container Build Tools: Podman,
Skopeo and Buildah
Podman Overview
What is Podman?
Figure 3: podman - manage pods, containers and OCI compliant container images
How is Podman different?
• Can be run as a regular user without requiring root.
• Can manage pods (groups of one or more containers that operate together).
• Lets you import Kubernetes definitions using podman play.
• Fork-exec model instead of client-server model (containers are child processes of podman).
• Compatible with Docker, Docker Hub or any OCI compliant container implementation.
15
Buildah
What is Buildah?
Figure 4: buildah - build container images from CLI or Dockerfiles
How is Buildah different?
• Containers can be build using simple CLI commands or shell scripts instead of Dockerfiles.
• Images can then be pushed to any container registry and can be used by any container engine, including
Podman, CRI-O, and Docker.
• Buildah is also often used to securely build containers while running inside of a locked down container by a
tool like Podman, OpenShift/Kubernetes or Docker. 16
Skopeo
What is Skopeo?
Figure 5: skopeo - inspect and copy containers and images between different storage
How does Skopeo help?
• It can copy images to and from a host, as well as to other container environments and registries.
• Skopeo can inspect images from container image registries, get images and image layers, and use
signatures to create and verify images.
17
Install podman, buildah and skopeo
Fedora 32 / RHEL 8
# Install podman, buildah and skopeo on Fedora 32
sudo dnf -y install podman buildah skopeo slirp4netns fuse-overlayfs
Ubuntu / Debian
sudo apt update && sudo apt -y install podman buildah skopeo
Getting help
podman version
podman --help # list available commands
man podman-ps # or commands like run, rm, rmi, image, build
podman info # display podman system information
https://podman.io/getting-started/installation
18
Container Registries
Container Image Registries available to OpenShift
quay.io
• Public / private container repository from Red Hat
• Quay is a container image registry that enables you to build, organize, distribute, and deploy containers.
• Quay gives you security over your repositories with image vulnerability scanning and robust access controls.
IBM Cloud Container Registry
• IBM Cloud multi-tenant private image registry
• store and distribute Docker images in a managed, private registry.
Image Registry Operator in OpenShift Container Platform
• The Image Registry Operator installs a single instance of the OpenShift Container Platform registry, and
manages all registry configuration, including setting up registry storage.
• Integrated OpenShift Container Platform registry: built-in container image registry that runs as a standard
workload on the cluster.
• Can be exposed externally (not exposed by default).
Other options
• JFrog Artifactory.
• GitHub / GitLab provide a container registry as well.
• Docker Hub (not recommended, will limit free pulls, security concerns).
19
Private Registry Security: Create a Secret
Get the secret base64
oc whoami | base64
oc whoami -t | base64
Create a Secret in the same namespace as the DataVolume secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: endpoint-secret
labels:
app: containerized-data-importer
type: Opaque
data:
accessKeyId: "" # <optional: your key or user name, base64 encoded>
secretKey: "" # <optional: your secret or password, base64 encoded>
podman create secret.yaml
podman login -u $(oc whoami) -p $(oc whoami -t) $REGISTRY
20
Private Registry Security: Create a Volume
Create your volume
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
...
spec:
source:
registry:
url: "docker://my-private-registry:5000/my-username/my-image"
secretRef: my-docker-creds
certConfigMap: my-registry-certs
...
Get the pvc
oc apply -f datavolume.yaml
oc get pvc
watch oc get pvc,dvs,pod
21
Registry TLS Certificate configuration
Create a ConfigMap with certificates in the same namespace as the DataVolume
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
...
spec:
source:
registry:
url: "docker://my-private-registry-host:5000/my-username/my-image"
certConfigMap: my-registry-certs
...
22
OpenShift Internal Registry
Exposing the OpenShift Container Registry
# Set the DefaultRoute to True:
oc patch configs.imageregistry.operator.openshift.io/cluster 
--patch '{"spec":{"defaultRoute":true}}' --type=merge
Log in with podman
HOST=$(oc get route default-route -n 
openshift-image-registry --template='{{ .spec.host }}')
podman login -u $(oc whoami) -p $(oc whoami -t) --tls-verify=false $HOST
Accessing the registry from inside the cluster
image-registry.openshift-image-registry.svc:5000
Accessing the registry from OUTSIDE the cluster
$HOST/namespace/myimage
23
Prerequisities and Build Tools
Image Build and Management Server
You need a Bare Metal server running RHEL 8 to build images using KVM (virt-install) and setup the virtctl
client.
Install KVM
yum install @virt virt-top libguestfs-tools 
virt-manager virt-install virt-viewer qemu-img
sudo systemctl enable --now libvirtd
Install virtctl client on RHEL 7
subscription-manager repos --enable rhel-7-server-cnv-2.4-rpms
yum -y install kubevirt-virtctl
Install the virtctl client on RHEL 8
subscription-manager repos --enable cnv-2.4-for-rhel-8-x86_64-rpms
dnf -y install kubevirt-virtctl
24
HTTP Server for Images with Basic Access
Install Apache
dnf install -y httpd httpd-tools
systemctl start httpd && systemctl enable httpd
Enable Basic Auth for /etc/apache2/apache2.conf
<Directory "/var/www/html">
AuthType Basic
AuthName "Restricted Content"
AuthUserFile /etc/apache2/.htpasswd
Require valid-user
</Directory>
Create a password file
htpasswd -c /etc/apache2/.htpasswd cmihai
Store OS images
cp image.qcow2 /var/www/html
25
Configure authentication in OpenShift for your HTTP Server
Secret
apiVersion: v1
kind: Secret
metadata:
name: endpoint-secret
labels:
app: containerized-data-importer
type: Opaque
data:
accessKeyId: ""
secretKey: ""
Storage
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
name: my-data-volume
spec:
source:
http:
url: "https://server/image.img"
secretRef: endpoint-secret
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Mi
26
Create Virtual Machine Wizard
Creating Virtual Machines
Creating Virtual Machines can be done with:
• Create Virtual Machine wizard
• YAML file with OpenShift Console
• Using the CLI
• Importing a VMware virtual machine or template
with the virtual machine wizard
Notes
• If either URL or Container are selected as the
Source in the General step, a rootdisk disk is
created and attached to the virtual machine as the
Bootable Disk. You can modify the rootdisk but you
cannot remove it.
• When you create your virtual machine using the
wizard, your virtual machine’s storage medium
must support Read-Write-Many (RWX) PVCs.
• Network Interface Cards (NICs) and storage disks
can be created and attached to virtual machines
after they have been created.
Create ephemeral Fedora VM
Figure 6: New VM from Container Image in internal registry
27
Create VM - General: Source
PXE
• Provision virtual machine from PXE menu.
• Requires a PXE-capable NIC in the cluster.
URL
Provision virtual machine from an image available
from an HTTP or S3 endpoint. Ex:
http://server/image.qcow2 It is possible to
configure basic authentication using a secret and
specify custom TLS certificates in a ConfigMap.
Container
Provision virtual machine from a bootable operating
system container located in a registry accessible from
the cluster. Ex: image-registry.openshift-image-
registry.svc:5000/virtual-machines/windows2016
Disk
Provision virtual machine from a disk. Ex: Select an
existing PVC in the Storage tab.
Source
Figure 7: Internal Registry Container
Note: for an external container registry - you need to
set up a secret with the credentials. Ex:
oc create secret generic <secret_name> 
--from-file=.dockercfg=<.dockercfg> 
--type=kubernetes.io/dockercfg
Source URL
Figure 8: HTTP Source URL
28
Create VM - General: Operating System
The following operating systems are supported (64-bit
x86 only) in OpenShift Virtualization 2.4.
Linux
• Fedora 31 or higher (desktop).
• Red Hat Enterprise Linux 6.x, 7.x, 8.x.
Linux systems need to be built with VirtIO drivers,
cloud-init and SSH enabled.
Windows
• Windows 10 (desktop).
• Windows Server 2012 R2, 2016, 2019.
Windows images need to be built with VirtIO drivers,
sysprep, cloud-init and RDP enabled. When installing
Windows from a DVD, you need to use a SATA disk.
Workload Profile
• High Performance
• Server
• Desktop
Create Virtual Machine: Operating System
Figure 9: Supported Operating Systems
Notes:
• OpenShift Virtualization is certified in Microsoft’s
Windows Server Virtualization Validation Program
(SVVP) to run Windows Server workloads on Red
Hat Enterprise Linux CoreOS 8 workers.
• Other operating system templates shipped with
OpenShift Virtualization are not supported.
29
Create VM - Networking
Attach one or more network interface
Network interface options:
• Model: VirtIO (requires drivers, fastest), e1000, e1000e, ne2kPCI, pcnet, rtl8139 (various simulated
drivers)
• Network: Select a Network Definition in OpenShift.
• Type: bridge, masquerade, sriov
30
Creating additional networks
Edit the networks operator
oc edit networks.operator.openshift.io cluster
Configure additional networks
additionalNetworks:
- name: test-network-1
namespace: default
rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-1",
"type": "bridge", "master":"bond0",
"ipam" :{"type":"static", "addresses":[{"address":"161.156.177.60/28"}]}}'
type: Raw
- name: test-network-2
namespace: default
rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-2",
"type": "bridge", "master":"bond0",
"ipam" :{"type":"static", "addresses":[{"address":"161.156.177.61/28"}]}}'
type: Raw
31
Storage: Add Disk
Disks
Source
One of: Blank, URL, Container (registry image),
Attached Cloned Disk (PVC), Attach Disk (PVC).
Size
GiB when using Blank, URL or Attached Cloned Disks.
Container will be dynamic, and Attach Disk - the size
of the PVC.
Interface
VirtIO, sata or scsi. Pick VirtIO when guest image has
VirtIO drivers installed.
Adding a blank disk
Storage Class
The StorageClass that is used to create the disk. Ex:
ibmc-block-bronze, ibmc-file-silver (as configured in
ROKS). See: oc get storageclass. 32
Storage: Advanced
Disks: Avanced Options
Figure 10: Add Disk: Advanced Options
Volume Mode
• Filesystem: Stores the virtual disk on a
filesystem-based volume.
• Block: Stores the virtual disk directly on the block
volume. Only use Block if the underlying storage
supports it.
Access Mode
• Single User (RWO): The disk can be mounted as
read/write by a single node.
• Shared Access (RWX): The disk can be mounted
as read/write by many nodes. This is required for
some features, such as live migration of virtual
machines between nodes.
• Read Only (ROX): The disk can be mounted as
read-only by many nodes.
33
Storage: Virtual machine storage volume types (persistent)
persistentVolumeClaim
• Attaches an available PV to a virtual machine.
• Attaching a PV allows for the virtual machine data to persist between sessions.
• Importing an existing virtual machine disk into a PVC by using CDI and attaching the PVC to a virtual
machine instance is the recommended method for importing existing virtual machines.
dataVolume
• DataVolumes build on the persistentVolumeClaim disk type by managing the process of preparing the
virtual machine disk via an import, clone, or upload operation.
• VMs that use this volume type are guaranteed not to start until the volume is ready.
• Specify type: dataVolume or type: ””. If you specify any other value for type, such as
persistentVolumeClaim, a warning is displayed, and the virtual machine does not start.
34
Storage: Virtual machine storage volume types (ephemeral)
ephemeral
• A local copy-on-write (COW) image that uses a network volume as a read-only backing store. The backing
volume must be a PersistentVolumeClaim.
• The ephemeral image is created when the virtual machine starts and stores all writes locally and is
discarded when the virtual machine is stopped, restarted, or deleted. The backing volume (PVC) is not
mutated in any way.
containerDisk
• References an image, such as a virtual machine disk, that is stored in the container image registry.
• The image is pulled from the registry and embedded in a volume when the virtual machine is created.
• A containerDisk volume is ephemeral and is discarded when the VM is stopped, restarted, or deleted.
• Container disks are not limited to a single virtual machine and are useful for creating large numbers of
virtual machine clones that do not require persistent storage.
• Only RAW and QCOW2 formats are supported. QCOW2 recommended for reduced image size.
emptyDisk
• Creates an additional sparse QCOW2 disk that is tied to the life-cycle of the virtual machine interface. Used
to store data that otherwise exceeds the limited temporary file system of an ephemeral disk.
• The data survives guest-initiated reboots in the virtual machine but is discarded when the virtual machine
stops or is restarted from the web console.
35
Storage: Virtual machine storage volume types (supporting)
cloudInitNoCloud
Attaches a disk that contains the referenced cloud-init NoCloud data source, providing user data and
metadata to the virtual machine. A cloud-init installation is required inside the virtual machine disk.
Example:
- cloudInitNoCloud:
userData: |
#!/bin/bash
echo "fedora" | passwd fedora --stdin
name: cloudinitdisk
36
Advanced: Cloud-init
Post-provisioning script used to setup users, passwords, insert SSH keys and customize the image.
Example cloud-init
#cloud-config
users:
- default
- name: cmihai
gecos: Mihai Criveti
selinux-user: staff_u
groups: users,wheel
ssh_pwauth: True
ssh_authorized_keys:
- ssh-rsa AA..vz user@domain.com
chpasswd:
list: |
root:password
cmihai:password
expire: False
37
Advanced: Virtual Hardware (CD-ROM)
ISO CD-ROM image
Figure 11: Add a CD-ROM ISO Image from URL
Boot order
You will need to change the boot order if you wish to
boot from the CD by editing the YAML:
oc edit vm example
devices:
disks:
- bootOrder: 1
cdrom:
bus: sata
name: cd-drive-1
38
Building OS Images
Building a Windows Image from ISO
Download container-native-virtualization/virtio-win - Red Hat Container Catalog.
podman login registry.redhat.io
podman pull registry.redhat.io/container-native-virtualization/virtio-win
Create a image disk at least 15GB in size
qemu-img create -f qcow2 w2016.qcow2 15G
Install Windows using virt-install
virt-install --connect qemu:///system 
--name ws2016 --ram 4096 --vcpus 2 
--network network=default,model=virtio 
--disk path=ws2016.qcow2,format=qcow2,device=disk,bus=sata 
--cdrom Windows_Server.ISO 
--disk path=virtio-win-0.1.189.iso,device=cdrom 
--vnc --os-type windows --os-variant win2k16
Installation
• Install QEMU guest agent and VirtIO Drivers
• Configure RDP
• Install Cloud Init and Sysprep the image
39
Building a Red Hat OS Image with Kickstart
Create a Kickstart file (response file)
You can manually install the OS to generate a .ks file. Installations from kickstart are automated, and you can
use this as part of a CI/CD OS build.
Kickstart install the OS
# virt-install 
--name guest1-rhel7 
--memory 2048 
--vcpus 2 
--disk size=8 
--location http://example.com/path/to/os 
--os-variant rhel7 
--initrd-inject /path/to/ks.cfg 
--extra-args="ks=file:/ks.cfg console=tty0 console=ttyS0,115200n8"
Setup QEMU guest agent on virtual machines
systemctl enable qemu-guest-agent
40
Processing and converting images
Compress the image
qemu-img convert -O qcow2 -c w2016.qcow2 windows2016.qcow2
qemu-img convert -O qcow2 -c r7.qcow2 rhel7.qcow2
Create a SHA256 for your images
This is optional, but good practice when uploading your images to a webserver, etc.
sha256sum *qcow2 > SHA256SUMS
Optionally, sign your image with GPG.
gpg --sign myfile
41
Creating and pushing images to the container registry
Create a Dockerfile
FROM scratch
ADD windows2016.qcow2 /disk/
Create a docker container
podman build -t cmihai/windows2016 .
Login to the container registry
REGISTRY="$(oc get route/default-route 
-n openshift-image-registry -o=jsonpath='{.spec.host}')"
podman login ${REGISTRY}
Tag and push the image to your desired namespace (ex: virtual-machines)
podman tag localhost/virtual-machine/fedora31 
${REGISTRY}/virtual-machines/windows2016
podman push ${REGISTRY}/virtual-machines/windows2016
42
Creating container images with Buildah
Create a Dockerfile in /tmp/vmdisk
cat << END > Dockerfile
FROM kubevirt/container-disk-v1alpha
ADD fedora32.qcow2 /disk
END
Build and push to registry
buildah bud -t vmidisk/fedora32:latest /tmp/vmdisk
buildah push --tls-verify=false 
vmidisk/fedora32:latest 
docker://cdi-docker-registry-host.cdi/fedora28:latest
43
Import the registry image into a Data volume
YAML
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
name: fedora31image
spec:
source:
registry:
url: "docker://image-registry.openshift-image-registry.svc:5000/
virtual-machines/fedora31"
pvc:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 20Gi
Get the image info
oc apply -f datavolume.yaml
oc get pvc, dvs, pods # look for importer-fedora31image
44
Uploading local disk images by using the virtctl tool
Creating an upload DataVolume YAML
apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
name: <upload-datavolume>
spec:
source:
upload: {}
pvc:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: <2Gi>
Create the volume
oc create -f <upload-datavolume>.yaml
Upload the image
virtctl image-upload dv <volume_name> 
--size=2G 
--image-path=</path/to/image>
Verify that a DataVolume was created
View all DataVolume objects
oc get dvs
45
Automating image builds with
Kickstart and Ansible
Building a Red Hat OS Image with Kickstart
Create a Kickstart file (response file)
You can manually install the OS to generate a .ks file. Installations from kickstart are automated, and you can
use this as part of a CI/CD OS build.
Kickstart install the OS
# virt-install 
--name guest1-rhel7 
--memory 2048 
--vcpus 2 
--disk size=8 
--location http://example.com/path/to/os 
--os-variant rhel7 
--initrd-inject /path/to/ks.cfg 
--extra-args="ks=file:/ks.cfg console=tty0 console=ttyS0,115200n8"
Setup QEMU guest agent on virtual machines
systemctl enable qemu-guest-agent
46
Kickstart File
RHEL 8 KickStart
ignoredisk --only-use=sda
clearpart --none --initlabel
text
repo --name="AppStream" --baseurl=file:///run/install/repo/AppStream
cdrom
keyboard --vckeymap=us --xlayouts='us'
lang en_US.UTF-8
...
part /boot --fstype="xfs" --ondisk=sda --size=512
part / --fstype="xfs" --ondisk=sda --size=15360
part /home --fstype="xfs" --ondisk=sda --size=10240
%post
sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers
/bin/echo 'UseDNS no' >> /etc/ssh/sshd_config
47
IBM Cloud: VPC Gen 2 Custom
Images
VPC Gen2 Image Format
Custom Image Support for VPC Gen 2
You can create your own custom image, and import it to IBM Cloud™ Virtual Private Cloud infrastructure from
IBM Cloud Object Storage. Then, you can use your custom image to create new virtual server instances that
run on the KVM hypervisor.
This is the same type of image used for OpenShift Virtualization, OpenStack, KVM, RHV, etc.
Requirements
• Contains a single file or volume
• Is in qcow2 format
• Is cloud-init enabled
• The operating system is supported as a stock image operating system (including RHEL 7/8, Windows 2012
R2 and 2016)
• Size doesn’t exceed 100 GB
48
Custom Image Example
49
Continuous Image Build
Packer: build multiple images from a single source
50
Packer: Variables
Variables to parametrized builds and hide secrets
{
"variables": {
"my_secret": "{{env `MY_SECRET`}}",
"not_a_secret": "plaintext",
"foo": "bar"
},
"sensitive-variables": ["my_secret", "foo"],
}
51
Packer: Builders
KVM Builder (qemu)
"builders": [{
"type": "qemu",
"accelerator": "kvm",
"format": "qcow2",
"disk_interface": "virtio-scsi",
"boot_command": [
"<up><wait><tab>",
" text inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/
{user `vm_name`}}.cfg net.ifnames=0",
"<enter><wait>"
],
"vm_name": "{{split build_type "-" 0}}-{{user `vm_name`}}",
"iso_urls": [
"{{user `iso_local_url`}}",
"{{user `iso_download_url`}}"
],
"iso_checksum": "{{user `iso_sha256`}}",
"output_directory": "{{user `builds_dir`}}/{{split build_type "-" 0}}
-{{user `vm_name`}}", }] 52
Provisioners: run post-install tasks
Chaining multiple provisioners
"provisioners": [
{
"type": "shell",
"script": "setup.sh"
},
{
"type": "ansible",
"playbook_file": "{{user `playbook_file`}}"
}],
53
Post-processors: compress or upload your image
Compress, post-process and upload the results
{
"post-processors": [
{
"type": "compress",
"format": "tar.gz"
},
{
"type": "upload",
"endpoint": "http://example.com"
}
]
}
54
Building a VirtualBox image for RHEL 8 using Kickstart
55
Layered Image Compliance with
OpenSCAP
OpenSCAP security report
56
Automatic Remediation as shell, ansible or puppet
57
Continuous Inspection and Automated Compliance
Install OpenSCAP
dnf install openscap-scanner
Generate a report
sudo oscap xccdf eval --report report.html 
--profile xccdf_org.ssgproject.content_profile_pci-dss 
/usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml
58
Cheet Sheet
Get info
oc get vms
59
Get virt-launcher details
Debug virt-launcher and look for the mounted storage
ls run/kubevirt/container-disks
mount | grep disk
60

Contenu connexe

Tendances

Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes IntroductionEric Gustafson
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewBob Killen
 
Red Hat OpenShift Container Platform Overview
Red Hat OpenShift Container Platform OverviewRed Hat OpenShift Container Platform Overview
Red Hat OpenShift Container Platform OverviewJames Falkner
 
Kubernetes Architecture
 Kubernetes Architecture Kubernetes Architecture
Kubernetes ArchitectureKnoldus Inc.
 
OpenShift Overview
OpenShift OverviewOpenShift Overview
OpenShift Overviewroundman
 
Introduction of Kubernetes - Trang Nguyen
Introduction of Kubernetes - Trang NguyenIntroduction of Kubernetes - Trang Nguyen
Introduction of Kubernetes - Trang NguyenTrang Nguyen
 
DevOps with Ansible
DevOps with AnsibleDevOps with Ansible
DevOps with AnsibleSwapnil Jain
 
VMware Tanzu Kubernetes Connect
VMware Tanzu Kubernetes ConnectVMware Tanzu Kubernetes Connect
VMware Tanzu Kubernetes ConnectVMware Tanzu
 
K8s in 3h - Kubernetes Fundamentals Training
K8s in 3h - Kubernetes Fundamentals TrainingK8s in 3h - Kubernetes Fundamentals Training
K8s in 3h - Kubernetes Fundamentals TrainingPiotr Perzyna
 
Open shift 4 infra deep dive
Open shift 4    infra deep diveOpen shift 4    infra deep dive
Open shift 4 infra deep diveWinton Winton
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetesGabriel Carro
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatAmazon Web Services
 
Cloud Native Application
Cloud Native ApplicationCloud Native Application
Cloud Native ApplicationVMUG IT
 
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...ShapeBlue
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Megan O'Keefe
 
Kubernetes - introduction
Kubernetes - introductionKubernetes - introduction
Kubernetes - introductionSparkbit
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopBob Killen
 
Openshift Container Platform
Openshift Container PlatformOpenshift Container Platform
Openshift Container PlatformDLT Solutions
 

Tendances (20)

Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
 
Kubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive OverviewKubernetes - A Comprehensive Overview
Kubernetes - A Comprehensive Overview
 
Red Hat OpenShift Container Platform Overview
Red Hat OpenShift Container Platform OverviewRed Hat OpenShift Container Platform Overview
Red Hat OpenShift Container Platform Overview
 
Kubernetes Architecture
 Kubernetes Architecture Kubernetes Architecture
Kubernetes Architecture
 
OpenShift Overview
OpenShift OverviewOpenShift Overview
OpenShift Overview
 
Introduction of Kubernetes - Trang Nguyen
Introduction of Kubernetes - Trang NguyenIntroduction of Kubernetes - Trang Nguyen
Introduction of Kubernetes - Trang Nguyen
 
DevOps with Ansible
DevOps with AnsibleDevOps with Ansible
DevOps with Ansible
 
VMware Tanzu Kubernetes Connect
VMware Tanzu Kubernetes ConnectVMware Tanzu Kubernetes Connect
VMware Tanzu Kubernetes Connect
 
K8s in 3h - Kubernetes Fundamentals Training
K8s in 3h - Kubernetes Fundamentals TrainingK8s in 3h - Kubernetes Fundamentals Training
K8s in 3h - Kubernetes Fundamentals Training
 
Open shift 4 infra deep dive
Open shift 4    infra deep diveOpen shift 4    infra deep dive
Open shift 4 infra deep dive
 
Introduction to kubernetes
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
 
Containers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red HatContainers Anywhere with OpenShift by Red Hat
Containers Anywhere with OpenShift by Red Hat
 
Cloud Native Application
Cloud Native ApplicationCloud Native Application
Cloud Native Application
 
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
KVM High Availability Regardless of Storage - Gabriel Brascher, VP of Apache ...
 
Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
 
Kubernetes - introduction
Kubernetes - introductionKubernetes - introduction
Kubernetes - introduction
 
Open shift 4-update
Open shift 4-updateOpen shift 4-update
Open shift 4-update
 
Introduction to Kubernetes Workshop
Introduction to Kubernetes WorkshopIntroduction to Kubernetes Workshop
Introduction to Kubernetes Workshop
 
Openshift Container Platform
Openshift Container PlatformOpenshift Container Platform
Openshift Container Platform
 

Similaire à OpenShift Virtualization - VM and OS Image Lifecycle

AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...
AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...
AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...Mihai Criveti
 
OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...
OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...
OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...Mihai Criveti
 
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...Mihai Criveti
 
Docker and IBM Integration Bus
Docker and IBM Integration BusDocker and IBM Integration Bus
Docker and IBM Integration BusGeza Geleji
 
Introduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerIntroduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerChris Taylor
 
Kubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanKubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanMihai Criveti
 
Using Packer to Migrate XenServer Infrastructure to CloudStack
Using Packer to Migrate XenServer Infrastructure to CloudStackUsing Packer to Migrate XenServer Infrastructure to CloudStack
Using Packer to Migrate XenServer Infrastructure to CloudStackTim Mackey
 
Docker introduction
Docker introductionDocker introduction
Docker introductionJo Ee Liew
 
Container and Cloud Native Application: What is VMware doing in this space? -...
Container and Cloud Native Application: What is VMware doing in this space? -...Container and Cloud Native Application: What is VMware doing in this space? -...
Container and Cloud Native Application: What is VMware doing in this space? -...gguglie
 
Docker presentasjon java bin
Docker presentasjon java binDocker presentasjon java bin
Docker presentasjon java binOlve Hansen
 
Ippevent : openshift Introduction
Ippevent : openshift IntroductionIppevent : openshift Introduction
Ippevent : openshift Introductionkanedafromparis
 
Docker-Hanoi @DKT , Presentation about Docker Ecosystem
Docker-Hanoi @DKT , Presentation about Docker EcosystemDocker-Hanoi @DKT , Presentation about Docker Ecosystem
Docker-Hanoi @DKT , Presentation about Docker EcosystemVan Phuc
 
Creating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplaceCreating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplaceAlexey Bokov
 
OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...
OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...
OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...NETWAYS
 
Choosing the Right Framework for Running Docker Containers in Prod
Choosing the Right Framework for Running Docker Containers in ProdChoosing the Right Framework for Running Docker Containers in Prod
Choosing the Right Framework for Running Docker Containers in ProdJosh Padnick
 
What's new in System Center 2012 R2: Virtual Machine Manager
What's new in System Center 2012 R2: Virtual Machine ManagerWhat's new in System Center 2012 R2: Virtual Machine Manager
What's new in System Center 2012 R2: Virtual Machine ManagerTomica Kaniski
 
Develop with docker 2014 aug
Develop with docker 2014 augDevelop with docker 2014 aug
Develop with docker 2014 augVincent De Smet
 

Similaire à OpenShift Virtualization - VM and OS Image Lifecycle (20)

AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...
AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...
AnsibleFest 2021 - DevSecOps with Ansible, OpenShift Virtualization, Packer a...
 
OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...
OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...
OpenShift Commons - Adopting Podman, Skopeo and Buildah for Building and Mana...
 
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
 
Docker and IBM Integration Bus
Docker and IBM Integration BusDocker and IBM Integration Bus
Docker and IBM Integration Bus
 
Docker basic
Docker basicDocker basic
Docker basic
 
Introduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerIntroduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and Docker
 
Kubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanKubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with Podman
 
Using Packer to Migrate XenServer Infrastructure to CloudStack
Using Packer to Migrate XenServer Infrastructure to CloudStackUsing Packer to Migrate XenServer Infrastructure to CloudStack
Using Packer to Migrate XenServer Infrastructure to CloudStack
 
Docker introduction
Docker introductionDocker introduction
Docker introduction
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Container and Cloud Native Application: What is VMware doing in this space? -...
Container and Cloud Native Application: What is VMware doing in this space? -...Container and Cloud Native Application: What is VMware doing in this space? -...
Container and Cloud Native Application: What is VMware doing in this space? -...
 
Docker presentasjon java bin
Docker presentasjon java binDocker presentasjon java bin
Docker presentasjon java bin
 
Ippevent : openshift Introduction
Ippevent : openshift IntroductionIppevent : openshift Introduction
Ippevent : openshift Introduction
 
Docker-Hanoi @DKT , Presentation about Docker Ecosystem
Docker-Hanoi @DKT , Presentation about Docker EcosystemDocker-Hanoi @DKT , Presentation about Docker Ecosystem
Docker-Hanoi @DKT , Presentation about Docker Ecosystem
 
Creating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplaceCreating a gallery image for Azure marketplace
Creating a gallery image for Azure marketplace
 
OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...
OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...
OSDC 2019 | KubeVirt: Converge IT infrastructure into one single Kubernetes p...
 
Choosing the Right Framework for Running Docker Containers in Prod
Choosing the Right Framework for Running Docker Containers in ProdChoosing the Right Framework for Running Docker Containers in Prod
Choosing the Right Framework for Running Docker Containers in Prod
 
10215 A 05
10215 A 0510215 A 05
10215 A 05
 
What's new in System Center 2012 R2: Virtual Machine Manager
What's new in System Center 2012 R2: Virtual Machine ManagerWhat's new in System Center 2012 R2: Virtual Machine Manager
What's new in System Center 2012 R2: Virtual Machine Manager
 
Develop with docker 2014 aug
Develop with docker 2014 augDevelop with docker 2014 aug
Develop with docker 2014 aug
 

Plus de Mihai Criveti

10 Limitations of Large Language Models and Mitigation Options
10 Limitations of Large Language Models and Mitigation Options10 Limitations of Large Language Models and Mitigation Options
10 Limitations of Large Language Models and Mitigation OptionsMihai Criveti
 
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...Mihai Criveti
 
Ansible Workshop for Pythonistas
Ansible Workshop for PythonistasAnsible Workshop for Pythonistas
Ansible Workshop for PythonistasMihai Criveti
 
Mihai Criveti - PyCon Ireland - Automate Everything
Mihai Criveti - PyCon Ireland - Automate EverythingMihai Criveti - PyCon Ireland - Automate Everything
Mihai Criveti - PyCon Ireland - Automate EverythingMihai Criveti
 
Data Science at Scale - The DevOps Approach
Data Science at Scale - The DevOps ApproachData Science at Scale - The DevOps Approach
Data Science at Scale - The DevOps ApproachMihai Criveti
 
ShipItCon - Continuous Deployment and Multicloud with Ansible and Kubernetes
ShipItCon - Continuous Deployment and Multicloud with Ansible and KubernetesShipItCon - Continuous Deployment and Multicloud with Ansible and Kubernetes
ShipItCon - Continuous Deployment and Multicloud with Ansible and KubernetesMihai Criveti
 
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...Mihai Criveti
 
Kubernetes Story - Day 3: Deploying and Scaling Applications on OpenShift
Kubernetes Story - Day 3: Deploying and Scaling Applications on OpenShiftKubernetes Story - Day 3: Deploying and Scaling Applications on OpenShift
Kubernetes Story - Day 3: Deploying and Scaling Applications on OpenShiftMihai Criveti
 
Container Technologies and Transformational value
Container Technologies and Transformational valueContainer Technologies and Transformational value
Container Technologies and Transformational valueMihai Criveti
 

Plus de Mihai Criveti (9)

10 Limitations of Large Language Models and Mitigation Options
10 Limitations of Large Language Models and Mitigation Options10 Limitations of Large Language Models and Mitigation Options
10 Limitations of Large Language Models and Mitigation Options
 
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...
Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...
 
Ansible Workshop for Pythonistas
Ansible Workshop for PythonistasAnsible Workshop for Pythonistas
Ansible Workshop for Pythonistas
 
Mihai Criveti - PyCon Ireland - Automate Everything
Mihai Criveti - PyCon Ireland - Automate EverythingMihai Criveti - PyCon Ireland - Automate Everything
Mihai Criveti - PyCon Ireland - Automate Everything
 
Data Science at Scale - The DevOps Approach
Data Science at Scale - The DevOps ApproachData Science at Scale - The DevOps Approach
Data Science at Scale - The DevOps Approach
 
ShipItCon - Continuous Deployment and Multicloud with Ansible and Kubernetes
ShipItCon - Continuous Deployment and Multicloud with Ansible and KubernetesShipItCon - Continuous Deployment and Multicloud with Ansible and Kubernetes
ShipItCon - Continuous Deployment and Multicloud with Ansible and Kubernetes
 
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
DevOps for Data Engineers - Automate Your Data Science Pipeline with Ansible,...
 
Kubernetes Story - Day 3: Deploying and Scaling Applications on OpenShift
Kubernetes Story - Day 3: Deploying and Scaling Applications on OpenShiftKubernetes Story - Day 3: Deploying and Scaling Applications on OpenShift
Kubernetes Story - Day 3: Deploying and Scaling Applications on OpenShift
 
Container Technologies and Transformational value
Container Technologies and Transformational valueContainer Technologies and Transformational value
Container Technologies and Transformational value
 

Dernier

Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Velvetech LLC
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odishasmiwainfosol
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样umasea
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWave PLM
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0
 
SpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at RuntimeSpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at Runtimeandrehoraa
 
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...OnePlan Solutions
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationBradBedford3
 
Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Hr365.us smith
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsAhmed Mohamed
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaHanief Utama
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commercemanigoyal112
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....kzayra69
 

Dernier (20)

Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
 
2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need It
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作
 
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort ServiceHot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
 
SpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at RuntimeSpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at Runtime
 
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
Tech Tuesday - Mastering Time Management Unlock the Power of OnePlan's Timesh...
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion Application
 
Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)Recruitment Management Software Benefits (Infographic)
Recruitment Management Software Benefits (Infographic)
 
Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML Diagrams
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdf
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief Utama
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commerce
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....What are the key points to focus on before starting to learn ETL Development....
What are the key points to focus on before starting to learn ETL Development....
 

OpenShift Virtualization - VM and OS Image Lifecycle

  • 1. OpenShift Virtualization - VM and Image Lifecycle Building and packaging OS Images with KVM, qemu-img and podman Mihai Criveti, CTO Cloud Native and Red Hat Solutions IBM, STSM, RHCE, OpenShift Certified November 23, 2020 1
  • 2. OpenShift Virtualization Overview Container Build Tools: Podman, Skopeo and Buildah Container Registries Prerequisities and Build Tools Create Virtual Machine Wizard Building OS Images Automating image builds with Kickstart and Ansible IBM Cloud: VPC Gen 2 Custom Images Continuous Image Build Layered Image Compliance with OpenSCAP 2
  • 4. Overview • Prerequisites • KVM and qemu tools for building OS images • qemu, cloud-init and virtio drivers • podman and skopeo for building container images • Virtual Machine Options • OS Support (RHEl 6-8, Fedora, Windows 10, 2012R2, 2016, 2019) • Type: Ephemeral or Persistent • Source: PXE, URL, Container or Disk • Virtual Machine Disks • Volume Mode: Filesystem or Block • Storage volume types: persistentVolumeClaim, dataVolume, ephemeral, containerDisk, emptyDisk, cloudInitNoCloud • CDI - Containerized Data Importer • CDI supported operations matrix • CDI operations that require scratch space • Building container images with KVM virt-install • Building a Fedora image • Installing VirtIO, qemu and cloud-init • Using qemu-img to compress the image • Packaging the image as a container using podman • Enable the OpenShift Registry for external access • Upload images with podman • Copy images with skopeo • Testing OS Images • QCOW2 images on HTTP Servers or S3 Storage • Container images from the container registry • Using the IBM Cloud OS Images! 3
  • 5. Provision OpenShift Virtualization on IBM Cloud Deploy ROKS • Create a ROKS cluster with at least 1 Bare Metal worker node. Make sure you select version 4.5.x or higher. Install the OpenShift Virtualization Operator and create a cluster • Install the OpenShift Virtualization Operator (2.4 or higher) • Create a OpenShift Virtualization Operator Deployment: kubevirt-hyperconverged cluster on the Bare Metal node(s). Create a new project namespace oc new-project virtual-machines You can now create Virtual Machines by clicking on Workloads > Virtualization. 4
  • 6. Red Hat Portfolio integration RHV 4.4 can manage OpenShift Virtualization VMs OpenShift virtualization is a feature of OpenShift Container Platform and is delivered, integrated and managed via the OpenShift Operator framework. OpenShift Virtualization integrates with Red Hat Virtualization. Figure 1: RHV Integration RHV Console With RHV 4.4, the RHV Manager allows you to easily add an existing OpenShift virtualization cluster as an external provider and then to perform basic management of underlying VMs side-by-side with RHV VMs. Figure 2: RHV Integration 5
  • 7. Migration VMware Migration You can import Virtual Machines from VMware. RHV Migration You can import Virtual Machines from RHV. 6
  • 8. Virtual Machine Types Ephemeral • When using a ephemeral storage volume type, or containerDisk. • The ephemeral image is created when the virtual machine starts and stores all writes locally. The ephemeral image is discarded when the virtual machine is stopped, restarted, or deleted. The backing volume (PVC) is not mutated in any way. Persistent (persistentVolumeClaim) • When using a persistentVolumeClaim • Attaches an available PV to a virtual machine. Attaching a PV allows for the virtual machine data to persist between sessions. • Importing an existing virtual machine disk into a PVC by using CDI and attaching the PVC to a virtual machine instance is the recommended method for importing existing virtual machines into OpenShift Container Platform. Note CDI: Containerized Data Importer. 7
  • 9. Storing VMI Disks in the Container Registry Use Cases Immutable VMIs booting from ephemeral disk • launch VMI workloads backed by local ephemeral storage. • VMI workload does not need to remain persistent across VMI restarts and the workload does not require live migration support. Questions: • I this emptyDir? • What happens when it fills up? • Can we setup a quota? 8
  • 10. Storage Features Features Live Migration Host-assisted VM disk cloning OpenShift Container Storage: RBD block-mode volumes Yes Yes OpenShift Virtualization hostpath provisioner No Yes Other multi-node writable storage Yes (1) Yes (1) Other single-node writable storage No Yes (2) 1. PVCs must request a ReadWriteMany access mode. 2. PVCs must request a ReadWriteOnce access mode. You cannot live migrate virtual machines that use: - A storage class with ReadWriteOnce (RWO) access mode - Passthrough features such as SRI-OV and GPU 9
  • 11. CDI: Containerized Data Importer 10
  • 13. CDI: Containerized Data Importer Overview CDI Function • persistent storage management add-on for Kubernetes. • provides a declarative way to build Virtual Machine Disks on PVCs for Kubevirt VMs • provides a way to populate PVCs with VM images or other data upon creation. • data can come from different sources: a URL, a container registry, another PVC (clone), or an upload from a client. Import from URL This method is selected when you create a DataVolume with an http source. Supports basic authentication (secret) and custom TLS certificates (ConfigMap). Import from container registry When a DataVolume has a registry source CDI will populate the volume with a Container Disk downloaded from the given image URL. 12
  • 14. CDI supported operations matrix DataVolumes DataVolume objects are custom resources that are provided by the Containerized Data Importer (CDI) project. DataVolumes orchestrate import, clone, and upload operations that are associated with an underlying PersistentVolumeClaim (PVC). DataVolumes are integrated with KubeVirt, and they prevent a virtual machine from being started before the PVC has been prepared. Understanding scratch space The Containerized Data Importer (CDI) requires scratch space (temporary storage) to complete some operations, such as importing and uploading virtual machine images. During this process, the CDI provisions a scratch space PVC equal to the size of the PVC backing the destination DataVolume (DV). The scratch space PVC is deleted after the operation completes or aborts. Note: The CDI requires requesting scratch space with a file volume mode, regardless of the PVC backing the origin DataVolume. If the origin PVC is backed by block volume mode, you must define a StorageClass capable of provisioning file volume mode PVCs. 13
  • 15. CDI operations that require scratch space Registry imports The CDI must download the image to a scratch space and extract the layers to find the image file. The image file is then passed to QEMU-IMG for conversion to a raw disk. Upload image QEMU-IMG does not accept input from STDIN. Instead, the image to upload is saved in scratch space before it can be passed to QEMU-IMG for conversion. HTTP imports of archived images QEMU-IMG does not know how to handle the archive formats CDI supports. Instead, the image is unarchived and saved into scratch space before it is passed to QEMU-IMG. HTTP imports of authenticated images QEMU-IMG inadequately handles authentication. Instead, the image is saved to scratch space and authenticated before it is passed to QEMU-IMG. HTTP imports of custom certificates QEMU-IMG inadequately handles custom certificates of HTTPS endpoints. Instead, the CDI downloads the image to scratch space before passing the file to QEMU-IMG. 14
  • 16. Container Build Tools: Podman, Skopeo and Buildah
  • 17. Podman Overview What is Podman? Figure 3: podman - manage pods, containers and OCI compliant container images How is Podman different? • Can be run as a regular user without requiring root. • Can manage pods (groups of one or more containers that operate together). • Lets you import Kubernetes definitions using podman play. • Fork-exec model instead of client-server model (containers are child processes of podman). • Compatible with Docker, Docker Hub or any OCI compliant container implementation. 15
  • 18. Buildah What is Buildah? Figure 4: buildah - build container images from CLI or Dockerfiles How is Buildah different? • Containers can be build using simple CLI commands or shell scripts instead of Dockerfiles. • Images can then be pushed to any container registry and can be used by any container engine, including Podman, CRI-O, and Docker. • Buildah is also often used to securely build containers while running inside of a locked down container by a tool like Podman, OpenShift/Kubernetes or Docker. 16
  • 19. Skopeo What is Skopeo? Figure 5: skopeo - inspect and copy containers and images between different storage How does Skopeo help? • It can copy images to and from a host, as well as to other container environments and registries. • Skopeo can inspect images from container image registries, get images and image layers, and use signatures to create and verify images. 17
  • 20. Install podman, buildah and skopeo Fedora 32 / RHEL 8 # Install podman, buildah and skopeo on Fedora 32 sudo dnf -y install podman buildah skopeo slirp4netns fuse-overlayfs Ubuntu / Debian sudo apt update && sudo apt -y install podman buildah skopeo Getting help podman version podman --help # list available commands man podman-ps # or commands like run, rm, rmi, image, build podman info # display podman system information https://podman.io/getting-started/installation 18
  • 22. Container Image Registries available to OpenShift quay.io • Public / private container repository from Red Hat • Quay is a container image registry that enables you to build, organize, distribute, and deploy containers. • Quay gives you security over your repositories with image vulnerability scanning and robust access controls. IBM Cloud Container Registry • IBM Cloud multi-tenant private image registry • store and distribute Docker images in a managed, private registry. Image Registry Operator in OpenShift Container Platform • The Image Registry Operator installs a single instance of the OpenShift Container Platform registry, and manages all registry configuration, including setting up registry storage. • Integrated OpenShift Container Platform registry: built-in container image registry that runs as a standard workload on the cluster. • Can be exposed externally (not exposed by default). Other options • JFrog Artifactory. • GitHub / GitLab provide a container registry as well. • Docker Hub (not recommended, will limit free pulls, security concerns). 19
  • 23. Private Registry Security: Create a Secret Get the secret base64 oc whoami | base64 oc whoami -t | base64 Create a Secret in the same namespace as the DataVolume secret.yaml apiVersion: v1 kind: Secret metadata: name: endpoint-secret labels: app: containerized-data-importer type: Opaque data: accessKeyId: "" # <optional: your key or user name, base64 encoded> secretKey: "" # <optional: your secret or password, base64 encoded> podman create secret.yaml podman login -u $(oc whoami) -p $(oc whoami -t) $REGISTRY 20
  • 24. Private Registry Security: Create a Volume Create your volume apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume ... spec: source: registry: url: "docker://my-private-registry:5000/my-username/my-image" secretRef: my-docker-creds certConfigMap: my-registry-certs ... Get the pvc oc apply -f datavolume.yaml oc get pvc watch oc get pvc,dvs,pod 21
  • 25. Registry TLS Certificate configuration Create a ConfigMap with certificates in the same namespace as the DataVolume apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume ... spec: source: registry: url: "docker://my-private-registry-host:5000/my-username/my-image" certConfigMap: my-registry-certs ... 22
  • 26. OpenShift Internal Registry Exposing the OpenShift Container Registry # Set the DefaultRoute to True: oc patch configs.imageregistry.operator.openshift.io/cluster --patch '{"spec":{"defaultRoute":true}}' --type=merge Log in with podman HOST=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') podman login -u $(oc whoami) -p $(oc whoami -t) --tls-verify=false $HOST Accessing the registry from inside the cluster image-registry.openshift-image-registry.svc:5000 Accessing the registry from OUTSIDE the cluster $HOST/namespace/myimage 23
  • 28. Image Build and Management Server You need a Bare Metal server running RHEL 8 to build images using KVM (virt-install) and setup the virtctl client. Install KVM yum install @virt virt-top libguestfs-tools virt-manager virt-install virt-viewer qemu-img sudo systemctl enable --now libvirtd Install virtctl client on RHEL 7 subscription-manager repos --enable rhel-7-server-cnv-2.4-rpms yum -y install kubevirt-virtctl Install the virtctl client on RHEL 8 subscription-manager repos --enable cnv-2.4-for-rhel-8-x86_64-rpms dnf -y install kubevirt-virtctl 24
  • 29. HTTP Server for Images with Basic Access Install Apache dnf install -y httpd httpd-tools systemctl start httpd && systemctl enable httpd Enable Basic Auth for /etc/apache2/apache2.conf <Directory "/var/www/html"> AuthType Basic AuthName "Restricted Content" AuthUserFile /etc/apache2/.htpasswd Require valid-user </Directory> Create a password file htpasswd -c /etc/apache2/.htpasswd cmihai Store OS images cp image.qcow2 /var/www/html 25
  • 30. Configure authentication in OpenShift for your HTTP Server Secret apiVersion: v1 kind: Secret metadata: name: endpoint-secret labels: app: containerized-data-importer type: Opaque data: accessKeyId: "" secretKey: "" Storage apiVersion: cdi.kubevirt.io/v1beta1 kind: DataVolume metadata: name: my-data-volume spec: source: http: url: "https://server/image.img" secretRef: endpoint-secret pvc: accessModes: - ReadWriteOnce resources: requests: storage: 500Mi 26
  • 32. Creating Virtual Machines Creating Virtual Machines can be done with: • Create Virtual Machine wizard • YAML file with OpenShift Console • Using the CLI • Importing a VMware virtual machine or template with the virtual machine wizard Notes • If either URL or Container are selected as the Source in the General step, a rootdisk disk is created and attached to the virtual machine as the Bootable Disk. You can modify the rootdisk but you cannot remove it. • When you create your virtual machine using the wizard, your virtual machine’s storage medium must support Read-Write-Many (RWX) PVCs. • Network Interface Cards (NICs) and storage disks can be created and attached to virtual machines after they have been created. Create ephemeral Fedora VM Figure 6: New VM from Container Image in internal registry 27
  • 33. Create VM - General: Source PXE • Provision virtual machine from PXE menu. • Requires a PXE-capable NIC in the cluster. URL Provision virtual machine from an image available from an HTTP or S3 endpoint. Ex: http://server/image.qcow2 It is possible to configure basic authentication using a secret and specify custom TLS certificates in a ConfigMap. Container Provision virtual machine from a bootable operating system container located in a registry accessible from the cluster. Ex: image-registry.openshift-image- registry.svc:5000/virtual-machines/windows2016 Disk Provision virtual machine from a disk. Ex: Select an existing PVC in the Storage tab. Source Figure 7: Internal Registry Container Note: for an external container registry - you need to set up a secret with the credentials. Ex: oc create secret generic <secret_name> --from-file=.dockercfg=<.dockercfg> --type=kubernetes.io/dockercfg Source URL Figure 8: HTTP Source URL 28
  • 34. Create VM - General: Operating System The following operating systems are supported (64-bit x86 only) in OpenShift Virtualization 2.4. Linux • Fedora 31 or higher (desktop). • Red Hat Enterprise Linux 6.x, 7.x, 8.x. Linux systems need to be built with VirtIO drivers, cloud-init and SSH enabled. Windows • Windows 10 (desktop). • Windows Server 2012 R2, 2016, 2019. Windows images need to be built with VirtIO drivers, sysprep, cloud-init and RDP enabled. When installing Windows from a DVD, you need to use a SATA disk. Workload Profile • High Performance • Server • Desktop Create Virtual Machine: Operating System Figure 9: Supported Operating Systems Notes: • OpenShift Virtualization is certified in Microsoft’s Windows Server Virtualization Validation Program (SVVP) to run Windows Server workloads on Red Hat Enterprise Linux CoreOS 8 workers. • Other operating system templates shipped with OpenShift Virtualization are not supported. 29
  • 35. Create VM - Networking Attach one or more network interface Network interface options: • Model: VirtIO (requires drivers, fastest), e1000, e1000e, ne2kPCI, pcnet, rtl8139 (various simulated drivers) • Network: Select a Network Definition in OpenShift. • Type: bridge, masquerade, sriov 30
  • 36. Creating additional networks Edit the networks operator oc edit networks.operator.openshift.io cluster Configure additional networks additionalNetworks: - name: test-network-1 namespace: default rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-1", "type": "bridge", "master":"bond0", "ipam" :{"type":"static", "addresses":[{"address":"161.156.177.60/28"}]}}' type: Raw - name: test-network-2 namespace: default rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "test-network-2", "type": "bridge", "master":"bond0", "ipam" :{"type":"static", "addresses":[{"address":"161.156.177.61/28"}]}}' type: Raw 31
  • 37. Storage: Add Disk Disks Source One of: Blank, URL, Container (registry image), Attached Cloned Disk (PVC), Attach Disk (PVC). Size GiB when using Blank, URL or Attached Cloned Disks. Container will be dynamic, and Attach Disk - the size of the PVC. Interface VirtIO, sata or scsi. Pick VirtIO when guest image has VirtIO drivers installed. Adding a blank disk Storage Class The StorageClass that is used to create the disk. Ex: ibmc-block-bronze, ibmc-file-silver (as configured in ROKS). See: oc get storageclass. 32
  • 38. Storage: Advanced Disks: Avanced Options Figure 10: Add Disk: Advanced Options Volume Mode • Filesystem: Stores the virtual disk on a filesystem-based volume. • Block: Stores the virtual disk directly on the block volume. Only use Block if the underlying storage supports it. Access Mode • Single User (RWO): The disk can be mounted as read/write by a single node. • Shared Access (RWX): The disk can be mounted as read/write by many nodes. This is required for some features, such as live migration of virtual machines between nodes. • Read Only (ROX): The disk can be mounted as read-only by many nodes. 33
  • 39. Storage: Virtual machine storage volume types (persistent) persistentVolumeClaim • Attaches an available PV to a virtual machine. • Attaching a PV allows for the virtual machine data to persist between sessions. • Importing an existing virtual machine disk into a PVC by using CDI and attaching the PVC to a virtual machine instance is the recommended method for importing existing virtual machines. dataVolume • DataVolumes build on the persistentVolumeClaim disk type by managing the process of preparing the virtual machine disk via an import, clone, or upload operation. • VMs that use this volume type are guaranteed not to start until the volume is ready. • Specify type: dataVolume or type: ””. If you specify any other value for type, such as persistentVolumeClaim, a warning is displayed, and the virtual machine does not start. 34
  • 40. Storage: Virtual machine storage volume types (ephemeral) ephemeral • A local copy-on-write (COW) image that uses a network volume as a read-only backing store. The backing volume must be a PersistentVolumeClaim. • The ephemeral image is created when the virtual machine starts and stores all writes locally and is discarded when the virtual machine is stopped, restarted, or deleted. The backing volume (PVC) is not mutated in any way. containerDisk • References an image, such as a virtual machine disk, that is stored in the container image registry. • The image is pulled from the registry and embedded in a volume when the virtual machine is created. • A containerDisk volume is ephemeral and is discarded when the VM is stopped, restarted, or deleted. • Container disks are not limited to a single virtual machine and are useful for creating large numbers of virtual machine clones that do not require persistent storage. • Only RAW and QCOW2 formats are supported. QCOW2 recommended for reduced image size. emptyDisk • Creates an additional sparse QCOW2 disk that is tied to the life-cycle of the virtual machine interface. Used to store data that otherwise exceeds the limited temporary file system of an ephemeral disk. • The data survives guest-initiated reboots in the virtual machine but is discarded when the virtual machine stops or is restarted from the web console. 35
  • 41. Storage: Virtual machine storage volume types (supporting) cloudInitNoCloud Attaches a disk that contains the referenced cloud-init NoCloud data source, providing user data and metadata to the virtual machine. A cloud-init installation is required inside the virtual machine disk. Example: - cloudInitNoCloud: userData: | #!/bin/bash echo "fedora" | passwd fedora --stdin name: cloudinitdisk 36
  • 42. Advanced: Cloud-init Post-provisioning script used to setup users, passwords, insert SSH keys and customize the image. Example cloud-init #cloud-config users: - default - name: cmihai gecos: Mihai Criveti selinux-user: staff_u groups: users,wheel ssh_pwauth: True ssh_authorized_keys: - ssh-rsa AA..vz user@domain.com chpasswd: list: | root:password cmihai:password expire: False 37
  • 43. Advanced: Virtual Hardware (CD-ROM) ISO CD-ROM image Figure 11: Add a CD-ROM ISO Image from URL Boot order You will need to change the boot order if you wish to boot from the CD by editing the YAML: oc edit vm example devices: disks: - bootOrder: 1 cdrom: bus: sata name: cd-drive-1 38
  • 45. Building a Windows Image from ISO Download container-native-virtualization/virtio-win - Red Hat Container Catalog. podman login registry.redhat.io podman pull registry.redhat.io/container-native-virtualization/virtio-win Create a image disk at least 15GB in size qemu-img create -f qcow2 w2016.qcow2 15G Install Windows using virt-install virt-install --connect qemu:///system --name ws2016 --ram 4096 --vcpus 2 --network network=default,model=virtio --disk path=ws2016.qcow2,format=qcow2,device=disk,bus=sata --cdrom Windows_Server.ISO --disk path=virtio-win-0.1.189.iso,device=cdrom --vnc --os-type windows --os-variant win2k16 Installation • Install QEMU guest agent and VirtIO Drivers • Configure RDP • Install Cloud Init and Sysprep the image 39
  • 46. Building a Red Hat OS Image with Kickstart Create a Kickstart file (response file) You can manually install the OS to generate a .ks file. Installations from kickstart are automated, and you can use this as part of a CI/CD OS build. Kickstart install the OS # virt-install --name guest1-rhel7 --memory 2048 --vcpus 2 --disk size=8 --location http://example.com/path/to/os --os-variant rhel7 --initrd-inject /path/to/ks.cfg --extra-args="ks=file:/ks.cfg console=tty0 console=ttyS0,115200n8" Setup QEMU guest agent on virtual machines systemctl enable qemu-guest-agent 40
  • 47. Processing and converting images Compress the image qemu-img convert -O qcow2 -c w2016.qcow2 windows2016.qcow2 qemu-img convert -O qcow2 -c r7.qcow2 rhel7.qcow2 Create a SHA256 for your images This is optional, but good practice when uploading your images to a webserver, etc. sha256sum *qcow2 > SHA256SUMS Optionally, sign your image with GPG. gpg --sign myfile 41
  • 48. Creating and pushing images to the container registry Create a Dockerfile FROM scratch ADD windows2016.qcow2 /disk/ Create a docker container podman build -t cmihai/windows2016 . Login to the container registry REGISTRY="$(oc get route/default-route -n openshift-image-registry -o=jsonpath='{.spec.host}')" podman login ${REGISTRY} Tag and push the image to your desired namespace (ex: virtual-machines) podman tag localhost/virtual-machine/fedora31 ${REGISTRY}/virtual-machines/windows2016 podman push ${REGISTRY}/virtual-machines/windows2016 42
  • 49. Creating container images with Buildah Create a Dockerfile in /tmp/vmdisk cat << END > Dockerfile FROM kubevirt/container-disk-v1alpha ADD fedora32.qcow2 /disk END Build and push to registry buildah bud -t vmidisk/fedora32:latest /tmp/vmdisk buildah push --tls-verify=false vmidisk/fedora32:latest docker://cdi-docker-registry-host.cdi/fedora28:latest 43
  • 50. Import the registry image into a Data volume YAML apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume metadata: name: fedora31image spec: source: registry: url: "docker://image-registry.openshift-image-registry.svc:5000/ virtual-machines/fedora31" pvc: accessModes: - ReadWriteMany resources: requests: storage: 20Gi Get the image info oc apply -f datavolume.yaml oc get pvc, dvs, pods # look for importer-fedora31image 44
  • 51. Uploading local disk images by using the virtctl tool Creating an upload DataVolume YAML apiVersion: cdi.kubevirt.io/v1alpha1 kind: DataVolume metadata: name: <upload-datavolume> spec: source: upload: {} pvc: accessModes: - ReadWriteOnce resources: requests: storage: <2Gi> Create the volume oc create -f <upload-datavolume>.yaml Upload the image virtctl image-upload dv <volume_name> --size=2G --image-path=</path/to/image> Verify that a DataVolume was created View all DataVolume objects oc get dvs 45
  • 52. Automating image builds with Kickstart and Ansible
  • 53. Building a Red Hat OS Image with Kickstart Create a Kickstart file (response file) You can manually install the OS to generate a .ks file. Installations from kickstart are automated, and you can use this as part of a CI/CD OS build. Kickstart install the OS # virt-install --name guest1-rhel7 --memory 2048 --vcpus 2 --disk size=8 --location http://example.com/path/to/os --os-variant rhel7 --initrd-inject /path/to/ks.cfg --extra-args="ks=file:/ks.cfg console=tty0 console=ttyS0,115200n8" Setup QEMU guest agent on virtual machines systemctl enable qemu-guest-agent 46
  • 54. Kickstart File RHEL 8 KickStart ignoredisk --only-use=sda clearpart --none --initlabel text repo --name="AppStream" --baseurl=file:///run/install/repo/AppStream cdrom keyboard --vckeymap=us --xlayouts='us' lang en_US.UTF-8 ... part /boot --fstype="xfs" --ondisk=sda --size=512 part / --fstype="xfs" --ondisk=sda --size=15360 part /home --fstype="xfs" --ondisk=sda --size=10240 %post sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers /bin/echo 'UseDNS no' >> /etc/ssh/sshd_config 47
  • 55. IBM Cloud: VPC Gen 2 Custom Images
  • 56. VPC Gen2 Image Format Custom Image Support for VPC Gen 2 You can create your own custom image, and import it to IBM Cloud™ Virtual Private Cloud infrastructure from IBM Cloud Object Storage. Then, you can use your custom image to create new virtual server instances that run on the KVM hypervisor. This is the same type of image used for OpenShift Virtualization, OpenStack, KVM, RHV, etc. Requirements • Contains a single file or volume • Is in qcow2 format • Is cloud-init enabled • The operating system is supported as a stock image operating system (including RHEL 7/8, Windows 2012 R2 and 2016) • Size doesn’t exceed 100 GB 48
  • 59. Packer: build multiple images from a single source 50
  • 60. Packer: Variables Variables to parametrized builds and hide secrets { "variables": { "my_secret": "{{env `MY_SECRET`}}", "not_a_secret": "plaintext", "foo": "bar" }, "sensitive-variables": ["my_secret", "foo"], } 51
  • 61. Packer: Builders KVM Builder (qemu) "builders": [{ "type": "qemu", "accelerator": "kvm", "format": "qcow2", "disk_interface": "virtio-scsi", "boot_command": [ "<up><wait><tab>", " text inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ {user `vm_name`}}.cfg net.ifnames=0", "<enter><wait>" ], "vm_name": "{{split build_type "-" 0}}-{{user `vm_name`}}", "iso_urls": [ "{{user `iso_local_url`}}", "{{user `iso_download_url`}}" ], "iso_checksum": "{{user `iso_sha256`}}", "output_directory": "{{user `builds_dir`}}/{{split build_type "-" 0}} -{{user `vm_name`}}", }] 52
  • 62. Provisioners: run post-install tasks Chaining multiple provisioners "provisioners": [ { "type": "shell", "script": "setup.sh" }, { "type": "ansible", "playbook_file": "{{user `playbook_file`}}" }], 53
  • 63. Post-processors: compress or upload your image Compress, post-process and upload the results { "post-processors": [ { "type": "compress", "format": "tar.gz" }, { "type": "upload", "endpoint": "http://example.com" } ] } 54
  • 64. Building a VirtualBox image for RHEL 8 using Kickstart 55
  • 65. Layered Image Compliance with OpenSCAP
  • 67. Automatic Remediation as shell, ansible or puppet 57
  • 68. Continuous Inspection and Automated Compliance Install OpenSCAP dnf install openscap-scanner Generate a report sudo oscap xccdf eval --report report.html --profile xccdf_org.ssgproject.content_profile_pci-dss /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml 58
  • 70. Get info oc get vms 59
  • 71. Get virt-launcher details Debug virt-launcher and look for the mounted storage ls run/kubevirt/container-disks mount | grep disk 60