Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Storage Workshop
OpenNebulaConf 2016
4th
edition
Agenda for the Workshop
● OpenNebula Storage Model
○ QCOW2 Drivers
● Lightning Talks
● Creating VM Images for OpenNebula
●...
OpenNebula Storage Model
Images DatastoreSystem Datastore
VM 1
VM 2
CentOS Image
Devuan Image
Ubuntu Image
Database Image ...
OpenNebula Storage Model
A Simplified View (!):
● Images reside in Images Datastore and are copied to
System Datastore
● V...
Image Datastore Drivers
● The drivers reside in
/var/lib/one/remotes/datastore
● These drivers manage monitoring, creation...
Image Datastore Drivers: fs
Add Image
● stat: get image size
● cp: copy image into datastore and retrieve ID
Transfer Drivers
● They manage system datastore
● Clone Images to System DS
● Copy images back to Images DS
● Creates cont...
Transfer Drivers: VM Lifecycle
● VMs have Prolog and Epilog stages
● Prolog and Epilog scripts found in
/var/lib/one/vms/<...
Transfer Drivers: VM Lifecycle
Some VM actions have trigger a TM action. Check
/usr/lib/one/mads/one_vmm_exec.rb. Attach d...
Transfer Manager: QCOW2
● QCOW2 format can create overlays (linked clones)
● This is used in QCOW2 driver
● Makes cloning ...
Transfer Manager: QCOW2 Clone
● A new image is created in system datastore:
qemu-img create -b <original> -f qcow2 <disk>
...
Transfer Manager: QCOW2 Snapshot
● Instead of cloning the image directly in disk.0 a directory
is created with the snapsho...
Lightning Talks
OpenNebulaConf 2016
4th
edition
Creating Images for OpenNebula
OpenNebulaConf 2016
4th
edition
Creating Images for OpenNebula
VM images need to be prepared to support OpenNebula
features:
● context packages: automatic...
qcow2 Format
$ qemu-img create -f qcow2 image.qcow2 10G
Metadata
Cluster Map
Cluster Data
qcow2 Image With Parent
$ qemu-img create -f qcow2 -o
backing_file=base.qcow2 image.qcow2
Metadata
Cluster Map
Cluster Dat...
● There are other ways to create your images:
○ virt-install
○ packer.io
○ foreman
○ etc...
● Sometimes modifying already ...
Mount Image
● Convert to raw and use mount -o loop
○ mount -o loop,offset=32256 image.raw /mnt
● Convert to raw and use lo...
libguestfs
From its webpage http://libguestfs.org:
libguestfs is a set of tools for accessing and modifying
virtual machin...
guestfish - Read or Edit Files
$ guestfish -ia image.qcow2
><fs> cat /var/log/service/error.log
><fs> vi /etc/service.conf...
virt-customize
● Starts custom VM and attach disks and connects to
network
● Change passwords, create users
● Move files
●...
OpenNebula Marketplace Images
● Download CentOS images
● Create CDROM with OpenNebula context packages
● Create script to ...
OpenNebula Images - Create CDROM
# Download context packages from github
curl -s
https://api.github.com/repos/OpenNebula/a...
OpenNebula Images - Prepare Script
mkdir /tmp/mount
mount LABEL=EXTRA /tmp/mount
# Install opennebula context package
rpm ...
OpenNebula Images - Calling virt-customize
# Create an overlay to preserve original image
$ qemu-img create -f qcow2 -b $o...
Optimizing Images
● qemu-img does not know anything about filesystems
● Blocks not allocated (sparse files) or that contai...
Optimizing Images - virt-sparsify
There are two ways of doing sparsification:
● Normal Sparsification:
○ Occupies the maxi...
Optimizing Images - Normal Sparsification
● Create overlay of the image
● Create a file in all filesystems and fill it wit...
Optimizing Images - In Place Sparsification
● Uses trim command, normally used for SSD disks
● Deallocates blocks from fil...
Optimizing Images - Compression
● qcow2 images can have the blocks compressed
● Compression rate is less that xz or bzip2
...
qemu-img tips
● There are two qcow2 file formats, pre version 0.10
and newer
○ CentOS 6 does not support post 0.10 version...
Cangallo!
OpenNebulaConf 2016
4th
edition
Cangallo (pronounced canga-io) is a command-line tool
written in ruby, that uses qemu-img and libguestfs to manage,
create...
Cangallo: Repository
In Cangallo images are registered into a repository:
$ canga list
NAME SIZE DAYS_AGO
default:centos7 ...
Cangallo: Cangafiles
Cangafiles are small yaml files that contain instructions on
how to create a derived image.
$ cat can...
Cangallo: Build Image
$ canga build cangafiles/centos7.canga
[ 0.0] Examining the guest ...
[ 7.0] Setting a random seed
[...
Cangallo: Use Images
● For testing in your laptop you can use overlays (empty
images with the repository as parent):
$ can...
Cangallo: Remote Repositories
● Repositories consist of qcow2 images, an index file
(yaml) and a signature
● The repositor...
OpenNebulaConf 2016
4th
edition
Platinum
Gold
Silver
Community
THANKS!
Prochain SlideShare
Chargement dans…5
×

OpenNebulaConf 2016 - Storage Hands-on Workshop by Javier Fontán, OpenNebula

2 862 vues

Publié le

In this 90-minute hands-on workshop, some of the key contributors to OpenNebula will walk attendees through the configuration and integration aspects of the storage subsystem in OpenNebula. The session will also include lightning talks by community members describing aspects related to Storage with OpenNebula:
Deployment scenarios
Integration
Tuning & debugging
Best practices

Publié dans : Technologie
  • Soyez le premier à commenter

OpenNebulaConf 2016 - Storage Hands-on Workshop by Javier Fontán, OpenNebula

  1. 1. Storage Workshop OpenNebulaConf 2016 4th edition
  2. 2. Agenda for the Workshop ● OpenNebula Storage Model ○ QCOW2 Drivers ● Lightning Talks ● Creating VM Images for OpenNebula ● Cangallo!
  3. 3. OpenNebula Storage Model Images DatastoreSystem Datastore VM 1 VM 2 CentOS Image Devuan Image Ubuntu Image Database Image Pers Virtual Router Image Empty FS Image VM-1 disk.0 VM-1 disk.1 VM-2 disk.0 VM-2 disk.1 Link
  4. 4. OpenNebula Storage Model A Simplified View (!): ● Images reside in Images Datastore and are copied to System Datastore ● VMs use images in System Datastore as disks ● Persistent Images are not copied but a link is created instead. The image in Images Datastore is modified ● Volatile disks can be created in the System Datastore ● Disk files have the names disk.0, disk.1, etc...
  5. 5. Image Datastore Drivers ● The drivers reside in /var/lib/one/remotes/datastore ● These drivers manage monitoring, creation, deletion, cloning... actions of images ○ clone: create an copy of an image ○ cp: add an image to the datastore ○ export: export to a marketplace ○ mkfs: create an empty image ○ monitor: get info about the datastore ○ rm: delete an image ○ snap_delete/flatten/revert: manage persistent snapshots ○ stat: get info about an image that is going to be imported
  6. 6. Image Datastore Drivers: fs Add Image ● stat: get image size ● cp: copy image into datastore and retrieve ID
  7. 7. Transfer Drivers ● They manage system datastore ● Clone Images to System DS ● Copy images back to Images DS ● Creates contextualization CDROM ● Reside in /var/lib/one/remotes/tm: ○ clone: copy image from Images DS to System DS ○ context: generate context CD ○ cpds: copy disk to Images DS ○ ln: link a persistent Image to System DS
  8. 8. Transfer Drivers: VM Lifecycle ● VMs have Prolog and Epilog stages ● Prolog and Epilog scripts found in /var/lib/one/vms/<id> # cat transfer.0.prolog CLONE qcow2 ursa:/var/lib/one//datastores/1/9c1f5c33cd858e3ae2ab07fa3b41c914 ursa:/var/lib/one//datastores/0/999/disk.0 999 1 CONTEXT qcow2 /var/lib/one/vms/999/context.sh /var/lib/one/vms/999/token.txt ursa:/var/lib/one//datastores/0/999/disk.1 999 0 # cat transfer.0.epilog DELETE qcow2 ursa:/var/lib/one//datastores/0/999/disk.0 999 1 DELETE qcow2 ursa:/var/lib/one//datastores/0/999 999 0
  9. 9. Transfer Drivers: VM Lifecycle Some VM actions have trigger a TM action. Check /usr/lib/one/mads/one_vmm_exec.rb. Attach disk: steps = [ # Perform a PROLOG on the disk { :driver => :tm, :action => :tm_attach, :parameters => tm_command.split }, # Run the attach vmm script { :driver => :vmm, :action => :attach_disk, :parameters => [ :deploy_id, :disk_target_path, target, target_index, drv_message ], :fail_actions => [ { :driver => :tm, :action => :tm_detach, :parameters => tm_rollback.split } ] } ]
  10. 10. Transfer Manager: QCOW2 ● QCOW2 format can create overlays (linked clones) ● This is used in QCOW2 driver ● Makes cloning faster and occupies less disk space ● It’s slower afterwards as it adds another indirection ● Enables live disk snapshotting ● Assumes a shared filesystem for both Images and System datastores
  11. 11. Transfer Manager: QCOW2 Clone ● A new image is created in system datastore: qemu-img create -b <original> -f qcow2 <disk> ● A new image is created in <disk> that contains only metadata. It’s base image is <original> ● When reading it checks if the block is in <disk> image, if it’s not there searches in <original>. This can be chained. ● When writing it checks if the block is already in <disk>, if not a new block is allocated, copies the block from <original> and writes there
  12. 12. Transfer Manager: QCOW2 Snapshot ● Instead of cloning the image directly in disk.0 a directory is created with the snapshots and a link is made to point to the actual snapshot: disk.0 -> /var/lib/one/datastores/0/6363/disk.0.snap/0 disk.0.snap/ disk.0.snap/0 (base image in Images Datastore) ● New snapshots with an external snapshot to snaps dir and the link updated: disk.0 -> /var/lib/one/datastores/0/6363/disk.0.snap/1 disk.0.snap/ disk.0.snap/0 (base image in Images Datastore) disk.0.snap/1 (base image snapshot 0)
  13. 13. Lightning Talks OpenNebulaConf 2016 4th edition
  14. 14. Creating Images for OpenNebula OpenNebulaConf 2016 4th edition
  15. 15. Creating Images for OpenNebula VM images need to be prepared to support OpenNebula features: ● context packages: automatic configuration ● ruby package: onegate dependency ● growpart: enable root partition resizing ● disable serial console: OpenNebula does not configure it and can cause problems ● open-vm-tools package: for VMware machines ● qemu guestfs package: enable disk freeze for live snapshots
  16. 16. qcow2 Format $ qemu-img create -f qcow2 image.qcow2 10G Metadata Cluster Map Cluster Data
  17. 17. qcow2 Image With Parent $ qemu-img create -f qcow2 -o backing_file=base.qcow2 image.qcow2 Metadata Cluster Map Cluster Data Metadata Cluster Map Cluster Data Parent
  18. 18. ● There are other ways to create your images: ○ virt-install ○ packer.io ○ foreman ○ etc... ● Sometimes modifying already created images is convenient ● Even if you use other image formats you can convert them
  19. 19. Mount Image ● Convert to raw and use mount -o loop ○ mount -o loop,offset=32256 image.raw /mnt ● Convert to raw and use losetup ○ losetup /dev/loop0 image.raw ○ kpartx -a /dev/loop0 ○ mount /dev/loop0p1 /mnt ● Use nbd ○ modprobe nbd ○ qemu-nbd -c /dev/nbd0 image.qcow2 ○ mount /dev/nbd0p1 /mnt
  20. 20. libguestfs From its webpage http://libguestfs.org: libguestfs is a set of tools for accessing and modifying virtual machine (VM) disk images. You can use this for viewing and editing files inside guests, scripting changes to VMs, monitoring disk used/free statistics, creating guests, P2V, V2V, performing backups, cloning VMs, building VMs, formatting disks, resizing disks, and much more.
  21. 21. guestfish - Read or Edit Files $ guestfish -ia image.qcow2 ><fs> cat /var/log/service/error.log ><fs> vi /etc/service.conf $ guestfish -ia image.qcow2 <<EOF upload service.conf /etc/service.conf tar-in ssh-configuration.tar /etc/sshd EOF $ guestfish --ro -i -c qemu:///system -d vm-name ><fs> cat /var/log/service/error.log
  22. 22. virt-customize ● Starts custom VM and attach disks and connects to network ● Change passwords, create users ● Move files ● Install packages ● Execute scripts virt-customize [--options] [ -d domname | -a disk.img [-a disk.img ...] ] [--chmod PERMISSIONS:FILE] [--commands-from-file FILENAME] [--copy SOURCE:DEST] [--copy-in LOCALPATH:REMOTEDIR] [--delete PATH] [--edit FILE:EXPR] [--firstboot SCRIPT] [--firstboot-command 'CMD+ARGS'] [--firstboot-install PKG,PKG..] [--hostname HOSTNAME] [--install PKG,PKG..] [--link TARGET:LINK[:LINK..]] [--mkdir DIR] [--move SOURCE:DEST] [--password USER:SELECTOR] [--root-password SELECTOR] [--run SCRIPT] [--run-command 'CMD+ARGS'] [--scrub FILE] [--sm-attach SELECTOR] [--sm-register] [--sm-remove] [--sm-unregister] [--ssh-inject USER[:SELECTOR]] [--truncate FILE] [--truncate-recursive PATH] [--timezone TIMEZONE] [--touch FILE] [--update] [--upload FILE:DEST] [--write FILE:CONTENT] [--no-logfile] [--password-crypto md5|sha256|sha512] [--selinux-relabel] [--sm-credentials SELECTOR]
  23. 23. OpenNebula Marketplace Images ● Download CentOS images ● Create CDROM with OpenNebula context packages ● Create script to modify the image ○ Mount CDROM ○ Install context packages ○ Remove cloud-init and NetworkManager packages ○ Install EPEL repository ○ Install growpart packages
  24. 24. OpenNebula Images - Create CDROM # Download context packages from github curl -s https://api.github.com/repos/OpenNebula/addon- context-linux/releases | jq -r '.[0].assets[].browser_download_url' | xargs -L1 wget -P repo # Create ISO image with label “EXTRA” genisoimage -o extra-packages.iso -R -J -V EXTRA repo/
  25. 25. OpenNebula Images - Prepare Script mkdir /tmp/mount mount LABEL=EXTRA /tmp/mount # Install opennebula context package rpm -Uvh /tmp/mount/one-context*rpm # Remove cloud-init and NetworkManager yum remove -y NetworkManager cloud-init # Install growpart and upgrade util-linux yum install -y epel-release --nogpgcheck yum install -y cloud-utils-growpart --nogpgcheck yum upgrade -y util-linux --nogpgcheck
  26. 26. OpenNebula Images - Calling virt-customize # Create an overlay to preserve original image $ qemu-img create -f qcow2 -b $orig $image # Run customization $ virt-customize --attach $ISO_IMAGE --run $script --format qcow2 -v -a $image --root-password disabled
  27. 27. Optimizing Images ● qemu-img does not know anything about filesystems ● Blocks not allocated (sparse files) or that contain zeroes are not copied ● Normal file deletion does not zero or deallocate blocks ● Swap partitions contain information if used ● This information can be stripped to make the images smaller ● virt-sparsify to the rescue!
  28. 28. Optimizing Images - virt-sparsify There are two ways of doing sparsification: ● Normal Sparsification: ○ Occupies the maximum space of the image ● In Place Sparsification: ○ Create an sparse qcow2 file
  29. 29. Optimizing Images - Normal Sparsification ● Create overlay of the image ● Create a file in all filesystems and fill it with zeroes until there is not more space and delete file ● Fill swap partitions with zeroes ● Convert it to a new qcow2 file skipping zero blocks $ TMPDIR=/var/tmp virt-sparsify original.qcow2 new.qcow2
  30. 30. Optimizing Images - In Place Sparsification ● Uses trim command, normally used for SSD disks ● Deallocates blocks from filesystem ● Does not require the total amount of disk space ● The qcow2 file contains holes and is not the best one for distribution ● Can be converted to a non sparse qcow2 file ● Can not be used with compression $ virt-sparsify --in-place original.qcow2 new.qcow2
  31. 31. Optimizing Images - Compression ● qcow2 images can have the blocks compressed ● Compression rate is less that xz or bzip2 ● Is more convenient as it can be directly used as is ● Use of these images trades disk space for CPU consumption ● Can be done directly in virt-sparsify with --compress (not In Place)
  32. 32. qemu-img tips ● There are two qcow2 file formats, pre version 0.10 and newer ○ CentOS 6 does not support post 0.10 version ○ On conversion or creation it can be specified with -o compat=0.10 ● qemu-img < 2.4 does not support creation of delta images with compression ○ This tool can be easily compiled manually ○ Download qemu 2.4 code ○ ./configure ○ make qemu-img
  33. 33. Cangallo! OpenNebulaConf 2016 4th edition
  34. 34. Cangallo (pronounced canga-io) is a command-line tool written in ruby, that uses qemu-img and libguestfs to manage, create and organize qcow2 images. Its repository holds images and deltas of derived images in a similar way as Docker but in a block level instead of file level. https://canga.io Cangallo!
  35. 35. Cangallo: Repository In Cangallo images are registered into a repository: $ canga list NAME SIZE DAYS_AGO default:centos7 370.2 Mb 134.2 default:debian8 455.8 Mb 134.2 default:ubuntu1404 247.6 Mb 134.2 default:ubuntu1604 290.1 Mb 134.2 ^ default:centos7/one 47.7 Mb 134.1 OpenNebula Compatible CentOS 7 ^ default:0b5e9704c8e898ee 51.0 Mb 134.1 OpenNebula Compatible Ubuntu 14.04 ^ default:0eb98b0f62e528e6 78.0 Mb 134.1 OpenNebula Compatible Ubuntu 16.04 default:centos7-1606 354.2 Mb 101.4 ^ default:centos7-1606/one 50.1 Mb 92.3 OpenNebula Compatible CentOS 7 ^ default:centos7-1606/one/vmware 69.7 Mb 59.3 OpenNebula Compatible CentOS 7 ^ default:debian7/one 69.6 Mb 4.0 OpenNebula Compatible Debian 8 ^ default:ubuntu1404/one 51.8 Mb 4.0 OpenNebula Compatible Ubuntu 14.04 ^ default:ubuntu1604/one 80.1 Mb 4.0 OpenNebula Compatible Ubuntu 16.04 default:centos7-1608 137.9 Mb 4.0 ^ default:centos7-1608/one 57.1 Mb 4.0 OpenNebula Compatible CentOS 7
  36. 36. Cangallo: Cangafiles Cangafiles are small yaml files that contain instructions on how to create a derived image. $ cat cangafiles/centos7.canga # vim:tabstop=2:sw=2:et:syntax=yaml: --- description: OpenNebula Compatible CentOS 7 os: CentOS 7 parent: centos7-1608 tag: centos7-1608/one tasks: - copy: context / - run: - rpm -Uvh /context/one-context*rpm - yum remove -y NetworkManager cloud-init - yum install -y epel-release cloud-utils-growpart ruby --nogpgcheck - yum install -y util-linux --nogpgcheck - delete: /context
  37. 37. Cangallo: Build Image $ canga build cangafiles/centos7.canga [ 0.0] Examining the guest ... [ 7.0] Setting a random seed [ 7.0] Copying: context to / [ 7.0] Running: rpm -Uvh /context/one-context*rpm [ 7.6] Running: yum remove -y NetworkManager cloud-init [ 11.1] Running: yum install -y epel-release cloud-utils-growpart ruby --nogpgcheck [ 25.8] Running: yum install -y util-linux --nogpgcheck [ 30.8] Running: rm -rf /context [ 31.4] Finishing off [ 3.1] Trimming /dev/sda1 [ 3.5] Sparsify in-place operation completed with no errors Calculating image sha256 with libguestfs (it will take some time) Image SHA256: 7da5deee65885250577807e50f18d727f6a1147957aac5b5783bc724df9bde4c Copying file to repository Deleting temporary image
  38. 38. Cangallo: Use Images ● For testing in your laptop you can use overlays (empty images with the repository as parent): $ canga overlay centos7/one centos7.qcow2 ● Export to an independent image $ canga export centos7/one centos7.qcow2 -c $ canga export centos7/one centos7.vmdk -f vmdk
  39. 39. Cangallo: Remote Repositories ● Repositories consist of qcow2 images, an index file (yaml) and a signature ● The repositories can be exposed to the world using an https server ● Cangallo is able to download the index and images from the repository $ cat ~/.cangallo/config.yaml default_repo: default repos: default: type: local path: /home/jfontan/projects/cangallo/repo remote: type: remote path: ~/.cangallo/remote url: http://localhost:8000
  40. 40. OpenNebulaConf 2016 4th edition Platinum Gold Silver Community THANKS!

×