The OpenVZ/Virtuozzo developers from Odin (ex Parallels) have been working on Linux container technologies since 1999. What was originally a separate patchset is now mostly merged into the upstream Linux kernel, enabling the way for projects like LXC and Docker. In the mean time, the OpenVZ/Virtuozzo Linux kernel is still one step ahead of the vanilla kernel when it comes to containers. The talk will provide details about recent efforts towards Docker and Virtuozzo interoperability. This development is twofold. The first goal is to run Docker inside an OpenVZ container and the second goal is to use the proven OpenVZ kernel as a backend for Docker (via libcontainer).

1. OpenVZ,
Virtuozzo,
and Docker
Kirill Kolyshkin <kir@openvz.org>
LinuxFest Northwest, 25 Apr 2015

2. openvz.org | criu.org | virtuozzo.com
Agenda
● The history of containers
● Virtuozzo relaunch
– and an important announce
● Docker: in, out, and on the side

3. openvz.org | criu.org | virtuozzo.com
1999: the idea
● Our chief scientist, an MIPT professor proposed
an idea of virtual environments as groups of
processes
● Three main components were:
– groups of processes with namespace isolation
– file system to share code and save RAM
– resource isolation / management

4. openvz.org | criu.org | virtuozzo.com
2000: the first steps
● Feb: 3 engineers started working on it in MIPT
– plus a sysadmin and a manager
– and a guy to write web management
● Jul: public beta testing:
– one server for VEs, one for web mgmt
– around 5000 VEs (virtual environments)
● Sep: hired a mathematician from MSU to work
on resource management (User Beancounters)

5. openvz.org | criu.org | virtuozzo.com
2001
● Virtuozzo for Windows (VZWIN) started
– really crazy idea!
– no source code – lot of reverse engineering
– implemented by live kernel patching
– in Windows everything is an object, so those
objects are virtualized / namespaced
– “most advanced software ever written for Windows”
● Linux-VServer project appeared

6. openvz.org | criu.org | virtuozzo.com
2002: release
● Jan: first Virtuozzo for Linux release: 2.0
● Feb: first commercial customers

7. openvz.org | criu.org | virtuozzo.com
2003
● Developing, developing, developing...

8. openvz.org | criu.org | virtuozzo.com
2004
● Feb: Solaris Zones/Containers released
– btw kudos to Sun for the term “containers”
● Dec: first Virtuozzo for Windows release

9. openvz.org | criu.org | virtuozzo.com
2005
● OpenVZ project was born
(better late than never)

10. openvz.org | criu.org | virtuozzo.com
2006
● Jan: port to kernel 2.6.15
● Apr: port to Fedora Core 5 kernel
● Aug: Debian includes OpenVZ kernel/tools
● Aug: rebase to RHEL 4 kernel
● Oct: port to SPARC[8] and PPC
● Nov: port to 2.6.18 kernel
● Nov: live migration

11. openvz.org | criu.org | virtuozzo.com
2007
● IBM AIX WPARs, HP-UX SRP containers
● Mar: Rebase to RHEL5 kernel
● Mar: Port to 2.6.20 kernel
● May: Knoppix-based OpenVZ Live CD
● Sep: CentOS-based OpenVZ Live CD

12. openvz.org | criu.org | virtuozzo.com
2008
● Lots of patches to upstream kernel:
– network namespace
– PID namespace
– IPC namespace
– et cetera
● LXC (userspace tool a la vzctl)
● Apr: port to kernel 2.6.25
● Oct: Port to ARM

13. openvz.org | criu.org | virtuozzo.com
2009
● Parallels appears in Top 10 Linux contributors

14. openvz.org | criu.org | virtuozzo.com
2010
● Port to RHEL6
● VSwap (RAM/swap limits, simplified UBC)
● ploop aka CT filesystem in a file
– on-demand allocation
– instant snapshots
– online resize, merge, compact
– write tracker (improved live migration)

15. openvz.org | criu.org | virtuozzo.com
2011
● Jul: initial proposal for CRIU
● The idea is to implement
most of CR in userspace
● Checkpoint: takes most of
task info from /proc etc.
● Restore: read dump,
recreate environment

16. openvz.org | criu.org | virtuozzo.com
2012
● Jul: initial CRIU release (v0.1)
● Oct: vzctl 4.0 with vanilla kernel support
● ???: started working on RHEL7-based kernel

17. openvz.org | criu.org | virtuozzo.com
2013
● Docker appeared
● lmctfy appeared
● CoreOS appeared
● vzctl adds io/iops limit support
● May: OpenVZ maintenance partnership
https://openvz.org/Support

18. openvz.org | criu.org | virtuozzo.com
2014
● vzctl 4.8, faster live mirgation
● Dec: Parallels announces merging OpenVZ
and Parallels Cloud Server into single common
open source codebase
blog.openvz.org/49158.html

19. openvz.org | criu.org | virtuozzo.com
Current OpenVZ problems
● Kernel development is not quite open
– devel@ list is deserted, no kernel git repo, ...
● OpenVZ and Virtuozzo have forked userspace
– Userspace development effort is doubled
– OpenVZ can't be easily upgraded to VZ
– Some features are missing (vzpkg anyone?)
– Unnecessary complications for QA and support

20. openvz.org | criu.org | virtuozzo.com
What is the new Virtuozzo?
● Open source development process
– for real, this time
– open discussions via public mailing list
– source code repository for the kernel
– https://openvz.org/Contribute
● Unified code base
● Bare metal distro (with installer etc.)
● KVM support (coming later)

21. openvz.org | criu.org | virtuozzo.com
The (long-awaited) announce
● Today we open src.openvz.org with
RHEL7-based VZ kernel git repo
src.openvz.org
● Site is still in beta, but registration works
● Will be integrated with JIRA bug tracking

22. openvz.org | criu.org | virtuozzo.com
RHEL7-based OpenVZ kernel
● Is beta
● Uses CRIU for migration :)
● Container API has changed, so
current vzctl doesn't work
● To contribute:
openvz.org/Contribute
openvz.org/How_to_submit_patches

23. openvz.org | criu.org | virtuozzo.com
Nearest future (sorry no ETAs)
● Publish VZ tools
● Establish all the processes
● Release beta
● …
● PROFIT!!!

24. openvz.org | criu.org | virtuozzo.com
What about Docker?
● We love Docker
● We support Docker everywhere
– Docker inside (in OpenVZ containers)
– Docker outside (using OpenVZ kernel)
– Docker live migration with CRIU

25. openvz.org | criu.org | virtuozzo.com
Docker inside VZ containers
● openvz.org/Docker_inside_CT
● Available since Feb (kernel 042stab105.4)
● Current limitations:
– no checkpointing / live migration
– only vfs graph driver
– no bridges inside Docker containers inside VZ CTs

26. openvz.org | criu.org | virtuozzo.com
Russian dolls for Docker
● allow nested net, pid, IPC, UTC namespaces
● allow bridges, disable venet inside nested netns
● allow eth devices renaming inside CT
● some sysfs and caps hacks here and there
● bogus cpusets and cgroups
● many small fixes here and there

27. openvz.org | criu.org | virtuozzo.com
Live demo goes here

28. openvz.org | criu.org | virtuozzo.com
Docker on top of OpenVZ
● Upstream kernel is not as good as OpenVZ
(resource mgmt, /proc and /sys virt., etc)
● Why not use OpenVZ kernel?
● Implementation: libct for libcontainer, Oct 2014
github.com/avagin/libct
● About to be included, pull request opened Mar:
github.com/docker/libcontainer/pull/434

29. openvz.org | criu.org | virtuozzo.com
Docker on the side: checkpointing
● CRIU 1.3 (1 Sep 2014), external CR works
– external as in Docker knows nothing about it
– docker ps, stop, kill, logs will not work correctly
● Solution: native docker checkpoint / restore
● CRIU 1.5.1, released 31 Mar 2015
github.com/SaiedKazemi/docker/wiki

30. openvz.org | criu.org | virtuozzo.com
Docker C/R use cases
● Live migration
● Restart Docker daemon w/o losing Cts
● Reboot the host w/o losing Cts
● “Forensic debugging” (CRIU image analisys)
● Docker CR demo: youtu.be/HFt9v6yqsXo

31. openvz.org | criu.org | virtuozzo.com
Contacts
● Google+: OpenVZ
● Twitter: @_openvz_
● Facebook: openvz.org
● blog.openvz.org
● openvz.org/Contacts
Kirill Kolyshkin <kir@openvz.org>

32. openvz.org | criu.org | virtuozzo.com
Future directions
● Deliver Virtuozzo 7
● Forth-generation resource management
– dynamic mgmt by a userspace daemon
– bursts, guarantees, etc.
● Port to POWER (ARM?)
● MetaPC (single system image, based on CRIU)