These slides were presented in the containerization meet-up organized by digital ocean meet-up group in Bangalore. The slides talk about using containers for storage to make the storage truly non-disruptive during upgrades. This is a quick introduction to OpenEBS as well.
2. Persistent storage is a need for stateful applications in containers
Storage should be always available (yes, always)
Scalable, flexible, software defined
Hybrid cloud is a reality (Multiple clouds and a Data center in your IT). Block storage but
backup to S3
Orchestration of storage volumes at large scale (10,000 + ) (provisioning, scheduling among
tiers, hosts, migration, backup, restore, upgrade)
Storage requirements in Container world
5. Why containerization of storage?
Applications
Persistent storage
volumes
Upgrade each
app (relaunch
container)..
One by one
Storage upgrade ?
Volumes are part of
monolithic storage
6. Why containerization of storage?
Applications
Persistent storage
volumes
Upgrade each
app (relaunch
container)..
One by one
Storage volumes are
containerized
● Storage functionality in
userspace
7. Containerized storage
Monolithic Storage Kernel
iSCSI
Software
A
Replication
Snapshot
Encryption
B
QoS
&
Mgmt
C
volume
1
volume
2
Containerized Storage
Kernel
(OpenEBS)
volume
1
Storage Container
VSM 1
Software
A, B, C
volume
2
Storage Container
VSM 2
Software
A’, B’, C
Storage software is
same for all
volumes
8. OpenEBS Building blocks and integration
For provisioning Exposes EBS API
Building blocks
Integration
9. OpenEBS - Containerized storage
Truly non disruptive storage upgrades, at volume level
Easy provisioning and connectivity via k8s (storage volume is
part of k8s pod yaml config file)
Easy connectivity via k8s (OpenEBS exposes AWS EBS API,
connect from k8s via “awsElasticBlockStore” API )
Tier-to-anything caching. Can serve the block storage from
remote storage with high performance
10. OpenEBS - Status
Started in late 2016
Alpha state, github already has the working elements (Maya-
Storage scheduler, storage containerization using docker)
Currently hacking EBS API, Snap/restore to S3, flannel type
networking integration
Launched “Bangalore OpenEBS Meetup” group
First meetup is on “25th January, 6 PM”
Hiring Go warriors, k8s enthusiasts, network and file system
experts, linux hackers. Contribute to design and code.
11. Remote Storage
Local Storage
OpenEBS Storage Hosts
Data
Storage
Driver
HTTPS
(manage)
Overview & Terminology
K8s master
K8s minions
OpenEBS
Maya
master
OpenEBS VSMs / Storage Pod
Pod
Network (Flannel*)
Network (Flannel)
12. VSM - Storage in Containers
Local
Disks
NAS or SAN Cloud
Storage
NVMe Flash
Maya Storage Orchestrator
Backend Containers
to Persist Data
(Cached, Protected)
Frontend ContainersVSM / Storage
Pod
OpenEBS
Storage Hosts
Storage
Data (iSCSI/TCMU)
Container (Docker)
Inline Replication
Multiple Storage Backends
This presentation is done at the containerization meet-up organized in Bangalore by the Digital ocean meet-up group. The presentation talks about how containers are used to provide truly non-disruptive storage upgrade capability at volume level
Compute layer is well on track to adopt the micro services architecture. Containerization can help the other layers such as networking and storage as well. Weave works talk about containerized networking. Storage is definitely moving towards that as storage is always a bulky layer and at scale have lots of dependencies on application maintenance windows during upgrade planning.
This slide shows the difference between applications that are containerized and storage volumes that are on top of monolithic storage.
For any upgrade planning of applications, the maintenance windows can be planned easily. But same is not the case with storage software updates. During upgrade of monolithic storage appliance, as shown in this slide, all storage volumes are affected.
In the containerized storage, the software updates can be done at the storage volumes level. Just like each application in compute layer is containerized, each storage volume is containerized. Now, the software maintenance for the storage layer is very simple and similar to how it is done at the compute layer
For storage to be containerized, the storage functionality need to be containerized, which means it needs to be moved from kernel to user space. In the containerized storage model, replication, snapshotting, encryption and storage protocols (iSCSI/NBD) are moved to user land. Each storage volume is represented by one or more containers.
OpenEBS builds containerization and orchestration using the known stable infrastructure components. For containerization, Docker is used. For block storage functionality (replication, encryption, easy connectivity to containers), Rancher longhorn is used. For the Maya orchestration platform Nomad (for clustering), consul (for cluster database) and flannel logic sans flannel db (for networking intelligence) are used. The provisioning integration is done into k8s, this means that the containerized storage volumes are provisioned right from the kubernetes along with the application containers. As amazon EBS is one of the most common storage infrastructures that the applications and orchestration systems are used to, OpenEBS exposes the AWS EBS APIs.
This slide summarizes the OpenEBS features at a high level.
This slide shows the dedication storage deployment model. Another model is hyper-converged model where the openebs components are deployed on the k8s minions.
Each storage pod or a VSM consists of two or more containers. The backend store for OpenEBS can be the regular local disks, remote SAN disks or the remote cloud storage disks such as AWS EBS or Google persistence disks or even S3 disks. In case the backend storage is plain local disks, OpenEBS provides a managed services for the backend (setting up a storage pool on them using RAID, hot spare replacement etc)
Join the growing community. Do follow us on twitter and our blog.
