In our March 2017 Rancher Online Meetup we dove into running persistent storage for containers in Rancher. The presentation included an overview of persistent storage in Rancher, an introduction to Quobyte and an overview of Portworx.

1. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Storage for Containers and
Cloud-Native Deployments
March 28, 2017
#ranchermeetup

2. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Shannon Williams
Co-Founder/VP Sales
@smw355
1
Will Chan
Co-Founder/VP Engineering
@willchan00
#ranchermeetup

3. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Bjorn Kolbeck
Co-Founder
Quobyte
@quobyte
2 #ranchermeetup
Gou Rao
Co-Founder and CTO
Portworx
@raogou

4. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
First things first…
This is a not a !
3 #ranchermeetup

5. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .4
There are rules for a meetup!
• We won’t be done on time
• Questions are always welcome
• Demo, then demo some more
• Things will break, be patient
#ranchermeetup

6. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .5
Join the conversation on Twitter
#ranchermeetup

7. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Agenda
• Rancher Overview – Shannon
• Storage Concepts and Data Persistence in Rancher – Will
• Quobyte Overview – Bjorn
• Portworx Overview – Gou
• Latest Rancher Release Details – Will
6 #ranchermeetup

8. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Rancher Labs
7 #ranchermeetup
An open-source software platform for
managing containers
A minimalist OS built explicitly to run
Docker

9. © 2017 Rancher Labs, Inc.
A complete container management
platform that makes it easy to…
8
INNOVATE WITH CONTAINERS
without compromising flexibility by empowering developers with fast
access to the latest tools
MANAGE APPLICATIONS
by simplifying day to day application lifecycle management
RUN CONTAINERS
with the most complete set of container and infrastructure
management capabilities
Production ready
✔ 20 million+ downloads
✔ Open platform for
innovating
✔ Easy to use interface
✔ Multi-tenant
✔ Role based access
✔ 24X7 support
✔ And more….

10. © 2017 Rancher Labs, Inc.
Complete
Container
Management
Platform
9
Application Catalog
Container Orchestration and SchedulingUser Mgmt
RBAC
AD/LDAP
SAML
Ops Mgmt
CI/CD
Registries
Monitoring
Networking
Multi-tenant Environments
Environment 1 Environment N
Infrastructure Services
Storage
……. ..Environment 2
Security DNS/LB

11. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .#ranchermeetup
Storage for
Containers
and Cloud-
Native
Deployments

12. © 2017 Rancher Labs, Inc.
Docker Volume Plugins
• Released as part of Docker 1.8 in Aug 2015.
• Since then, there has been many volume
plugins created including storage plugins to:
• cloud providers (AWS, GCE, Azure, DO)
• storage vendors (NetApp, EMC, Quobyte, Portworx)
• more frameworks (rex-ray, openstorage)
• While is this great, you still have to download it,
install it, and manage it…

13. © 2017 Rancher Labs, Inc.
Volume Plugin Manager
• Rancher is your Docker volume plugin manager
and provides the following:
• Distribution – automatically deploys volume plugins to desired hosts managed by
Rancher so they are made available immediately
• Version Management and Upgrades – provides version management and allows
users to automatically update plugins on all hosts with a single click (or API call)
• Lifecycle Management – Provides health checks and automatic redeployment of
plugins if things go wrong
• Plugin-aware Container Scheduling – provides container scheduling based on
deployment of plugins so you never schedule containers on hosts that have no
access to storage

14. © 2017 Rancher Labs, Inc.
How is this done?
• Leverage Rancher’s catalog to for version
management, upgrades, and distribution.
• Leverage Rancher for lifecycle management of
the volume plugin which needs to be packaged
as a Docker container.
• Leverage Rancher for container scheduling.

15. © 2017 Rancher Labs, Inc.
Supported Plugins

16. © 2017 Rancher Labs, Inc.
Supported Plugins
• Rancher already supports NFS with EBS and
EFS made GA in 1.6 (end of April)
• Open source contributions include plugins for
Rex-ray
• Vendor contributions include Netapp, Portworx,
and Quobyte

17. © 2017 Rancher Labs, Inc.
Docker Plugins
Storage Vendors Rancher Supported Cloud Providers

18. QUOBYTE - THE DATA CENTER
FILE SYSTEM
Fast and Reliable SoftwareStorage

19. Introduction
• Björn Kolbeck
Co-Founder atQuobyte
2006
2010-2013
2013
1
8

20. Google-Style
Infrastructure
Data Center
“Warehouse-scale”
Container Infrastructure
Linux + Containers +Borg
Google File System
Replication + ErasureCoding
Applications
1
9

21. GIFEE
… but I’m not
Google?!
2
0
• Scalability
• Agility /Flexibility
• Fault-tolerance
• Commodityhardware
• Containers (Kubernetes)
• Storage?
– Google has customstack
– Everyone else: POSIX & legacy apps

22. Quobyte
Data Center File
System
Scale-out shared POSIX filesystem
Linux Linux Linux
Applications
MySQL, LAMP, Cassandra, ElasticSearch…
2
1
Commodity Servers
no RAID, NVRAM, journalingdevice…

23. Data Center File System
Reliability in Software: Storage must be
cattle!
• Split-brain safereplication
• Erasure Coding
• End-to-end checksums
• Never trust thehardware!
• Ignore: Broken/slow drives, servers, network cards,packet loss,
network splits, brokenswitches…
2
2

24. Data Center File
SystemLinear Scalabiliy without
Bottlenecks
• 2x4x6xServers = 2x4x6xIOPS, throughput, capacity
• No bottleneck: Big data, video, webserver, hpc…
• Grow compute &storage together
2
3

25. Data Center File System
High Performance for All
Workloads
• High performance for all workloadtypes
– Throughput
– IOPS
– Metadata intensive
(small fileworkloads)
– Consistent low latency <1ms
• Share data viainterfaces
– File system (Linux, Mac,Win)
– S3
– Hadoop
stat read(4k) write(4k)
420µs
2
4
150µs100µs

26. Data Center File System
Smart Data Placement: Optimize your
Apps
➡ Tiering
➡ Isolation
➡Application-aware
data placement
• Filename
• Extension
• Application
• Age /Access
2
5
HDDSSDDatabase

27. Data Center File System
Smart Data Placement: Cassandra fast and
cheap
Commit log(*.log)
• Appendworkload
• Replication:x3
• NVMe
• Transactions <500usec
SSTables(*.db)
• Read-only
•EC: x1.375
• Read optimizedSSDs
• Lowest $/IOPS
10

28. Quobyte +
Containers
Integrated
• Persistent Volume Plugin (>=1.4)
– Global, replicas,container…
– Automatic volume provisioning
• UserId mapping (Quobyteblog)
• Volume plugin
• https://github.com/quobyte/docker-volume
27

29. Get Quobyte
Today!
• www.quobyte.com/get-it
• Soon: RancherCatalog
• hello@quobyte.com
• Meet usat
28

30. © 2017 Rancher Labs, Inc.
Data Services for Cloud
Native Applications

31. © 2017 Rancher Labs, Inc.
Portworx + Rancher: The Transformation to Cloud Native DevOps Infrastructure
X86 Commodity Servers or Cloud
Monitoring
Orchestration/Scheduling
Minimal OS
Portworx
Cloud Native Infrastructure
Fixed Infrastructure
Security
Kubernetes w/ Rancher
Containers
HP
EMC
NetApp
IBM
Cisco
VMware
Dell
Oracle
RedHat
Microsoft

32. © 2017 Rancher Labs, Inc.
The Problem: Static, Out of Band Storage
Provisioning
SSD
HDD
nginx python mysql
SAN
html5 rest sql
Apps hit a wall going to production Traditional Storage forced into a container, Out-of-Band
Applications App Unaware Storage Cloud
Connectors like RexRay
SSD
EBS

33. © 2017 Rancher Labs, Inc.
Persistence
IOPs
Connectors SSD
HDD
SAN
EBS
S3
Admins Realize that Apps Don’t Want
Volumes
1. Don’t make data protection a science project
2. Stop trying to connect legacy SAN/NAS/SDS to containers – They are not container or application
aware
3. Your applications want on-demand services delivered programmatically via the scheduler
4. Storage services need to be “app-aware”, for example Cassandra-aware – Traditional storage is not.

34. © 2017 Rancher Labs, Inc.
The Solution: Data Services built for DevOps for ANY Infrastructure
nginx python mysql
html5 rest sql
SSD
HDD
SAN
Cloud
Portworx
SSD
EBS
• App-aware
• Any infrastructure
• Programmatic
No more volumes or storage per application to manage
Rancher with Kubernetes

35. © 2017 Rancher Labs, Inc.
Programmatic Data Services for DevOps
WordPress WordPress
PX
MySQL
MySQL
persistence
encryption
CoS
namespace
Auth/ACLs
HA Operational Experience
Delivered as a lightweight
container with support for
volumes and graph. Runs on any
server, any cloud.
Converged Performance
Data collocated with container
compute
Container Granular Controls
Encryption, CoS, replication,
snapshots are done at container
granularity
Hybrid Cloud Ops
Cloud agnostic deployment and
data accessibility across multiple
clouds
S3 Object

36. © 2017 Rancher Labs, Inc.
Portworx is Topology and Application Aware
Application
…
x86 Servers x86 Servers
rack
x86 Servers x86 Servers
rack
x86 Servers x86 Servers
rack
vol volvol vol vol
Scheduler
Portworx places an application’s data ON THE NODE that it executes on
Container granular volumes on ANY infrastructure

37. © 2017 Rancher Labs, Inc.
Portworx vs ScaleIO or CEPH
x86 Servers x86 Servers
rack
…
x86 Servers x86 Servers
rack
Portworx Places an Application’ss Data Local to a Few
Topology Constrained Servers. Cassandra runs
converged.
x86 Servers x86 Servers
rack
…
x86 Servers x86 Servers
rack
x86 Servers
ScaleIO and CEPH shard an application’s data across
many servers to get capacity. This breaks convergence.
You also don’t have application granular settings on the
volumes.

38. © 2017 Rancher Labs, Inc.
Portworx is Enterprise Grade Software

39. © 2017 Rancher Labs, Inc.
Demo: Portworx in the Rancher Catalog
Demo

40. © 2017 Rancher Labs, Inc.
Why Rancher: from the Portworx Perspective
▪ Start to finish
• Launch Containers and Infrastructure services in seconds
• Rich Community + Enterprise Catalog of services
▪ Deploy anywhere
• Support for top cloud providers like Amazon, Digital Ocean, Packet …
• Broad community of users moving to cloud-native microservices

41. © 2017 Rancher Labs, Inc.
Portworx is Open Core
OpenStorage.org
Scheduler Data Layer: OpenStorage is a massively scalable data layer for
Containers and Schedulers like Kubernetes and Mesosphere
LCFS
Data Path: LCFS provides core data path functionality for Docker images.
Check it out on https://github.com/portworx/lcfs
PORX
Bundled Data Services: Try out the Portworx community edition at
https://github.com/portworx/px-dev

42. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .41
Demo
#ranchermeetup

43. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Latest Release
42
Rancher 1.5 – March 3, 2017
Key Features:
- Templating support for catalog
items
- API Interceptor
- Enhanced Network Policies
- Webhooks – Host Scaling,
DockerHub upgrade
- Metadata Improvements
- Portainer.io for Swarm UI

44. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Next Releases
43
Rancher 1.6 – Late April
Key features:
- Catalog Enhancements – Ability to add catalogs per environment
- HAProxy Enhancements -
- Certifying Rancher-EBS -
- Scheduling Enhancements - schedule containers evenly across
pools of hosts
- K8S 1.6 Support with Docker 1.13/Docker 17.03 and etcdv3

45. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Getting Started
Rancher and RancherOS are in GitHub – Get Involved!
44 #ranchermeetup
http://github.com/rancher

46. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Even better - try.rancher.com…
45

47. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Then join a free training class…
46
http://rancher.com/training

48. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .
Thank you
rancher.com
#ranchermeetup