1. Ceph storage on MicroK8s!
Angelos Kolaitis
Senior Engineer @MicroK8s
1.29 Release Lead Shadow
MicroK8s
Friday 1st of September, 1:30 pm GMT
https://meet.google.com/hjy-uogt-tax
Philip Williams
Product Manager
Ceph
&

2. Agenda
● What is MicroCeph?
● MicroCeph advantages
● Deployment of MicroCeph
● MicroCeph on MicroK8s

3. What is MicroCeph?
Consistent, Isolated and Upgrade Friendly
● Streamlined and effortless deployment
● Minimal setup and maintenance overheads
● MicroCeph is containerised snap with all of its
dependencies and runs fully isolated from the
underlying host
● Using channels for the distribution, going from
latest/stable to latest/edge is easy

4. MicroCeph advantages
Storage for non-experts
● Easiest way to deploy Ceph, just a few commands
● Designed for non-experts
● Highly scalable
○ dev workstation, to edge, to datacenter
● Reliable and resilient
● Open-source distributed storage system
● Flexibility for object, block, and file-level storage
● Supports encryption at rest too!

5. snap install microceph
microceph cluster bootstrap
microceph cluster add node[x]
microceph cluster join pasted-output-from-node1
microceph.ceph status
microceph disk add /dev/sd[x] --wipe
microceph.ceph status
microceph.ceph osd status
MicroCeph deployment
microceph.com
QR Code?

6. MicroCeph demo

8. Ceph Storage in MicroK8s
MicroK8s +

9. - Kubernetes solves container orchestration issues at scale
- I can run my massive distributed workloads at scale (machine learning, video
transcoding, my SaaS business)
- Where can I store the end results?
- We need persistent storage!
- CSI (Container Storage Interface)
- Kubernetes does not solve the problem of distributed storage, since
performance, stability requirements might differ
- Instead, it defines a standard interface to consume any storage
- Which CSI driver? Depends!
- On a public cloud? aws-ebs-csi-driver, azurefile-csi-driver, …
- On a private cloud? cinder-csi-driver, …
- On bare-metal? BYOS!
- Ceph
Distributed Storage In Kubernetes

10. - Ceph CSI!
- Supports CephRBD (ideal
for RWO volumes)
- Supports CephFS (supports
RWX volumes)
Ceph storage in Kubernetes

11. sudo microk8s enable rook-ceph
Deploy rook-operator on the
MicroK8s cluster
Rook can:
- deploy a Ceph cluster inside
the MicroK8s cluster, using
disks from the hosts
- integrate with external
Ceph clusters and manage
them as citizens in
Kubernetes
MicroK8s Rook Ceph addon (1/2)

12. sudo microk8s enable rook-ceph
Why Rook?
- Can detect the right CSI
version depending on running
K8s and Ceph
- (Optionally) Can be used to
manage Ceph from
Kubernetes
- (Optionally) Simplifies ceph
auth clients and keyrings
- (Optionally) Can deploy a
Ceph cluster on top of the
Kubernetes nodes
MicroK8s Rook Ceph addon (2/2)

13. Install MicroCeph
$ sudo snap install microceph --channel quincy/stable
$ sudo microceph cluster bootstrap
$ sudo microceph disk add $disks --wipe --encrypt
MicroK8s / MicroCeph (1/4)

14. Install MicroK8s
$ sudo snap install microk8s --classic --channel 1.28/stable
MicroK8s / MicroCeph (2/4)

15. Enable Ceph addon
$ sudo microk8s enable rook-ceph
MicroK8s / MicroCeph (3/4)

16. Consume MicroCeph
$ sudo microk8s connect-external-ceph
MicroK8s / MicroCeph (4/4)

17. microk8s connect-external-ceph
- Connect to Ceph cluster and initialize RBD pool
- Detect appropriate Ceph CSI version to deploy
- Generate Ceph client keyrings for ceph-csi
- Create Kubernetes resources for connecting to Ceph (ceph mons, keyrings)
- Create StorageClasses for consuming Ceph storage
MicroK8s / MicroCeph

18. Deploy microceph and microk8s on an edge environment and use Ceph for
storage
$ sudo snap install microceph --channel quincy/stable
$ sudo snap install microk8s --classic --channel 1.28/stable
$ sudo microk8s enable rook-ceph
$ sudo microk8s connect-external-ceph
Scenarios (1/4)

19. More complex Ceph configuration, use an RBD pool backed by specific OSDs
(e.g. tenant separation, use only SSD disks, …)
$ ceph osd crush rule create-replicated replicated_ssd default host ssd
$ rbd pool create microk8s-ssd-pool
$ ceph osd pool set microk8s-ssd-pool crush_rule replicated_ssd
$ microk8s connect-external-ceph --rbd-pool microk8s-ssd-pool
Scenarios (2/4)

20. CephFS for RWX support
$ ceph fs volume create microk8s-cephfs0
$ microk8s connect-external-ceph
Scenarios (3/4)

21. Integrate with any Ceph cluster
$ microk8s connect-external-ceph
--ceph-conf ./ceph.conf
--ceph-keyring ./keyring.conf
Scenarios (4/4)

22. Can I Try This Out?
Sure!
1.28 cluster in 5 min
$ sudo snap install microk8s --classic --channel 1.28/stable
MicroK8s releases on 1.28/edge within 24 hours after upstream release
MicroK8s

23. Thank you
Interaction
• Websites
https://microk8s.io/
https://github.com/canonical/microk8s
• Discussion on
https://discuss.kubernetes.io/tag/microk8s
or in Slack on the #microk8s channel
• Report Issues
https://github.com/canonical/microk8s/issues
• MicroK8s Community Sync
Every Friday, 1:30pm GMT
Video call link: https://meet.google.com/hjy-uogt-tax
Upcoming sessions from MicroK8s 1.28
- 1.28 Upstream Updates
- MicroK8s CIS conformance
- MicroK8s Dual Stack
- MicroK8s Ceph integration (feat MicroCeph)
Upcoming sessions calendar
https://www.meetup.com/microk8s-meetup-group/
MicroK8s

24. Thank you. Questions?