Link: https://youtu.be/xFZTysIfsxs
https://go.dok.community/slack
https://dok.community/
From the DoK Day EU 2022 (https://youtu.be/Xi-h4XNd5tE)
Redis is a widely used open-source in-memory data store and cache that has become a key component in the development of scalable microservice systems. While all of the main cloud providers provide fully managed Redis services (Amazon ElastiCache, Azure Cache for Redis, and GCP Memorystore), it may also be simply implemented in Kubernetes if you require additional control over the Redis configurations.
This session will lead you through setting up a Redis cluster on Kubernetes with data persistence. The lesson would concentrate on establishing a "prod"-ready Redis cluster that considers storage and persistence even after restarts and discusses recommended practices and considerations.
With the help of monitoring and hardware optimizations for your cluster, you'll learn about measuring your performance and best practices for Redis on Kubernetes. We would conclude whether operators or Helm chart will be a more mature option for your Data on Kubernetes needs with Redis and a few tips as a production checklist.
-----
Ryan Gray is a Technical Consultant with Portainer.io. Before that he spent a couple of decades building ISPs, carrier Ethernet networks, data centers, and software. He is currently obsessed with container orchestration, GitOps, edge computing, and why naming things is so hard.
-----
Hrittik is a Community Advocate at Portainer and a student in his sophomore year. Hrittik focuses on building tech communities and software with the help of cloud native technologies. He has been a speaker at conferences like the Cloud Summit and conducted various events that have collectively empowered students and professionals on DevOps and Cloud.
1. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
2. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Today’s Agenda:
● Why Redis?
● Data on Kubernetes. Why?
● Redis on Kubernetes. How?
● Checklist Redis for Production
3. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Why Redis?
● High availability and scalability
● Replication and persistence
● Performance
● Flexible data structures
● Simplicity and ease-of-use
4. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Why Data on Kubernetes?
Why NOT?
5. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
How to Deploy Redis?
● Redis Operator
● Helm Charts
6. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
What is Portainer?
● Container management made easy.
● We would use this today to deploy Redis via Bitnami’s
Redis Helm Chart.
7. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Install Redis from Bitnami Helm Chart
with Portainer
- Log into Portainer and after clicking the “Namespaces” menu,
click “Add namespace with form”
8. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Install Redis from Bitnami Helm Chart
with Portainer
- Name the namespace “redis”
- Turn off “Resource assignment” (simplicity for this demo)
- Click “Create namespace”
9. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Exposing Redis
- Select the “redis” namespace
- Give it the name “redis”
In Portianer’s “Show Custom Values” for the Helm chart
- Set *.service.type to NodePort
- Set *.service.NodePorts.redis to something between 30000 and 32767
*Found on or near lines 431 and 441
10. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Considerations
1. Persistence strategy
2. Disabling THP
3. Benchmarking resources
11. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Persistence Strategy
Redis itself provides several persistence options:
● RDB (Redis Database): Performs point-in-time snapshots of your dataset at specified intervals.
● AOF (Append Only File): Logs every write operation that the server receives. When the server starts
up, the log is replayed to reconstruct the original dataset. Commands are logged in the same format
as the Redis protocol itself, in an append-only fashion.
● No persistence: Persistence can be disabled completely. In this state, Redis data will exist for as
long as the server is running.
● RDB + AOF: AOF and RDB can be used at the same time. In this state, when Redis restarts the AOF
file will be used to reconstruct the original dataset because the AOF file is guaranteed to be the most
complete.
12. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Disabling THP
After deploying Redis, you are likely to see the following warning…
WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will
create latency and memory usage issues with Redis. To fix this issue run the command 'echo
never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local
in order to retain the setting after a reboot. Redis must be restarted after THP is disabled
This can be automated using init-containers.
13. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Benchmarking Resources
● Redis Benchmark: included with Redis
● Memtier Benchmark: also developed by Redis Labs
● Redis Memory Analyzer: Python tool by GameNet
● YCSB: from Yahoo Cloud
● PerfKit Benchmarker: from Google Cloud
● Redis RDB tools: parses Redis dump.rdb files
● Harvest: samples Redis keys and shows top key
prefixes
14. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Thank you!
15. Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Editor's Notes
Most applications DO need storage: k8s is mature enough to help you with your storage needs