SlideShare a Scribd company logo
1 of 15
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Today’s Agenda:
● Why Redis?
● Data on Kubernetes. Why?
● Redis on Kubernetes. How?
● Checklist Redis for Production
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Why Redis?
● High availability and scalability
● Replication and persistence
● Performance
● Flexible data structures
● Simplicity and ease-of-use
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Why Data on Kubernetes?
Why NOT?
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
How to Deploy Redis?
● Redis Operator
● Helm Charts
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
What is Portainer?
● Container management made easy.
● We would use this today to deploy Redis via Bitnami’s
Redis Helm Chart.
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Install Redis from Bitnami Helm Chart
with Portainer
- Log into Portainer and after clicking the “Namespaces” menu,
click “Add namespace with form”
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Install Redis from Bitnami Helm Chart
with Portainer
- Name the namespace “redis”
- Turn off “Resource assignment” (simplicity for this demo)
- Click “Create namespace”
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Exposing Redis
- Select the “redis” namespace
- Give it the name “redis”
In Portianer’s “Show Custom Values” for the Helm chart
- Set *.service.type to NodePort
- Set *.service.NodePorts.redis to something between 30000 and 32767
*Found on or near lines 431 and 441
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Considerations
1. Persistence strategy
2. Disabling THP
3. Benchmarking resources
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Persistence Strategy
Redis itself provides several persistence options:
● RDB (Redis Database): Performs point-in-time snapshots of your dataset at specified intervals.
● AOF (Append Only File): Logs every write operation that the server receives. When the server starts
up, the log is replayed to reconstruct the original dataset. Commands are logged in the same format
as the Redis protocol itself, in an append-only fashion.
● No persistence: Persistence can be disabled completely. In this state, Redis data will exist for as
long as the server is running.
● RDB + AOF: AOF and RDB can be used at the same time. In this state, when Redis restarts the AOF
file will be used to reconstruct the original dataset because the AOF file is guaranteed to be the most
complete.
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Disabling THP
After deploying Redis, you are likely to see the following warning…
WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will
create latency and memory usage issues with Redis. To fix this issue run the command 'echo
never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local
in order to retain the setting after a reboot. Redis must be restarted after THP is disabled
This can be automated using init-containers.
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Benchmarking Resources
● Redis Benchmark: included with Redis
● Memtier Benchmark: also developed by Redis Labs
● Redis Memory Analyzer: Python tool by GameNet
● YCSB: from Yahoo Cloud
● PerfKit Benchmarker: from Google Cloud
● Redis RDB tools: parses Redis dump.rdb files
● Harvest: samples Redis keys and shows top key
prefixes
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”
Thank you!
Hrittik Roy | Ryan Gray
DoK Day Europe 2022 @ KubeCon
“Resilient Redis”

More Related Content

More from DoKC

The Kubernetes Native Database
The Kubernetes Native DatabaseThe Kubernetes Native Database
The Kubernetes Native Database
DoKC
 
Overcoming challenges with protecting and migrating data in multi-cloud K8s e...
Overcoming challenges with protecting and migrating data in multi-cloud K8s e...Overcoming challenges with protecting and migrating data in multi-cloud K8s e...
Overcoming challenges with protecting and migrating data in multi-cloud K8s e...
DoKC
 
We will Dok You! - The journey to adopt stateful workloads on k8s
We will Dok You! - The journey to adopt stateful workloads on k8sWe will Dok You! - The journey to adopt stateful workloads on k8s
We will Dok You! - The journey to adopt stateful workloads on k8s
DoKC
 
Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...
Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...
Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...
DoKC
 
Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?
DoKC
 
What we've learned from running a PostgreSQL managed service on Kubernetes
What we've learned from running a PostgreSQL managed service on KubernetesWhat we've learned from running a PostgreSQL managed service on Kubernetes
What we've learned from running a PostgreSQL managed service on Kubernetes
DoKC
 
Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...
Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...
Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...
DoKC
 
Using Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” serviceUsing Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” service
DoKC
 
The many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent dataThe many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent data
DoKC
 

More from DoKC (20)

Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
Run PostgreSQL in Warp Speed Using NVMe/TCP in the CloudRun PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
 
The Kubernetes Native Database
The Kubernetes Native DatabaseThe Kubernetes Native Database
The Kubernetes Native Database
 
ING Data Services hosted on ICHP DoK Amsterdam 2023
ING Data Services hosted on ICHP DoK Amsterdam 2023ING Data Services hosted on ICHP DoK Amsterdam 2023
ING Data Services hosted on ICHP DoK Amsterdam 2023
 
Implementing data and databases on K8s within the Dutch government
Implementing data and databases on K8s within the Dutch governmentImplementing data and databases on K8s within the Dutch government
Implementing data and databases on K8s within the Dutch government
 
StatefulSets in K8s - DoK Talks #154
StatefulSets in K8s - DoK Talks #154StatefulSets in K8s - DoK Talks #154
StatefulSets in K8s - DoK Talks #154
 
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
 
Analytics with Apache Superset and ClickHouse - DoK Talks #151
Analytics with Apache Superset and ClickHouse - DoK Talks #151Analytics with Apache Superset and ClickHouse - DoK Talks #151
Analytics with Apache Superset and ClickHouse - DoK Talks #151
 
Overcoming challenges with protecting and migrating data in multi-cloud K8s e...
Overcoming challenges with protecting and migrating data in multi-cloud K8s e...Overcoming challenges with protecting and migrating data in multi-cloud K8s e...
Overcoming challenges with protecting and migrating data in multi-cloud K8s e...
 
Evaluating Cloud Native Storage Vendors - DoK Talks #147
Evaluating Cloud Native Storage Vendors - DoK Talks #147Evaluating Cloud Native Storage Vendors - DoK Talks #147
Evaluating Cloud Native Storage Vendors - DoK Talks #147
 
Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...
Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...
Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your State...
 
We will Dok You! - The journey to adopt stateful workloads on k8s
We will Dok You! - The journey to adopt stateful workloads on k8sWe will Dok You! - The journey to adopt stateful workloads on k8s
We will Dok You! - The journey to adopt stateful workloads on k8s
 
Mastering MongoDB on Kubernetes, the power of operators
Mastering MongoDB on Kubernetes, the power of operators Mastering MongoDB on Kubernetes, the power of operators
Mastering MongoDB on Kubernetes, the power of operators
 
Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...
Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...
Leveraging Running Stateful Workloads on Kubernetes for the Benefit of Develo...
 
Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?
 
What's New in Kubernetes Storage
What's New in Kubernetes StorageWhat's New in Kubernetes Storage
What's New in Kubernetes Storage
 
What we've learned from running a PostgreSQL managed service on Kubernetes
What we've learned from running a PostgreSQL managed service on KubernetesWhat we've learned from running a PostgreSQL managed service on Kubernetes
What we've learned from running a PostgreSQL managed service on Kubernetes
 
Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...
Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...
Weathering The Cloud Storm: Modern Data Management Patterns for Reliability a...
 
Using Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” serviceUsing Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” service
 
The many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent dataThe many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent data
 
The Data on Kubernetes Landscape
The Data on Kubernetes LandscapeThe Data on Kubernetes Landscape
The Data on Kubernetes Landscape
 

Resilient Redis

  • 1. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis”
  • 2. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Today’s Agenda: ● Why Redis? ● Data on Kubernetes. Why? ● Redis on Kubernetes. How? ● Checklist Redis for Production
  • 3. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Why Redis? ● High availability and scalability ● Replication and persistence ● Performance ● Flexible data structures ● Simplicity and ease-of-use
  • 4. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Why Data on Kubernetes? Why NOT?
  • 5. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” How to Deploy Redis? ● Redis Operator ● Helm Charts
  • 6. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” What is Portainer? ● Container management made easy. ● We would use this today to deploy Redis via Bitnami’s Redis Helm Chart.
  • 7. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Install Redis from Bitnami Helm Chart with Portainer - Log into Portainer and after clicking the “Namespaces” menu, click “Add namespace with form”
  • 8. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Install Redis from Bitnami Helm Chart with Portainer - Name the namespace “redis” - Turn off “Resource assignment” (simplicity for this demo) - Click “Create namespace”
  • 9. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Exposing Redis - Select the “redis” namespace - Give it the name “redis” In Portianer’s “Show Custom Values” for the Helm chart - Set *.service.type to NodePort - Set *.service.NodePorts.redis to something between 30000 and 32767 *Found on or near lines 431 and 441
  • 10. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Considerations 1. Persistence strategy 2. Disabling THP 3. Benchmarking resources
  • 11. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Persistence Strategy Redis itself provides several persistence options: ● RDB (Redis Database): Performs point-in-time snapshots of your dataset at specified intervals. ● AOF (Append Only File): Logs every write operation that the server receives. When the server starts up, the log is replayed to reconstruct the original dataset. Commands are logged in the same format as the Redis protocol itself, in an append-only fashion. ● No persistence: Persistence can be disabled completely. In this state, Redis data will exist for as long as the server is running. ● RDB + AOF: AOF and RDB can be used at the same time. In this state, when Redis restarts the AOF file will be used to reconstruct the original dataset because the AOF file is guaranteed to be the most complete.
  • 12. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Disabling THP After deploying Redis, you are likely to see the following warning… WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled This can be automated using init-containers.
  • 13. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Benchmarking Resources ● Redis Benchmark: included with Redis ● Memtier Benchmark: also developed by Redis Labs ● Redis Memory Analyzer: Python tool by GameNet ● YCSB: from Yahoo Cloud ● PerfKit Benchmarker: from Google Cloud ● Redis RDB tools: parses Redis dump.rdb files ● Harvest: samples Redis keys and shows top key prefixes
  • 14. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis” Thank you!
  • 15. Hrittik Roy | Ryan Gray DoK Day Europe 2022 @ KubeCon “Resilient Redis”

Editor's Notes

  1. Most applications DO need storage: k8s is mature enough to help you with your storage needs