SlideShare une entreprise Scribd logo
1  sur  41
Redis Scaling and
Observability
Agenda
➔ Redis Quiz
➔ Redis Overview
➔ Setup of Redis
➔ Scaling in Redis
➔ Observability of Redis
Redis Overview
Q1 - According to your understanding, what do you mean by Redis?
A. It’s a NoSQL database B. It’s a caching
system
C. It’s a SQL
database.
D. Both A and B
D. Both A and B
Redis Overview
Q2 - What are the different type of Redis setup are possible?
● Standalone setup
● Replicated cluster
setup
● Sharded cluster setup
Redis Overview
Q2 - Can we use Redis as a system with the traditional SQL databases?
A. True B. False
A. True
Redis Overview
In-Memory Database Data stored on system/server’s memory
Usage of Redis Often used as Cache to improve performance
Redis can also be used as full fledged database
as well.
Application Redis Database
Usage of Redis
Client Application Redis MySQL DB
1. Looks in cache system.
3. Cache the
new data
Cache hit simply
return the data.
2. Cache miss will look into the main
persistent database.
Key Properties of Redis
In-Memory Database Fast and Performant Schemaless
Types of Redis Setup
Sharded Cluster
Replicated Cluster
Standalone Standalone
Leader
Follower Follower
Leader Leader Leader
Follower Follower Follower
Standalone Setup
Redis
Server
Client
Applications
A single application or multiple
applications can connect to the
Redis server for writing and
reading keys.
Challenges of Standalone Setup
Out of Memory Data Bottleneck Failover Scalability
Replication
Leader
A B C D
Follower
A B C D
Follower
A B C D
Sharding
Leader 1
A B
Leader 2
C D
Leader 3
E F
Replication + Sharding
Leader 1
A B
Leader 2
C D
Leader 3
E F
Follower 1
A B
Follower 2
C D
Follower 3
E F
Replication of data from
leader to follower
Redirection of the
data
Replicated Setup
Redis
Leader
Client
Applications
Redis Follower
Redis Follower
Redis follower is replicating the
data that is being written on the
redis leader server
Client application will write the
data on the leader server of
Redis
All read requests will be redirected to the
followers node to reduce the network noise
from leader node of Redis.
Replicated Setup with Sentinel
Redis
Leader
Redis Follower
Redis Follower
Redis follower is replicating the
data that is being written on the
redis leader server
Redis
Sentinel
Redis sentinel keeping watch on
replicated cluster, if leader will fail, it will
promote the follower to the leader
System
Failure
Redis
Leader
Redis Follower
Challenges of Replicated Setup
Redis
Leader
Redis
Follower
Redis
Follower
● Dataset can become large
for leader and follower node
which can crash Redis
process.
● A single master can become
a bottleneck for the data
Sharded Cluster Setup
Leader 1 Leader 2 Leader 3
Follower 1 Follower 2 Folower 3
Leader is sharding data
between the leader
nodes
Followers replicating
data from leader nodes
Redirection for data
search and write
Write requests can be
sent on leader nodes
Read requests can be
sent on follower nodes
Client Application
Client Application
Setup Comparison
Standalone
Replicated
Setup
Sharded
Setup
Data
Bottleneck
High Medium Low
Out of
Memory Risk
High Medium Low
Setup Comparison
Standalone
Replicated
Setup
Sharded
Setup
No chances Medium
chances
High chances
Low Medium High
Failover
Scalability
Complexity
of Setup
Low Medium High
Scaling Redis Setup
Redis
Standalone
Redis
Leader
Redis
Follower
Redis
Follower
Leader 1 Leader 2 Leader 3
Follower 1 Follower 2 Folower 3
Replicated Setup
Sharded Setup
Scaling Replicated Setup
Redis
Leader
Redis Follower
Redis Follower
Redis Sentinel
Redis Follower
Redis Sentinel
Redis Sentinel
Scaling replication
followers
Scaling sentinel
for failover and
cluster
Scaling Sharded Setup
Redis Leader 1
A B
Redis Leader 2
C D
Redis Leader 3
F G
Redis Follower 1
A B
Redis Follower 2 Redis Follower 3
E H
C D E F G H
Redis Leader 4
Redis Follower 4
H
E
H
E
Resharding
Challenges in Scaling Redis
Sentinel configuration
and scaling
Resharding of data in
cluster mode
Complexity in scaling
Redis
Monitoring of Redis
Redis
Redis
Exporter
Prometheus
Grafana
Dashbaord
Prometheus pulling metrics
from the redis exporter at
defined interval
Grafana using
Prometheus as the
datasource for
metrics
Panels and dashboards
created on Grafana
platform
Important Monitoring Metrics for Redis
Uptime Latency Expired vs
non-expired
keys
Size of DB
CPU Memory Througput Connection
Count
Grafana Dashboard
Redis Setup on Kubernetes
Standalone
Redis Pod
Leader
Follower Follower
Leader
Follower
Leader Leader
Follower Follower
Challenges of Redis On Kubernetes
Dynamic IP update of
redis pod
Scaling Redis cluster on-
fly
Monitoring of Redis
system
Production ready
configuration
Security in terms of
Kubernetes and Redis
Managing persistent
data
Solution of Redis on Kubernetes
OT Redis Operator
Host, Run and Scale Redis cluster on
Kubernetes.
Features of Redis Operator
Multiple Redis version
supported
Standalone and Cluster
mode supported
Automated failover and
recovery is supported
In built monitoring
support with exporter
Password and
password-less support
TLS support for security
ipv4 ipv6
Ipv4 and Ipv6 support
for Redis
Detailed monitoring
dashboard of Grafana
On-fly upgrade without
downtime
Redis Operator Architecture
Standalone Setup of Redis
Cluster Setup of Redis
Enabling Monitoring
Redis Pod
Redis
Redis
Exporter
Upgrading Redis
v6 v7
TLS Integration
Contributors Behind Redis Operator
Abhishek Dubey,
Sr. DevOps Consultant,
Opstree Solutions
Sandeep Rawat,
CTO, Opstree Solutions
Shubham Gupta,
DevOps Intern, Opstree
Solutions
Mathieu Cesbron, Developer
from France
David Collam, Staff
Solutions Engineer,
Jetstack (Cert Manager)
Ivyan, Software Engineer,
DaoCloud
Contributors Behind Redis Operator
Andrew Thompson,
Software Engineer,
CrowdStrike
John, Software Engineer,
Naver
Hubert FONGARNAND,
Software Engineer, Ippon
Anish Chakraborty, Sr.
Software Engineer, Spotify
Tanuj Dwivedi, Sr. Software
Engineer, Dream11
Philippe Martin, Software
Engineer, RedHat
And Many More….
Thank you!
Contact:- abhishek.dubey@opstree.com, Sandeep@opstree.com
http://opstree.com
http://buildpiper.io

Contenu connexe

Tendances

Tendances (20)

Deploying & Scaling your Odoo Server
Deploying & Scaling your Odoo ServerDeploying & Scaling your Odoo Server
Deploying & Scaling your Odoo Server
 
[2019] 200만 동접 게임을 위한 MySQL 샤딩
[2019] 200만 동접 게임을 위한 MySQL 샤딩[2019] 200만 동접 게임을 위한 MySQL 샤딩
[2019] 200만 동접 게임을 위한 MySQL 샤딩
 
New Framework - ORM
New Framework - ORMNew Framework - ORM
New Framework - ORM
 
Introduction to node.js
Introduction to node.jsIntroduction to node.js
Introduction to node.js
 
NEXT.JS
NEXT.JSNEXT.JS
NEXT.JS
 
Altinity Quickstart for ClickHouse
Altinity Quickstart for ClickHouseAltinity Quickstart for ClickHouse
Altinity Quickstart for ClickHouse
 
NDC 11 자이언트 서버의 비밀
NDC 11 자이언트 서버의 비밀NDC 11 자이언트 서버의 비밀
NDC 11 자이언트 서버의 비밀
 
What is Game Server ?
What is Game Server ?What is Game Server ?
What is Game Server ?
 
Introduction to NodeJS
Introduction to NodeJSIntroduction to NodeJS
Introduction to NodeJS
 
Cloud arch patterns
Cloud arch patternsCloud arch patterns
Cloud arch patterns
 
[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요
[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요
[2B7]시즌2 멀티쓰레드프로그래밍이 왜 이리 힘드나요
 
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
All About JSON and ClickHouse - Tips, Tricks and New Features-2022-07-26-FINA...
 
The MySQL SYS Schema
The MySQL SYS SchemaThe MySQL SYS Schema
The MySQL SYS Schema
 
ClickHouse Keeper
ClickHouse KeeperClickHouse Keeper
ClickHouse Keeper
 
Redis at LINE
Redis at LINERedis at LINE
Redis at LINE
 
Cassandra sharding and consistency (lightning talk)
Cassandra sharding and consistency (lightning talk)Cassandra sharding and consistency (lightning talk)
Cassandra sharding and consistency (lightning talk)
 
Node js
Node jsNode js
Node js
 
Database
DatabaseDatabase
Database
 
gRPC Design and Implementation
gRPC Design and ImplementationgRPC Design and Implementation
gRPC Design and Implementation
 
Introduction to GraphQL
Introduction to GraphQLIntroduction to GraphQL
Introduction to GraphQL
 

Similaire à Redis and its Scaling and Obersvability

Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...
Dinh Pham
 
Windows Server 2012 R2 Jump Start - Intro
Windows Server 2012 R2 Jump Start - IntroWindows Server 2012 R2 Jump Start - Intro
Windows Server 2012 R2 Jump Start - Intro
Paulo Freitas
 
RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...
RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...
RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...
Redis Labs
 

Similaire à Redis and its Scaling and Obersvability (20)

Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...Beyond relational database - Building high performance websites using Redis a...
Beyond relational database - Building high performance websites using Redis a...
 
Redis meetup
Redis meetupRedis meetup
Redis meetup
 
Redis. Seattle Data Science and Data Engineering Meetup
Redis. Seattle Data Science and Data Engineering MeetupRedis. Seattle Data Science and Data Engineering Meetup
Redis. Seattle Data Science and Data Engineering Meetup
 
An Introduction to Redis for .NET Developers.pdf
An Introduction to Redis for .NET Developers.pdfAn Introduction to Redis for .NET Developers.pdf
An Introduction to Redis for .NET Developers.pdf
 
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
Moving Beyond Cache by Yiftach Shoolman Redis Labs - Redis Day Seattle 2020
 
Add Redis to Postgres to Make Your Microservices Go Boom!
Add Redis to Postgres to Make Your Microservices Go Boom!Add Redis to Postgres to Make Your Microservices Go Boom!
Add Redis to Postgres to Make Your Microservices Go Boom!
 
Windows Server 2012 R2 Jump Start - Intro
Windows Server 2012 R2 Jump Start - IntroWindows Server 2012 R2 Jump Start - Intro
Windows Server 2012 R2 Jump Start - Intro
 
R & Python on Hadoop
R & Python on HadoopR & Python on Hadoop
R & Python on Hadoop
 
What's new with enterprise Redis - Leena Joshi, Redis Labs
What's new with enterprise Redis - Leena Joshi, Redis LabsWhat's new with enterprise Redis - Leena Joshi, Redis Labs
What's new with enterprise Redis - Leena Joshi, Redis Labs
 
The Fastest Way to Redis on Pivotal Cloud Foundry
The Fastest Way to Redis on Pivotal Cloud FoundryThe Fastest Way to Redis on Pivotal Cloud Foundry
The Fastest Way to Redis on Pivotal Cloud Foundry
 
RedisConf 2016 - Redis usage and ecosystem
RedisConf 2016 - Redis usage and ecosystemRedisConf 2016 - Redis usage and ecosystem
RedisConf 2016 - Redis usage and ecosystem
 
RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...
RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...
RedisConf17 - Redis Enterprise: Continuous Availability, Unlimited Scaling, S...
 
#GeodeSummit - Redis to Geode Adaptor
#GeodeSummit - Redis to Geode Adaptor#GeodeSummit - Redis to Geode Adaptor
#GeodeSummit - Redis to Geode Adaptor
 
MariaDB: Connect Storage Engine
MariaDB: Connect Storage EngineMariaDB: Connect Storage Engine
MariaDB: Connect Storage Engine
 
windows server 2012 R2
windows server 2012 R2windows server 2012 R2
windows server 2012 R2
 
Windows server Interview question and answers
Windows server Interview question and answersWindows server Interview question and answers
Windows server Interview question and answers
 
Redis data structure and Performance Optimization
Redis data structure and Performance OptimizationRedis data structure and Performance Optimization
Redis data structure and Performance Optimization
 
Redis as a Cache Boosting Performance and Scalability
Redis as a Cache Boosting Performance and ScalabilityRedis as a Cache Boosting Performance and Scalability
Redis as a Cache Boosting Performance and Scalability
 
Redis vs Memcached
Redis vs MemcachedRedis vs Memcached
Redis vs Memcached
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 

Dernier

TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
Wonjun Hwang
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 

Dernier (20)

Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxCyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistan
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 

Redis and its Scaling and Obersvability