3. What is REDIS ?
Redis can persist data to the disk Redis is not only a key-value store
Redis is a different evolution path in the key-value databases where values are
complex data types that are closely related to fundamental data structures and
are exposed to the programmer as such, without additional abstraction layers.
Official webpage: http://redis.io Source Code: https://github.com/antirez/redis Online Demo: http://try.redis.io
Can be used as Database, a Caching layer or a Message broker.
Redis is an advanced key-value store, where keys can contain data structures such
as strings, hashes, lists, sets, and sorted sets. Supporting a set of atomic
operations on these data types.
Redis is fast
4. What is not REDIS
Redis is not a replacement for Relational Databases nor Document Stores.
It might be used complementary to a SQL relational store, and/or NoSQL
document store.
Even when Redis offers configurable mechanisms for persistency, increased
persistency will tend to increase latency and decrease throughput.
Best used for rapidly changing data with a foreseeable database size (should fit
mostly in memory).
NoSQL comparisons:
http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
http://www.infoivy.com/2013/07/nosql-database-comparison-chart-only.html
5. Redis Use Cases
Caching
Counting things
Blocking queues
Pub/Sub (service bus)
MVC Output Cache provider
Backplane for SignalR
ASP.NET Session State provider*
Online user data (shopping cart, …)
* ASP.NET session state providers comparison: http://www.slideshare.net/devopsguys/best-performing-aspnet-session-state-providers
… Any real-time, cross-platform, cross-application communication
6. When to consider Redis
Speed is critical
More than just key-value pairs
Dataset can fit in memory
Dataset is not critical
From: http://www.slideshare.net/autonomous/redis-overview-for-software-architecture-forum
7. Redis Architecture
Written in ANSI C by Salvatore Sanfilippo (@antirez).
Works in most POSIX systems like Linux, BSD and OS X.
Linux is the recommended
No official support for Windows, but Microsoft develops and maintains an open
source Win-64 port of Redis*
Redis is a single-threaded server, not designed to benefit from multiple CPU cores.
Several Redis instances can be launched to scale out on several cores.
All operations are atomic (no two commands can run at the same time).
It executes most commands in O(1) complexity and with minimal lines of code.
*Redis on Windows: https://github.com/MSOpenTech/redis
9. Redis data types
*Redis data types internals: https://cs.brown.edu/courses/cs227/archives/2011/slides/mar07-redis.pdf
Redis Data Type Contains Read/write ability
String
Binary-safe strings (up to 512 MB), Integers or
Floating point values, Bitmaps.
Operate on the whole string, parts, increment/decrement
the integers and floats, get/set bits by position.
Hash Unordered hash table of keys to string values
Add, fetch, or remove individual ítems by key, fetch the
whole hash.
List Doubly linked list of strings
Push or pop items from both ends, trim based on offsets,
read individual or multiple items, find or remove items by
value.
Set Unordered collection of unique strings
Add, fetch, or remove individual items, check
membership, intersect, union, difference, fetch random
items.
Sorted Set
Ordered mapping of string members to
floating-point scores, ordered by score
Add, fetch, or remove individual items, fetch items based
on score ranges or member value.
Geospatial
index
Sorted set implementation using geospatial
information as the score
Add, fetch or remove individual items, search by
coordinates and radius, calculate distance.
HyperLogLog
Probabilistic data structure to count unique
things using 12Kb of memory
Add individual or multiple items, get the cardinality.
Value1Key1
Value2Key2
Lon.:
-103.55328
Lat.:
20.63373
Value
10000110
...10
I m a string!
...0000110
ACBD
CBCA
C: 250A: 250D: 0.3B: 0.1
12. Redis Commands - Basic
Get/Set strings SET [key value] / GET [key]
redis> SET foo “hello!“ O(1)
OK
redis> GET foo
“hello!“
Increment numbers INCRBY [key increment]
redis> SET bar 223 O(1)
OK
redis> INCRBY bar 1000
(integer) 1223
Get multiple keys at once MGET [key key …]
redis> MGET foo bar O(N) : N=# of keys.
1. “hello!"
2. "1223“
Set multiple keys at once MSET [key value key value …]
> MSET foo “hello!” bar 1223 O(N) : N=# of keys.
OK Set key expiration EXPIRE [key seconds]
O(1)
redis> EXPIRE foo 10
(integer) 1
Rename a key RENAME [key newkey]
redis> RENAME bar new_bar O(1)
OK
Update a value retrieving the old one GETSET [key value]
redis> GETSET foo “bye!” O(1)
“hello!"
redis> GET foo
“bye!"
Key removal DEL [key …]
O(1)
redis> DEL foo
(integer) 1
Test for existence EXISTS [key …]
O(1)
redis> EXISTS foo
(integer) 1
Get the length of a string STRLEN [key]
O(1)
redis> STRLEN foo
(integer) 6
Get the type of a key TYPE [key]
O(1)
redis> TYPE foo
string
Strings Keys
Get key time-to-live TTL [key]
O(1)
redis> TTL foo
(integer) 10
13. Redis Commands – Lists & Hashes
Push on either end RPUSH/LPUSH [key value]
redis> RPUSH jobs “foo” O(1)
(integer) 1
redis> LPUSH jobs “bar”
(integer) 1
Pop from either end RPOP/LPOP [key]
redis> RPOP jobs O(1)
“foo”
redis> LPOP jobs
“bar”
Blocking Pop BRPOP/BLPOP [key]
redis> BLPOP jobs O(1)
redis> BRPOP jobs
Get a range of elements LRANGE [key start stop]
redis> LRANGE jobs 0 -1 O(N)
1. “bar"
2. “foo“
Pop and Push to another list RPOPLPUSH [src dst]
O(1)
redis> RPOPLPUSH jobs proc
“foo”
Get an element by index LINDEX [key index]
redis> LINDEX jobs 1 O(N)
“foo”
Set a hashed value HSET [key field value]
O(1)
redis> HSET user:1 name John
(integer) 1
Set multiple fields HMSET [key field value …]
O(1)
redis> HMSET user:1 lastname Smith visits 1
OK
Get a hashed value HGET [key field]
O(1)
redis> HGET user:1 name
“John”
Get all the values in a hash HGETALL [key]
O(N) : N=size of hash.
redis> HGETALL user:1
1) "name"
2) "John"
3) "lastname"
4) "Smith"
5) "visits"
6) "1"
Increment a hashed value HINCRBY [key field incr]
O(1)
redis> HINCRBY user:1 visits 1
(integer) 2
Lists Hashes
14. Redis Commands – Sets & Sorted sets
Add member to a set SADD [key member ...]
redis> SADD admins “Peter” O(1)
(integer) 1
redis> SADD users “John” “Peter”
(integer) 2
Pop a random element SPOP [key]
O(1)
redis> SPOP users
“John”
Get all elements SMEMBERS [key]
redis> SMEMBERS users O(N) : N=size of set.
1) "Peter"
2) "John"
Intersect multiple sets SINTER [key key …]
redis> SINTER users admins O(N)
1. “Peter"
Union multiple sets SUNION [key key …]
redis> SUNION users admins O(N)
1) "Peter"
2) "John“
Diff. multiple sets DIFF [key key …]
O(N)
redis> SDIFF users admins
1) "John“
Add member to a sorted set ZADD [key score member]
redis> ZADD scores 100 “John” O(log(N))
(integer) 1
redis> ZADD scores 50 “Peter” 200 “Charles” 1000 “Mary”
(integer) 3
Get the rank of a member ZRANK [key member]
O(log(N))
redis> ZRANK scores “Mary”
(integer) 3
Get elements by score range ZRANGEBYSCORE [key min max]
O(log(N))
redis> ZRANGEBYSCORE scores 200 +inf WITHSCORES
1) “Charles“
2) 200
3) “Mary“
4) 1000
Increment score of member ZINCRBY [key incr member]
O(log(N))
redis> ZINCRBY scores 10 “Mary”
“1010”
Remove range by score ZREMRANGEBYSCORE [key min max]
O(log(N))
redis> ZREMRANGEBYSCORE scores 0 100
(integer) 2
Sets Sorted sets
15. Scaling Redis
Persistence
Redis provides two mechanisms to deal with persistence: Redis database snapshots (RDB) and
append-only files (AOF).
Master
Slave
Slave
Master
Master
Master
Master
Master
Master
SlaveMaster
Redis
Disk
Replication
A Redis instance known as the master, ensures that one or more instances kwown as the slaves,
become exact copies of the master. Clients can connect to the master or to the slaves. Slaves are
read only by default.
Partitioning
Breaking up data and distributing it across different hosts in a cluster.
Can be implemented in different layers:
Client: Partitioning on client-side code.
Proxy: An extra layer that proxies all redis queries and performs partitioning
(i.e. Twemproxy).
Query Router: instances will make sure to forward the query to the right node.
(i.e Redis Cluster).
Failover
Manual
Automatic with Redis Sentinel (for master-slave topology)
Automatic with Redis Cluster (for cluster topology)
16. Redis topologies
I. Standalone
II. Sentinel (automatic failover)
III. Twemproxy (distribute data)
IV. Cluster (automatic failover and distribute data)
17. Redis topologies I - Standalone
Master-slave
The master data is optionally replicated to slaves.
The slaves provides data redundancy, reads offloading and save-to-disk offloading.
Clients can connect to the Master for read/write operations or to the Slaves for
read operations.
Slaves can also replicate to its own slaves.
There is no automatic failover.
Redis
Master
Redis
Slave
Redis
Slave
Master only Master-slave multi-level
Redis
Master
Redis
Slave
Redis
Slave
Redis
Slave
Redis
SlaveRedis
Master
18. Redis topologies II - Sentinel
Master-slave with Sentinel
Redis Sentinel provides a reliable automatic failover in a master/slave topology,
automatically promoting a slave to master if the existing master fails.
Sentinel does not distribute data across nodes.
19. Twemproxy (single)
Twemproxy* works as a proxy between the clients and many Redis instances.
Is able to automatically distribute data among different standalone Redis instances.
Supports consistent hashing with different strategies and hashing functions.
Multi-key commands and transactions are not supported.
Twemproxy (load balanced)
*Twemproxy is a project from Twitter and is not part of redis: https://github.com/twitter/twemproxy
Redis topologies III - Twemproxy
**Illustrations from Redis Essentials book by Maxwell Dayvson Da Silva and Hugo Lopes Tavares
20. Redis Cluster
Redis Cluster distributes data across different Redis instances and perform automatic
failover if any problem happens to any master instance.
All nodes are directly connected with a service channel.
The keyspace is divided into hash slots. Different nodes will hold a subset of hash slots.
Multi-key commands are only allowed for keys in the same hash slot.
Redis topologies IV - Cluster
Master A
Master B
Slave A1
Slave A2
Slave B1
Master C
Slave B1
Slots 0-6000
Slots 6001-12000
Slots 12001-16383
Servicechannel
Slave B12
Slave B11
Slave A22
Slave A21
21. Redis advantages
Performance
Availability
Fault-Tolerance
Scalability (adaptability)
Portability
Support for complex data types and structures
Atomic operations and Transactions
Pipelining (send multiple commands at once)
LUA scripting support
LRU eviction of keys
Keys with limited time-to-live
Simple to install, setup and manage
Highly configurable
Supported on many languages
Straightforward and well documented API
Open Source
Available on Azure
NoSQL & SQL response performance comparison
(from https://redislabs.com/blog/the-proven-redis-performance)
22. Questions
Redis Use Cases
Redis Architecture
Redis Data Types
Redis Commands
Redis Scalability
Redis Topologies
Redis Official webpage: http://redis.io Online Demo: http://try.redis.io
Source Code: https://github.com/antirez/redis This PPT: http://bit.ly/1N87DJR
Redis/Memcached comparison: http://db-engines.com/en/system/Memcached%3BRiak%3BRedis
Most accepted meaning of NoSQL is Not Only SQL.
Because Redis does so much, it's hard to precisely categorise it. It feels substantially different to the other NoSQL software.
Performance is Redis's best feature. Mainly due to keeping the entire dataset in memory, and only periodically syncing to disk.
Persistence to disk means you can use Redis as a real database instead of just a volatile cache.
Redis is like Memcached, but with built-in persistence(snapshotting or journaling to disk), more datatypes, built-in pub/sub, transactions and Lua scripting.
Redis is FAR faster than any RDBMS, but it also can't do as much. If you can map your RDBMS use case to Redis then it might be worth taking a look, but if you can't don't force it.
Highly scalable data store shared by multiple processes, multiple applications, or multiple servers.
You can communicate cross-platform, cross-server, or cross-application just as easily makes it a pretty great choice for many use cases. Its speed makes it great as a caching layer.
Popular sites using Redis: Twitter, Instagram, Vine, Github, StackOverflow, Pinterest, VMWare, Snapchat, Digg and more…
Redis is primarily an in-memory database. So there's no guarantee that data has actually been written to disk when a Redis command returns. So a server crash is very likely to lose some data (eventual consistency).
There are a number of ways in which you can improve durability, generally by trading off performance.
>>>>>>>> Download (https://github.com/MSOpenTech/redis/releases), Start Redis-Server, Start redis-cli, Show a couple of commands (SET, GET, EXPIRE, TTL)
Binary-safe strings means that values are essentially byte strings, and can contain any byte (including the null byte).
A hash allow compound objects to be stored in a single Redis value.
A list works fine as a queue or as a stack, with blocking capabilities.
Sorted sets works like a set but each member has an associated score. The set is kept ordered by that score, and range queries are extremely efficient, sorted in either direction.
HyperLogLog
HyperLogLog provides a good approximation of the cardinality of a set using a very small amount of memory. In Redis it only uses 12kbytes per key to count with a standard error of 0.81%, and there is virtually no limit to the number of items you can count.
redis-cli is definitely the best way of experimenting with Redis. Run without arguments it connects to the local Redis server and provides an interactive prompt.
>>>>>>>> Run redis-benchmark
>>>>>>>> Show page http://redis.io/commands
Lists
Lists are doubly-linked lists.
You can push and pop at both sides, extract range, resize, etc.
Random access and ranges at O(N)
Hashes
Hash tables as values
Think of an object store with atomic access to object members
Sets
Sets are sets of unique values w/push, pop, etc.
Sets can be intersected/diffed/union'ed server side.
Sorted sets
Same as sets, but with score per element
Ranked ranges, aggregation of scores on INTERSECT
Persistence
The RDB persistence performs point-in-time snapshots of your dataset at specified intervals.
The AOF persistence logs every write operation received by the server, that will be played again at server startup, reconstructing the original dataset.
It is possible to combine both AOF and RDB in the same instance. In this case, when Redis restarts, the AOF file will be used to reconstruct the original dataset.
Replication
Slave write mode can be enabled but will be ephemeral (until restart or resync).
Slaves provides data redundancy, reads offloading and save-to-disk offloading.
Redis Sentinel provides high availability for Redis. In practical terms this means that using Sentinel you can create a Redis deployment that resists without human intervention to certain kind of failures.
Sentinel processes cooperates for Failure Detection (multiple Sentinels agree about the fact a given master is no longer available).
Sentinel works even if not all the Sentinel processes are working, making the system robust against failures.
>>>>>>>> Run the cluster
High-Performance: designed for speed (in-memory, O(1) ops).
High-Availability: Sentinel, Cluster, Replication, Partitioning.
Fault-Tolerance: Failover mechanisms. (Manual or Automatic).
Scalability: Scales down as well as up. It's equally useful for tiny projects as it is for huge services.
Portability: Runs on multiple OS and multiple client languages.