SlideShare une entreprise Scribd logo
1  sur  25
Redis
REmote DIctionary Server
  Ryan Findley, Philly.rb, 2010-02-16
Disclaimer
• I’m not using this in production (yet).
Kind of like Memcache


• Key-value store
• All data lives in memory
• Keys can expire (or not)
• Fast, Light-weight
And More
•   Persistence

•   Multiple databases

•   Queryable keyspace

•   Support for integer counters

•   Higher level data structures

•   Atomic operations

•   Ability to paginate lists without mutating them

•   Master-slave replication

•   Optional VM feature (new)
More:
Command Reference: http://
code.google.com/p/redis/wiki/



                                 Other Goodness
CommandReference




                   • Written in C (C99 standard)
                   • No dependencies
                   • Detailed command reference that lists time
                        complexity in Big-O notation.

                   • Very easy to build / install (it’ll probably “just
                        work”)
Persistence(1/2) -
           Snapshotting
• This is the default behavior
• Periodic, asynchronous dump to disk
• Can be every N changes or every N seconds
• For example, every 15 min if at least 1 change
 and every 5 min if at least 10 changes

• Since data is written asynchronously, data can
 be lost during a crash.
Persistence(2/2) - Append
                              Only File
See more here:
http://code.google.com/p/redis/
wiki/AppendOnlyFileHowto




               • Available as of version 1.1
               • Works like a journal
               • Every write command is logged ASAP
               • Commands replayed when the server is
                    restarted

               • Configurable speed/safety (safest by default)
More here:
http://code.google.com/
p/redis/wiki/
SelectCommand

http://code.google.com/
p/redis/wiki/
MoveCommand
                          Multiple Databases

               • Use the SELECT command; 0-15 are valid by
                    default, but you can add more in redis.conf

               • Useful for segmenting key namespaces,
                    especially when key queries are needed

               • MOVE command can be used as a locking
                    primitive
Command Reference
More:

http://code.google.com/p/
redis/wiki/
IntroductionToRedisDataTypes
                               Data Structures

              • Strings
              • Strings-as-integers (used by INCR/DECR)
              • List of Strings
              • Set of Strings (unique list)
              • Sorted Set of Strings (and weights) (ZSET)
Command overview

• Strings: get, set, increment, decrement
• Lists: push, pop, length, range, trim
• Sets: add, remove, move, length, intersect
 union, diff, random

• Other: save, lastsave can be used to force &
 verify disk persistence
More:

http://code.google.com/
p/redis/wiki/
RpoplpushCommand
                           Atomic Operations
               •    LPUSH / RPUSH: append to head/tail of list

               •    LPOP / RPOP: return & remove first/last element of list

               •    RPOPLPUSH: return & remove the last element of
                    source list and push to the head of the destination list

               •    GETSET: set a key to a new value and return the old
                    value

               •    MGET/MSET: get or set multiple keys to multiple values

               •    SMOVE: move a value from one set to another

               •    No way to group commands (transactions)
More here:
http://code.google.com/
p/redis/wiki/
ReplicationHowto

                                Replication
               • Slaves can be used for scalability or redundancy
               • A master can have multiple slaves.
               • Slaves are able to accept other slave connections
               • Redis replication is non-blocking on the master,
                    but blocking on the slave (can’t respond to
                    queries during initial sync)

               • Slaves are able to automatically reconnect after
                    an outage
Blog post about Redis
VM:

http://antirez.com/
post/redis-virtual-
memory-story.html            Optional VM feature
                •     Doesn’t use OS virtual memory

                •     Still new, may not scale well in certain situations.

                •     Don’t know how this will affect replication (initial
                      sync / re-sync may be slow, but shouldn’t block the
                      master)

                •     All keys stay in memory. Minimum req’s:

                        •   1 million keys ~ 160M

                        •   10 million keys ~ 1.6G
Syntax highlighted
sample here:

https://gist.github.com/
b16679cbb5b9573e078c
                                Setup

             • git clone git://github.com/antirez/redis.git &&
                 cd redis

             • make
             • ./redis-server
             • make test
             • (review sample config file)
Redis in Ruby


• Redis library: “gem install redis”
• All Redis commands can be called like methods
 on the connection object as described in the
 Command Reference
Example Uses


• A Memcached replacement (fast)
• A work queue (sets & lists)
• Fast auto-completion (persistent, queryable)
• ORDER BY RAND() replacement (sets)
Memcache Replacement
•   Why? Same as Memcached, but you get more “for
    free”:

•   Many databases from a single instance of Redis
    (instead of using namespaces)

•   Ability to easily backup/transfer state (dump.rdb)

•   Watch live commands on a running instance with the
    MONITOR command (instead of restarting with -v)

•   Opportunity to use Redis for other things once
    you’ve switched
Memcache Replacement
                      Note: namespace was replaced
                      with a DB number
• It’s pretty easy:
More:
                                                     Redis allows negative
http:/
                         Work Queue
      /oxfordrepo.blogspot.com/
2010/01/usage-stats-and-redis.html
                                                     array indices to count
                                                     backwards from the end.




                     A VERY simple example
              • LPUSH and RPOP were made for this:
1
2
3
4
5
6
“Not a ‘better DelayedJob’”.
More:                                                          Compare them and see what is
                                                               best for your project
http://github.com/
defunkt/resque

http://github.com/
blog/542-introducing-
                        Work Queue - Resque

               •     Created by Chris Wanstrath. Heavily inspired by
                     DelayedJob. Currently used by GitHub. Provides:

               •     A Ruby library for creating, querying, and
                     processing jobs

               •     A Rake task for starting a worker which
                     processes jobs

               •     A Sinatra app for monitoring queues, jobs, and
                     workers.
Fast Auto-Completion
• Lives in memory, doesn’t expire, persists
Replace MySQL ORDER
More:

                              BY RAND()
http://nosql.mypopescu.com/post/295433623/redis-
usecase-replacing-mysql-order-by-rand




                • ORDER BY RAND() is very slow (even with a
                     LIMIT clause)

                • Duplicating the list of IDs as a Redis set is
                     relatively cheap

                • SRANDMEMBER is fast
Other Interesting Uses of
             Redis
•   Nginx HTTP Redis module for caching with Redis
    (ngx_http_redis)

•   LLOOGG.com: Realtime site usage statistics, use
    alongside Google analytics

•   EZMobius’ Fair Work Scheduler example

•   Sikwamic: Simple Redis-backed Comet server

•   RestMQ - A REST/JSON/HTTP based message queue
    built on Redis

•   redis-textsearch - A simple full-text search for multiple
    data stores
Further reading
•   http://code.google.com/p/redis/wiki/CommandReference

•   http://antirez.com/post/redis-virtual-memory-story.html

•   http://www.slideshare.net/ezmobius/redis-remote-dictionary-server

•   http://www.paperplanes.de/2010/2/16/
    a_collection_of_redis_use_cases.html

•   http://www.dorkalev.com/2010/02/sikwamic-simple-key-value-with-
    comet.html

•   http://github.com/blog/542-introducing-resque

•   http://nosql.mypopescu.com/post/295433623/redis-usecase-replacing-
    mysql-order-by-rand

•   http://github.com/nateware/redis-textsearch

Contenu connexe

En vedette

The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
DataStax
 
Découverte de Redis
Découverte de RedisDécouverte de Redis
Découverte de Redis
JEMLI Fathi
 

En vedette (17)

Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Webinar: How MongoDB is Used to Manage Reference Data - May 2014
Webinar: How MongoDB is Used to Manage Reference Data - May 2014Webinar: How MongoDB is Used to Manage Reference Data - May 2014
Webinar: How MongoDB is Used to Manage Reference Data - May 2014
 
Securing Cassandra for Compliance
Securing Cassandra for ComplianceSecuring Cassandra for Compliance
Securing Cassandra for Compliance
 
Redis And python at pycon_2011
Redis And python at pycon_2011Redis And python at pycon_2011
Redis And python at pycon_2011
 
Redis 101
Redis 101Redis 101
Redis 101
 
Redis/Lessons learned
Redis/Lessons learnedRedis/Lessons learned
Redis/Lessons learned
 
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
The Promise and Perils of Encrypting Cassandra Data (Ameesh Divatia, Baffle, ...
 
Distributed Applications with Apache Zookeeper
Distributed Applications with Apache ZookeeperDistributed Applications with Apache Zookeeper
Distributed Applications with Apache Zookeeper
 
Redis
RedisRedis
Redis
 
Apache ZooKeeper
Apache ZooKeeperApache ZooKeeper
Apache ZooKeeper
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
BigData_Chp4: NOSQL
BigData_Chp4: NOSQLBigData_Chp4: NOSQL
BigData_Chp4: NOSQL
 
Everything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to askEverything you always wanted to know about Redis but were afraid to ask
Everything you always wanted to know about Redis but were afraid to ask
 
Introduction to Apache ZooKeeper
Introduction to Apache ZooKeeperIntroduction to Apache ZooKeeper
Introduction to Apache ZooKeeper
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
 
Découverte de Redis
Découverte de RedisDécouverte de Redis
Découverte de Redis
 
Redis - (nosqlfr meetup #2)
Redis - (nosqlfr meetup #2) Redis - (nosqlfr meetup #2)
Redis - (nosqlfr meetup #2)
 

Dernier

Dernier (20)

Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 

Redis Overview

  • 1. Redis REmote DIctionary Server Ryan Findley, Philly.rb, 2010-02-16
  • 2. Disclaimer • I’m not using this in production (yet).
  • 3. Kind of like Memcache • Key-value store • All data lives in memory • Keys can expire (or not) • Fast, Light-weight
  • 4. And More • Persistence • Multiple databases • Queryable keyspace • Support for integer counters • Higher level data structures • Atomic operations • Ability to paginate lists without mutating them • Master-slave replication • Optional VM feature (new)
  • 5. More: Command Reference: http:// code.google.com/p/redis/wiki/ Other Goodness CommandReference • Written in C (C99 standard) • No dependencies • Detailed command reference that lists time complexity in Big-O notation. • Very easy to build / install (it’ll probably “just work”)
  • 6. Persistence(1/2) - Snapshotting • This is the default behavior • Periodic, asynchronous dump to disk • Can be every N changes or every N seconds • For example, every 15 min if at least 1 change and every 5 min if at least 10 changes • Since data is written asynchronously, data can be lost during a crash.
  • 7. Persistence(2/2) - Append Only File See more here: http://code.google.com/p/redis/ wiki/AppendOnlyFileHowto • Available as of version 1.1 • Works like a journal • Every write command is logged ASAP • Commands replayed when the server is restarted • Configurable speed/safety (safest by default)
  • 8. More here: http://code.google.com/ p/redis/wiki/ SelectCommand http://code.google.com/ p/redis/wiki/ MoveCommand Multiple Databases • Use the SELECT command; 0-15 are valid by default, but you can add more in redis.conf • Useful for segmenting key namespaces, especially when key queries are needed • MOVE command can be used as a locking primitive
  • 10. More: http://code.google.com/p/ redis/wiki/ IntroductionToRedisDataTypes Data Structures • Strings • Strings-as-integers (used by INCR/DECR) • List of Strings • Set of Strings (unique list) • Sorted Set of Strings (and weights) (ZSET)
  • 11. Command overview • Strings: get, set, increment, decrement • Lists: push, pop, length, range, trim • Sets: add, remove, move, length, intersect union, diff, random • Other: save, lastsave can be used to force & verify disk persistence
  • 12. More: http://code.google.com/ p/redis/wiki/ RpoplpushCommand Atomic Operations • LPUSH / RPUSH: append to head/tail of list • LPOP / RPOP: return & remove first/last element of list • RPOPLPUSH: return & remove the last element of source list and push to the head of the destination list • GETSET: set a key to a new value and return the old value • MGET/MSET: get or set multiple keys to multiple values • SMOVE: move a value from one set to another • No way to group commands (transactions)
  • 13. More here: http://code.google.com/ p/redis/wiki/ ReplicationHowto Replication • Slaves can be used for scalability or redundancy • A master can have multiple slaves. • Slaves are able to accept other slave connections • Redis replication is non-blocking on the master, but blocking on the slave (can’t respond to queries during initial sync) • Slaves are able to automatically reconnect after an outage
  • 14. Blog post about Redis VM: http://antirez.com/ post/redis-virtual- memory-story.html Optional VM feature • Doesn’t use OS virtual memory • Still new, may not scale well in certain situations. • Don’t know how this will affect replication (initial sync / re-sync may be slow, but shouldn’t block the master) • All keys stay in memory. Minimum req’s: • 1 million keys ~ 160M • 10 million keys ~ 1.6G
  • 15. Syntax highlighted sample here: https://gist.github.com/ b16679cbb5b9573e078c Setup • git clone git://github.com/antirez/redis.git && cd redis • make • ./redis-server • make test • (review sample config file)
  • 16. Redis in Ruby • Redis library: “gem install redis” • All Redis commands can be called like methods on the connection object as described in the Command Reference
  • 17. Example Uses • A Memcached replacement (fast) • A work queue (sets & lists) • Fast auto-completion (persistent, queryable) • ORDER BY RAND() replacement (sets)
  • 18. Memcache Replacement • Why? Same as Memcached, but you get more “for free”: • Many databases from a single instance of Redis (instead of using namespaces) • Ability to easily backup/transfer state (dump.rdb) • Watch live commands on a running instance with the MONITOR command (instead of restarting with -v) • Opportunity to use Redis for other things once you’ve switched
  • 19. Memcache Replacement Note: namespace was replaced with a DB number • It’s pretty easy:
  • 20. More: Redis allows negative http:/ Work Queue /oxfordrepo.blogspot.com/ 2010/01/usage-stats-and-redis.html array indices to count backwards from the end. A VERY simple example • LPUSH and RPOP were made for this: 1 2 3 4 5 6
  • 21. “Not a ‘better DelayedJob’”. More: Compare them and see what is best for your project http://github.com/ defunkt/resque http://github.com/ blog/542-introducing- Work Queue - Resque • Created by Chris Wanstrath. Heavily inspired by DelayedJob. Currently used by GitHub. Provides: • A Ruby library for creating, querying, and processing jobs • A Rake task for starting a worker which processes jobs • A Sinatra app for monitoring queues, jobs, and workers.
  • 22. Fast Auto-Completion • Lives in memory, doesn’t expire, persists
  • 23. Replace MySQL ORDER More: BY RAND() http://nosql.mypopescu.com/post/295433623/redis- usecase-replacing-mysql-order-by-rand • ORDER BY RAND() is very slow (even with a LIMIT clause) • Duplicating the list of IDs as a Redis set is relatively cheap • SRANDMEMBER is fast
  • 24. Other Interesting Uses of Redis • Nginx HTTP Redis module for caching with Redis (ngx_http_redis) • LLOOGG.com: Realtime site usage statistics, use alongside Google analytics • EZMobius’ Fair Work Scheduler example • Sikwamic: Simple Redis-backed Comet server • RestMQ - A REST/JSON/HTTP based message queue built on Redis • redis-textsearch - A simple full-text search for multiple data stores
  • 25. Further reading • http://code.google.com/p/redis/wiki/CommandReference • http://antirez.com/post/redis-virtual-memory-story.html • http://www.slideshare.net/ezmobius/redis-remote-dictionary-server • http://www.paperplanes.de/2010/2/16/ a_collection_of_redis_use_cases.html • http://www.dorkalev.com/2010/02/sikwamic-simple-key-value-with- comet.html • http://github.com/blog/542-introducing-resque • http://nosql.mypopescu.com/post/295433623/redis-usecase-replacing- mysql-order-by-rand • http://github.com/nateware/redis-textsearch