A quick presentation not so much on how spectacular the Redis performance is, but how quickly you can get stuff done and how I did it. The presentation as I gave at the Dutch NoSQL Meetup on Wednesday April 11. http://www.meetup.com/nosql-nl/events/55488462/

1. Redis
Erwin van der Koogh - Erronis

2. Warning!
This is not about how blazingly fast Redisis
Or how many keys it can store
This is for a prototype. Our load is less than
a request/min and we have < 100.000 keys
This is to show how easy it is to use Redis in
development to knock something up.

3. What is Redis?
Simple In-memory key-value store
With a twist
Complex data structures
Strong persistence
Powerful control flows
Extremely fast

4. Problem
API for Electronic Programming Guide
Complete dump of upcoming 7 days every
hour
Build 10 days of history
Flexible query options begin= & end=
Hundreds of channels

5. First solution - Import
Read in dump, convert to JSON
Store channelId in Set channels
Make a SortedSet for channel
Store JSON with date/time as score

6. First solution - Query
Get all channelIds for Set channels
Get all items between begin and end from
SortedSet
return

7. Problem 2
That was fine for the first import, but the
import 1 hour later had to overwrite most of
the data

8. Second solution
Add new entries in a temp SortedSet
Take date/time from first entry in SortedSet
Copy current SortedSet to merge SortedSet
Remove items in merge Set newer than first
new entry
Merge merge and tmp SortedSet into new
SortedSet

9. Problem 3
We needed to update some of the data
Updating complex entries (JSON) in a
(Sorted)Set is error-prone

10. Third Solution - Import
Instead of saving JSON in SortedSet save
them in a hash
Put only the key of the hash in the
SortedSet

11. Third Solution - Query
Get all channelIds from channel Set
Get all Ids for entries between begin & end
Get all information from hash for every id
Start streaming every item in as they are
coming in

12. Lessons Learned
In-memory makes a difference, lots of
queries are ok
Learn to love single-threaded
Use the complex data-structure for easy
querying (List, Set, SortedSet)
MONITOR is your friend