Presentation held in the seminar on "NotOnlySQL databases." Builds upon slides in the former presentation "Redis — memcached on steroids." Features challenges when deploying Redis on Amazon Web Services within the context of the Yahoo! Cloud Serving Benchmark.

1. “
Redis is
more than just
a key-value store,
it’s a lifestyle.
–– Mathias Meyer

2. Fundamental data structures
for a fundamental API
a data structure server
lists sets
sorted hash
sets tables

3. user1234567890
field0: abc…
field1: def…
field2: ghi…
field3: jkl…
field4: mno…
field5: pqr…
field6: stu…
field7: vwx…
field8: yzä…
field9: öüß…

4. user1234567890
field0: abc…
hash
field1: def…
field2: ghi…
field3: jkl…
RedisClient.java
tables
field4: mno…
field5: pqr…
field6: stu…
field7: vwx…
field8: yzä…
field9: öüß…
insert HMSET
read HMGET
update HMSET
delete DEL

5. Fundamental data structures
for a fundamental API
a data structure server
lists sets
sorted hash
sets tables

6. factory settings hand-tuned
0.5M initial size
2.54MB 1.62MB
hash-max-zipmap-value 2048

8. The Harry Potter Approach
WWVD
What Would Voldemort Do
101 @Override
102 public int scan(…) {
104 logger.warn(
⁞ "Voldemort does not support Scan semantics");
105 return OK;
106 }
db/voldemort/src/com/yahoo/ycsb/db/VoldemortClient.java

9. sorted
sets

10. user1 user2 user3 user5 user8
1 2 3 5 8…

11. user1 user2 user3 user5 user8
not that easy

12. record name can be anything
thus: position = hash(name)
user1 user2 user3 user5 user8
not that easy

13. sorted
sets
50% performance hit

15. workload A
throughput
target

16. workload E
throughput
target

17. workload F
throughput
target

18. “
I think the… problem is…
believing that
there can be
“one true datastore”.
Different technologies
excel at different things.
–– Weixi Yen

19. “
I think the… problem is…
believing that
there can be
“one true datastore”.
Different technologies
excel at different things.
–– Weixi Yen

20. Memory storage is #1
Redis key space
http://spotfireblog.tibco.com/wp-content/uploads/in-memory-analytics.jpg

21. network bandwidth contention
memory

22. 200 ops/sec
single-threaded

23. user1234567890
field0: abc…
field1: def…
field2: ghi…
field3: jkl…
field4: mno…
field5: pqr…
field6: stu…
= 1 KB
field7: vwx…
field8: yzä…
field9: öüß…

24. user1234567890
field0: abc…
field1: def…
field2: ghi…
field3: jkl…
field4: mno…
field5: pqr…
field6: stu…
= 1 KB
factory page size := 32 bytes
field7: vwx…
field8: yzä…
field9: öüß…

25. user1234567890
field0: abc…
field1: def…
field2: ghi…
field3: jkl…
field4: mno…
field5: pqr…
field6: stu…
= 1 KB
factory page size := 32 bytes
field7: vwx…
field8: yzä…
field9: öüß…
values must be swapped in their entirety,
even for complex types

26. user1234567890
field0: abc…
field1: def…
field2: ghi…
field3: jkl…
field4: mno…
field5: pqr…
field6: stu…
= 1 KB
factory page size := 32 bytes
field7: vwx…
field8: yzä…
field9: öüß…
values must be swapped in their entirety,
even for complex types
= 32 pages 

27. user1234567890
field0: abc…
field1: def…
field2: ghi…
field3: jkl…
field4: mno…
field5: pqr…
field6: stu…
= 1 KB
factory page size := 32 bytes
field7: vwx…
field8: yzä…
field9: öüß…
values must be swapped in their entirety,
even for complex types
= 32 pages 
vm-page-size et al

28. user1234567890
field0: abc…
field1: def…
field2: ghi…
field3: jkl…
field4: mno…
field5: pqr…
field6: stu…
= 1 KB
field7: vwx…
field8: yzä…
50% performance hit
field9: öüß…
= 32 pages 

29. 32-bit 64-bit
4 GB 8 TB
large pointers

30. redis-cli INFO

31. Memory storage is #1
meh, persistence
memory
overhead
http://www.flickr.com/photos/generationbass/4827013488/

32. Memory storage is #1
meh, persistence
load phase fails randomly memory
overhead
http://www.flickr.com/photos/generationbass/4827013488/

33. there are no silver bullets