Redis Labs manages over 160k+ HA databases, 10k clustered databases, without data loss in spite of one node failure a day and one data center outage per month. Using Enterprise
Redis(RLEC), Redis Labs delivers seamless zero downtime scaling, true high availability with persistence, cross-rack/zone/
datacenter replication and instant automatic failover. Learn how. Join this session for a deep dive into how enterprise Redis makes for no-hassle Redis deployments and the roadmap for new Redis capabilities. Discover new cost savings with Redis on Flash for cost-effective high performance operations and analytics
5. 5
Which In Turn Means:
Infinite Seamless Scaling
True High Availability
Top notch expert support
6. 6
Redis Labs Enhances OSS Redis
Redis Labs Node
Open Source
Zero latency proxy Cluster Manager
REST API
Odd number of
nodes needed to
handle network
splits- not three
copies of data
Redis Labs Cluster
• Shared nothing cluster
architecture
• Fully compatible with open
source commands & data
structures
Proprietary
7. 7
The Same Technology Runs Redis Cloud
Cluster
Management Path
Proxies
Node Watchdog
Cluster Watchdog
Node 1 Node 2 Node N (uneven number)…
Redis
Shards
Unique multi-tenant “Docker” like architecture enables running hundreds of databases over a single,
average cloud instance without performance degradation and with maximum security provisions
Data Path
Distributed Proxies
Single or Multiple Endpoints
50,000+ Customers
8. 8
Tremendous Customer Traction
Redis Cloud
Available since mid-2013
6000+ enterprise customers
Redis Labs Enterprise Cluster (RLEC)
Available since early-2015
100+ enterprise customers
9. 9
Always On - Highly
Available & Persistent
Simple, Seamless
Clustering. Linear
Scalability.
Enterprise-Class
Management and
Support
Enterprise-Class Redis – The Benefits
Stable & Predictable
Top Performance
Operational Cost
Savings
10. 10
Simple, Seamless
Scaling and Clustering
Auto- scaling/re-sharding/re-balancing
No downtime while scaling
Supports cross-shard operations
Simple, Seamless Clustering. Linear Scalability.
Linear Scalability
11. 11
Always On - Highly
Available & Persistent
Seamless cross
datacenter/region/cloud
replication
Instant auto-failover
Persistence, backups and DR
Always On - Highly-Available & Persistent
12. 12
Stable & Predictable
Top Performance
Consistent high performance
achieved under any load or
cluster size
Database processed by multiple
cores
Built-in performance
enhancement techniques
Stable & Predictable Top Performance
13. 13
Operational Cost Savings
OSS Redis Redis Labs
More efficient hardware utilization: fewer servers,
lower power & cooling and operational costs
Reduced manual labor through automation -
reduced time writing scripts, scaling,
configuration, monitoring, re-balancing and more
Run Redis on flash memory as RAM extender –
up to 10 times cheaper
Reduced downtime incidents
Shorten time to deploy Redis by over 50%
14. 14
Enterprise Management
and Support
UI, CLI, REST API -based
management & alerting
Proven technology supporting
thousands of customers
24x7 enterprise support,
top notch Redis expertise
Enterprise-Class Management & Support
15. 15
Redis Labs: Fastest Recovery, No Data Loss
%oftimesdatawaslost
Averagetimetorecoverinseconds
Redis Labs recovers in 5 seconds and does not lose data.
All other vendors lose data and take many minutes to recover
Vendors evaluated include
(not in order)
• Heroku Redis
• AWS ElastiCache
• Microsoft AzureCache
• Compose.io
16. 16
Redis Labs: The Only True HA Redis
16
Failure Event In-memory
Replication
Multi-DC/Zone
replication
Auto-failover AOF Data
Persistence
Backup (using
snapshots)
Multi-
region/Cloud
replication
Process failure Instant recovery* Slow recovery
Node failure Instant recovery* Slow recovery
Multi-node failure Instant recovery*
Network split Instant recovery*
Zone/Rack failure Instant recovery* Slow recovery Fast recovery
Region/Cloud failure Slow recovery Fast recovery
Typeofoutage
Essential features for high availability
*Auto-failover should run on same nodes as Redis deployment
Redis Labs provides all the essential HA features that protect against every type of outage
18. 18
Why Analyze Data In-Memory?
“Information is the oil of 21st century, and analytics is the combustion engine”- Peter
Sondergaard, Gartner Analyst
19. 19
Decision Speeds Are Accelerating..
“Big Data” gains
popularity as tools
become available to
harness it
Batch insights start to
drive business
Real time insights
automate decision-
making
2005 2012 - 2015 2016…
THE DATA REVOLUTION IS MATURING..
20. 20
The Race Is On..
INSIGHTS FROM YOUR DATA NEED TO BE
INSTANTANEOUS
COST EFFECTIVE
23. 23
Redis on Flash Concepts
• Flash used as a RAM extender (NOT as a persistent storage)
• Global key list in RAM; ‘hot’ values in RAM; ‘cold’ values on Flash.
• Multi-threaded & async Redis when accessing objects on Flash.
Utilizes multi-core and Flash concurrency architecture
• 100% compatibility with Redis
24. 24
How to Achieve Optimal Price/Performance
By dynamically setting RAM/Flash ratio
26. 26
A Real Life Example With Redis On Flash
Customer Scenario:
• Genome dataset
• Key sizes: 32B, value sizes : 5-12B
• No of keys: 250 x109… 250 x1012
Key1: AAAAAAAAAAAAAAAAAAAAAAAACCCCAAA Value1 = Freq=4 IE=A OE=A
Key2: AAAAAAAAAAAAAAAAAAAAAAAAACAACCC Value2 = Freq=7 IE=A,C,T,G OE=A,C,T,G
*IE – Inside End sequences
*OE – Outside End sequences
27. 27
Optimizing Redis Usage
RAW
• # of keys 250x10^9
• Key size = 32B
• Value size = 8B (average)
• Overhead per object (key+value) = ±61B (key) + 9 (value) = 70
• Internal fragmentation per object = 14B
• RAM size = ±31TB
28. 28
Further Optimizations
Encoding keys and values to
compress sizes
• # of keys 1.25x10^9
• Key size = 4B
• Value size = 3612B
• RAM overhead per object (key+value) = ±40B
• RAM size = ±55GB // for optimal performance we used 500GB to keep 10/90 RAM/Flash ratio
• Flash size = ±4.5TB
Using Redis Hashes to store
compressed keys/values (200
keys and values per hash)
1 2
29. 29
Memory Usage and Cost Comparison
Redis on RAM
Strings
Redis on Flash
Hashes
RAM size 31TB 0.5 TB
Flash size - 4.5TB
EC2 instances 155 x r3.8xlarge 2 x i2.8xlarge
1yr costs
(reserved
instances)
$2,017,325 $49,862
Savings with Flash
& Hashes %
97.6%
31. 31
Spark & Redis - Connector & Service Layer
Data Source
Serving Layer
Spark SQL &
Data Frame
RDD,
Data Source,
Data Set
RDD,
Data Source,
Data Set
Analytics & BI
32. 32
Spark & Redis – Internal Accelerator
Data Source
RDD,
Data Source,
Data Set
RDD,
Data Source,
Data Set
Spark SQL &
Data Frame
Analytics & BI
RDD,
Data Source,
Data Set,
Redis API
Data Sink
33. 33
Accelerate Spark Time-Series with Redis
Redis sorted sets accelerate time series data
processing by 100 times compared to other in-
memory K/V stores
Example time series data: Stock prices for 1024
stocks over 32 years
34. 34
Spark-Redis Package : The Results
Redis faster by upto 100 times compared to HDFS
and over 45 times compared to Tachyon or Spark
36. 36
36
Modules Extend Redis’ Use Case Coverage
MongoDB
Cassandra/
Datastax
Couchbase Redis (original) Redis + Modules
Single View Coming Soon
Personalization
Catalog Coming Soon
IoT
Real-Time Analytics
Content Management Coming Soon
Messaging
Fraud Detection Coming Soon
Graph Coming Soon
Time Series
Caching
Text Search Coming Soon
Image Processing Coming Soon
Machine Learning Coming Soon
Linear Algebra Coming Soon
Probabilistic data structures
for processing continuous,
Coming Soon
More
37. 37
Modules Turn Redis into a Multi-Model Database
37
MongoDB
Cassandra/
Datastax
Couchbase Redis (original) Redis + Modules
Document-Based Coming Soon
Column-Based Coming Soon
Key-Value Data Structures Data Structures
Graph Coming Soon
38. 38
Redis Module Hub
• A Redis Module Marketplace – for everyone
• Every problem a developer solves with Redis – now extended to the
Enterprise
• Will help developers monetize their work and reach enterprise
Redis users
• Will give enterprise Redis users the confidence and peace of mind
to easily deploy modules
www.redismodules.com
40. 40
40
3.15
2.40
21.00
8.70
24.57
10.61
0.00
5.00
10.00
15.00
20.00
25.00
30.00
Full text search Prefix search
Average Latency (msec)
RLEC Elasticsearch Solr
20,045
6,831
690
3,686
621
3,133
0
5,000
10,000
15,000
20,000
25,000
Full text search Prefix search
Ops/sec
RLEC Elasticsearch Solr
85% higher
32x higher
7.8x faster 4.1x faster
redisearch
The world fastest text search engine
41. 41
What Can Modules Do
41
• All modules are certified by Redis Labs for full compliance with OSS
Redis, Redis Cloud and Redis Labs Enterprise Cluster (RLEC)
Full Text Search Enhanced JSON Graph Operations Secondary Indexes
Linear Algebra SQL Support Image Processing
N-Dimension
Queries …
DRAM prices have been relatively stable over the years – and it continues to be expensive. Technologies such as Flash offer performance that is 3-4 orders of magnitude slower but 10 times cheaper. Emerging technologies such as Flash offer performance that is only an order of magnitude slower at 3 times lower cost. This makes for quite an attractive cost-performance tradeoff!
We extended Redis to take advantage of the multithreaded and asyn nature of Flash/other slower memory. Not only that, we added the capability to recognize “fast” and “slow” memory – with a configurable ratio so that all keys and hot values can be stored in the fast memory and cold values in slow memory such as Flash, 3 D Cross point or Storage Class memory for optimum performance.
NVMe – is easily x40 the throughput of SATA based Flash
We encoded and compressed the keys and values–keys were 31 bytes of string of 4 nucleobases. We encoded so that they could be represented with 2 bits per nucleobase – 62 bits and values were similarly compressed.
Flash works at 4KB blocks size, so hash sizes < 4KB.
Limited hashes to 200 entries to achieve the 4kb size per hash