The speed at which you can extract insights from your data is increasingly a competitive edge for your business. Data and analytics have to be at lightning fast speeds to seriously impact your user acquisition.
Join this webinar featuring Forrester analyst Noel Yuhanna and Leena Joshi, VP Product Marketing at Redis Labs to learn how you can glean insights faster with new open source data processing frameworks like Spark and Redis.
In this webinar you will learn:
* Why analytics has to run at the real time speed of business
* How this can be achieved with next generation Big Data tools
* How data structures can optimize your hybrid transaction-analytics processing scenarios
Boost PC performance: How more available memory can improve productivity
Running Analytics at the Speed of Your Business
1. Home of Redis
Analytics at the Speed of
Business with Redis and Spark
Leena Joshi
VP Product Marketing
Noel Yuhanna
Principal Analyst, Forrester
2. 2
Agenda
• Why Data & Analytics Need to be Real
Time
• Drivers and Challenges for Real time
analytics
• The Roadmap to Fast Data
• Recommendations
• Brief Introduction to Redis
• Analytics with Redis
• Redis –Spark Integration
• Making Analytics Cost Effective
• Extended analytics with Redis Modules
Noel Yuhanna – 20 min Leena Joshi – 20 min
3. Running Analytics At The Speed Of
Your Business
Noel Yuhanna, Principal Analyst
RedisLabs Webinar
6. Today business users think of analytics as a set of boring reports
and dashboards … they don’t want yesterdays data tomorrow!
of enterprise data
in used for analytics….
12%
17. 17
Who We Are
The open source home and commercial provider
of Redis
Open source. The leading in-memory data
structure store, supporting any high
performance operational or analytic use case.
18. 18
Redis is a Game Changer
Simplicity
(through Data Structures)
Extensibility
(through Redis Modules)
Performance
ListsSorted Sets
Hashes Hyperlog-logs
Geospatial
Indexes
Bitmaps
SetsStrings
Bit field
19. 19
• Used by developers like “Lego” blocks
• Enables data to be processed on the database level
rather than the application level
• Turns complex functionality into a single command
such as:
"Get the e-mail address of the user with the highest score in a game that
started on July 24th at 11:00pm PST”
ZREVRANGE 07242015_2300 0 0
Simplicity: Data Structures - Redis’ Building Blocks
ListsSorted Sets
Hashes
Hyperlog-
logs
Geospatial
IndexesBitmaps
SetsStrings
• Enable solving complex problems by creating relations between
data structures, using standard or custom (Lua) commands
• The result: cleaner, more elegant code, faster execution time
20. 20
Extensibility: Modules Extend Redis Infinitely
• Add-ons using a Redis API for seamlessly adding to
it use cases and data structures
• Modules enjoy Redis’ simplicity, super high
performance, infinite scalability and high availability
• Modules can be created by anyone. Certified by Redis Labs.
Full Text Search Enhanced JSON Graph Operations Secondary Indexes
Linear Algebra SQL Support Image Processing
N-Dimension
Queries …
21. 21
Performance: the Most Powerful Database
Highest Throughput at Lowest Latency
in High Volume of Writes Scenario
Lowest number of servers needed to
deliver 1 Million writes/second
300
50 50
2
0
50
100
150
200
250
300
350
Benchmarks performed by Avalon Consulting Group Benchmarks published in the Google blog
22. 22
Redis Cloud
Available since mid-2013
6,100+ enterprise customers
Redis Labs Enterprise Cluster (RLEC)
Available since early-2015
100+ enterprise customers
Wide Adoption
24. 24
Popular Redis Use Cases
Geo SearchData Ingestion Social Functionality
Following, Followers, Relations Location-based ApplicationsHigh Throughput Buffering
Job & Queue Caching
Any Business Application Any Web or Mobile App
High Speed Transactions Time-Series
Business Applications
Analytics
Real-time Computations Time-Based Analysis
25. 25
Example : Redis For Bid Management
The Application Problem
• Many users bidding on items
• Need to instantly show who’s
leading, in what order and by how
much
• May also need to display analytics
like how many users are bidding in
what range
• Disk-based DBMS-es are too slow for
real-time, high scale calculations
Why Redis Rocks This
• Sorted sets automatically keep list of
users and scores updated and in
order (ZADD)
• ZRANGE, ZREVRANGE will get your
top users
• ZRANK will get any users rank
instantaneously
• ZCOUNT will return a count of users
in a range,
• ZRANGEBYSCORE will return all the
users in a range by their bids
27. 27
Example : Redis For Recommendations
The Application Problem
• Users, items, likes, dislikes, similarities
• Set comparisons of user likes, user
dislikes should help create similarity
scores, which can then be stored in a
sorted set
• Set comparisons of similar user
likes/dislikes with items not purchased
by current user should yield suggestions
• High speed and low latency
requirements
Why Redis Rocks This
• Redis Sets are unordered collections
of strings- SADD to add objects to
each tag
• Set operations executed in –
memory, blazing fast speeds
• SINTER, SINTERSTORE to intersect
multiple sets
• SUNIONSTORE to add multiple sets
• SISMEMBER to determine membership,
SMEMBERS to retrieve all values
• Sets and Sorted sets combined are a
great choice for recommendation
engines
31. 31
Accelerate Spark Time-Series with Redis
Redis sorted sets accelerate time series data
processing by 100 times compared to other in-
memory K/V stores
Example time series data: Stock prices for 1024
stocks over 32 years
32. 32
Accelerating Spark Time-Series with Redis
Redis is faster by upto 100 times compared to HDFS
and over 45 times compared to Tachyon or Spark
33. 33
More Details About the Redis & Spark Integration
Github link: Spark-Redis Connector Package
https://github.com/RedisLabs/spark-redis
How to get started with Spark and Redis:
https://redislabs.com/solutions/spark-and-redis
Blog: https://redislabs.com/blog/connecting-spark-
and-redis
39. 39
Customer Example : Redis on Flash
• Genome dataset: 31TBs of raw data
• Optimized data set through encoding
and using Redis Hashes
• Resulting data runs high speed
analyses with 55GB of RAM
and 4.5TB of Flash
• 97% annual savings compared to a
pure RAM solution
Redis on RAM Redis on Flash
RAM Size 5TB 0.5TB
Flash size N/A 4.5TB
Servers
on AWS :
21x r3.8xlarge
on P8:
2x s822 LC
1yr costs $489,333 $15,677
P8 savings 97%
41. 41
What Can Modules Do
41
• All modules are certified by Redis Labs for full compliance with OSS
Redis, Redis Cloud and Redis Labs Enterprise Cluster (RLEC)
Full Text Search Enhanced JSON Graph Operations Secondary Indexes
Linear Algebra SQL Support Image Processing
N-Dimension
Queries …
42. 42
42
3.15
2.40
21.00
8.70
24.57
10.61
0.00
5.00
10.00
15.00
20.00
25.00
30.00
Full text search Prefix search
Average Latency (msec)
RLEC Elasticsearch Solr
20,045
6,831
690
3,686
621
3,133
0
5,000
10,000
15,000
20,000
25,000
Full text search Prefix search
Ops/sec
RLEC Elasticsearch Solr
85% higher
32x higher
7.8x faster 4.1x faster
redisearch
The world fastest text search engine