The Essentials of Digital Experience Monitoring_ A Comprehensive Guide.pdf
Highly scalable caching service on cloud - Redis
1. HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.com
Highly Scalable Caching Service on Cloud
Krishna Kumar & Irfan Rehman
Huawei India Cloud Platforms Team
2. HUAWEI TECHNOLOGIES CO., LTD. 2
We will discuss
Introduction
In-memory caching
Industry use cases
Types of cloud service provisioning
The problem to scale..
Resourse scheduler - Solution 1
Resourse scheduler - Solution 2
Demo
Q&A
3. HUAWEI TECHNOLOGIES CO., LTD. 3
Introduction to Cloud Caching Service
Cache
Component that stores data so future requests for that data can be served faster.
Distributed Cache
Extension of the traditional concept of cache used in a single locale.
A distributed cache may span multiple servers so that it can grow in size and in
transactional capacity. It is mainly used to store application data residing
in database and web session data. The distributed cache architecture is built on
scale-out, with inherent partitioning capabilities to spread the workload.
Cloud Service
Any resource that is provided over the Internet. The most common cloud service
resources are Software as a Service (SaaS), Platform as a Service (PaaS) and
Infrastructure as a Service (IaaS).
users
cache
4. HUAWEI TECHNOLOGIES CO., LTD. 4
• We focus on in-memory caches and exclude
others (ssd, multi-tier, etc) from our discussion
5. HUAWEI TECHNOLOGIES CO., LTD. 5
Major in-Memory Cache Uses Cases
Database offloading
in case of slow database or disk access
Web content caching
in case of high latency and narrow bandwidth to actual servers
Low latency data access
for time critical operations
Pub/Sub mechanisms
for multiple services and multiple consumers
Message queues
distributed network message passing mechanisms
CDN caches
in-memory caches especially for content index and ranking, not
necessarily for content itself
Caching and computation
sorted sets, lists, queues, etc
6. HUAWEI TECHNOLOGIES CO., LTD. 6
Leading Industry Cache Offerings
Redis
Memcached
Apache Hbase
CounchBase
Basho Riak
Oracle Coherence
DataStax
Cloudant
AeroSpike
See other players in the domain of caching:
https://www.g2crowd.com/products/redis/competitors/alternatives
8. HUAWEI TECHNOLOGIES CO., LTD. 8
Most popular in-memory cache in the world
Redis is an open source (BSD licensed), in-memory data structure store, used as database,
cache and message broker. It supports data structures such as strings, hashes, lists, sets,
sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius
queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and
different levels of on-disk persistence, and provides high availability via Redis Sentinel and
automatic partitioning with Redis Cluster.
Web
App
DB
Web
App
Web
App
Web
App
Volatile/temp data
Why Redis NoSQL?
No lock execution
Effectively uses individual core per
instance
Lot of data structure use cases
Blazingly fast for a single core
performer
10. HUAWEI TECHNOLOGIES CO., LTD. 10
Redis Service in cloud – major Vendors
1) Amazon - Elastic Cache: it easy to deploy, operate, and scale an in-memory cache in the cloud. It
support Redis Master/Slavce replication and Multi AZ - https://aws.amazon.com/elasticache/
2) Microsoft - gives you Redis reliable hosting and monitoring - https://azure.microsoft.com/en-
us/documentation/services/redis-cache/
3) Google – Deploy Redis on compute Engine. With Redis-as-a-Service, Redislabs will manage and
monitor your instance on Google Cloud Platform. https://cloud.google.com/solutions/redis/
4) IBM - Support Redis service on SoftLayer -
https://marketplace.ibmcloud.com/apps/87#!overview
5) RedHat OpenShift – Redis support for community-contributed cartridges -
https://blog.openshift.com/enabling-redis-for-your-app/
6) Digital Ocean – Supports using Redis in the cloud -
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis
7) Alibaba Aliyun – Aliyun Key-Value Store (KVStore for short) is an online Key-Value storage service
compatible with the open-source Redis protocol -
http://www.aliyun.com/product/kvstore/?lang=en
8) Redis Labs – Enterprise cluster; offered as a fully managed service, also used as third party
offering by other vendors.
11. HUAWEI TECHNOLOGIES CO., LTD. 11
One view of Redis service provisioning in cloud
Instance on Bare metal
- Toughest to maintain, but provides best performance
- A single instance wastes rest of the cpu resources (complicated
multi-tenancy)
Instance on Virtual Machine
- Lesser performance
- Scaling means provisioning another VM
Instance in a container
- Lesser performance, given the network translations
- Scaling is easy, but seamless is difficult if the redis cluster is non
uniform (server is different type, proxy is another, etc)
• Eventually all the above is resources.
• Service Instances could be placed anywhere in the pool.
• In fact different types could coexist, and communicate, if the
network allows so.
12. HUAWEI TECHNOLOGIES CO., LTD. 12
• The problem to scale given resources..
needs resource scheduling…
13. HUAWEI TECHNOLOGIES CO., LTD. 13
Redis service instance topologies
When a customer requests for a redis instance from cloud it could be one for the following configuration.
Single Redis-Server
•CPU = 1
•mem = variable
•Disk = optional
Master/Slave Redis
• Primarily for replication
• CPU= 1 + N Slaves
• mem = variable
• Disk = optional
master
Slave 1 Slave 2
Proxy
Redis Cluster with
Proxy
• Only Using twem proxy or
Codis for data shard
• Popular with older version
of redis eg:2.8
• CPU= Number of nodes
Redis 3.0 Cluster
•Latest offering from Redis ; No proxy required.
•Possible to horizontally scale ; CPU= Number of nodes
14. HUAWEI TECHNOLOGIES CO., LTD. 14
Proxy based sharding
Twitter handled redis cluster by implementing twemproxy
Shard aware clients
External Cluster manager
Wayfair an online furniture store managed redis cluster using zookeeper
Enterprise Cluster
Redis Labs Enterprise cluster - https://redislabs.com/redis-cloud
Redis 3.0 http://redis.io/download
How traditionally Industry handled Redis so far…
16. HUAWEI TECHNOLOGIES CO., LTD. 16
Mesos Kubernetes
Apache Mesos is a cluster manager that provides efficient
resource isolation and sharing across distributed
applications, or frameworks.
Apache Mesos abstracts CPU, memory, storage, and other
compute resources away from machines (physical or
virtual), enabling fault-tolerant and elastic distributed
systems.
Google Kubernetes brings 'Google style' cluster
management capabilities to the world of Docker
containers on the data center scenarios.
For production Google uses Omega/Borg, not Kubernetes
which is open sourced in June 2014.
• Mesos using same principle as Linux Kernel but higher
layer of abstraction.
• It can run Hadoop, MPI, Hypertable, Spark, and other
frameworks on a dynamically shared pool of nodes.
• Manage a cluster of Linux containers as a single system.
• There is a Kubernetes Framework for Mesos by
Mesosphere.
Was developed at the University of California, Berkeley
year 2009 developed by Benjamin Hindman, Andy
Konwinski, Matel Zaharia as well as professor Ion Stoica
Become top level Apache project from July 2013
Project Started at Google
Recently launched 1.0 production code (2nd half 2015)
Getting more popular day by day along with Docker.
Works with various Frameworks like Marathon, Cronos. You can
write your own. There is a core algorithm(DRF) to share
resources and you can define your own scheduling policy.
Frameworks does Fault Tolerance, Load balancing, Quotas,
Discovery, etc.
Kubernetes establishes robust declarative primitives for
maintaining the desired state requested by the user. Self-
healing mechanisms, such as auto-restarting, re-scheduling,
and replicating containers require active controllers, not just
imperative orchestration.
Introduction
18. HUAWEI TECHNOLOGIES CO., LTD. 18
Solution 1 : Kubernetes (Cluster manager)
We could provide redis instance in docker containers
Kubernetes has a Mesos framework which can co-exist with
other frameworks/workloads in the same Mesos cluster
The concept of ‘POD’ can be used to club multiple containers
together. Kubernetes ensures that these containers are started
on the same slave
Kubernetes automatically provides fault- tolerance
Why kubernetes?
19. HUAWEI TECHNOLOGIES CO., LTD. 19
Mesos
Master
Kubernetes
Framework for
Mesos
Cloud service
broker Kube- Executor kube- Executor Kube- Executor Kube- Executor
Redis
Request
from cloud
Each Service instance will be a POD, provisioned
for given type
Solution 1 : Kubernetes as fw of mesos
20. HUAWEI TECHNOLOGIES CO., LTD. 20Page 20
Cannot add additional containers in a already running POD. Very
necessary for Redis 3.0 horizontal scalability. Below discussion in
google groups explains the implementation difficulties in
Kubernetes https://groups.google.com/forum/#!topic/google-
containers/_Gz9dH2hqg4
However it’s a popular demand from the kubernetes community
Limited Flexibility by using Kubernetes Framework
Solution 1 : Kubernetes (Cluster manager) Contd..
Limitations..
21. HUAWEI TECHNOLOGIES CO., LTD. 21Page 21
Implement a simplistic Mesos framework for Long Running
Process
Use “mesos-go” an opensource framework library written in
golang for development
Check feasibility for vertical scaling of redis-instances
Collect statistics against each redis instance
Enable the fw to ensure instances can communicate to create a
cluster (user 3.0)
Solution 2: Mesos Framework dealing directly with Redis
Instances
Scope
22. HUAWEI TECHNOLOGIES CO., LTD. 22
New Redis Mesos Framework
Mesos
Master
Redis
Framework for
Mesos
Cloud service
broker Redis Executor Redis Executor Redis Executor Redis Executor
Redis
Request
from cloud
24. HUAWEI TECHNOLOGIES CO., LTD. 24Page 24
To build a Mesos framework we have to develop two components.
Scheduler – Talk to the mesos master and submit tasks
Executor – Execute your tasks at the slave
By Default Mesos considers each work load is a task not LRP (Long Running Process)
What is a Mesos framework (a developers perspective):
Usual Logic of Scheduler Usual Logic of Executor
Loop (Wait for offer from Mesos Master)
if (offer.Resource > Task.Resource) then
Select Offer
else
Reject Offer
Repeat
Loop (wait for tasks to Launch )
Update task.State = STARTED
execute task
Update task.State = Finished / Error
Repeat
Solution 2: Mesos Framework dealing directly with Redis
Instances
25. HUAWEI TECHNOLOGIES CO., LTD. 25Page 25
Its easier to write a Mesos framework
We have better controller & flexibility over the running redis
instances
We can collect better statistics on the running tasks
We can plan migration of running tasks among different slaves
We can group bunch of running processes and allocate them in a
single slave (like kubernetes POD)
We would be able to offer Redis 3.0 and scale Horizontally
We could use any container technology of our choice like Docker
or RKT or RunC in bare-metal servers for additional instance
security
WHY and what more can we do?
Solution 2: Mesos Framework dealing directly with Redis
Instances
28. HUAWEI TECHNOLOGIES CO., LTD. 28
Astro Project – Spark SQL on Hbase
Astro is fully distributed SQL on Hbase solution based on Spark Ecosystem
100% Open source
Any contributions are welcome.
HBase
Spark
Astro
Spark Application
SQL ML Graph Stream
Community
Package
•Astro query processing is fully distributed.
•CRUD(Create, Read, Update and Delete)
operations support
•Fully utilizing Spark SQL query optimizer.
•Advanced optimizations to push down
computations close to storage.
•Co-Location along with Hbase.
As Astro plugs into spark, Astro query
results can be seamlessly used for
further in-depth analysis using Spark SQL
DataFrame, MLLib, GraphX etc…
Address:
https://github.com/HuaweiBigData/astro
Compared to existing SQL on Hbase solution the major Advantages are: