Riak CS Build Your Own Cloud Storage by Pavan Venkatesh

1. Build Your Own Cloud Storage

2. About Me
Solutions Engineer
Pavan@basho.com

3. About Basho
Our Mission is to Be The Leader in Distributed Systems
• Founded January 2008
• 115+ employees
• Headquartered in Cambridge, with
regional offices in San Francisco,
Washington DC, London and Tokyo
• Makers of Riak & Riak CS
• Thousands of Users Worldwide including
over 20% of the Fortune 50
• 30,000+ downloads per month now up
from 19,500 in Dec 2011
• Strategic Partners include Citrix, IDC
Frontier, Yahoo! Japan, and Microsoft

4. Product Suite
Basho’s
distributed
systems
technologies
EnterpriseDS
Open
Source
Distributed
Database
Commercial
Distributed
Database
Distributed
Cloud
Storage
Pla7orm
• Always-­‐available,
scalable,
ops-­‐friendly
NoSQL
database
• Over
35,000
downloads
per
month
• Thousands
of
users
worldwide
• Version
1.0
unveiled
September
2011
• Adds
mulK-­‐data
center
replicaKon
&
Basho
24x7
support
• Enables
Secondary
Search
and
MapReduce
Clusters
• Requires
commercial
contract
and
secure
download
• Version
1.4
in
May
2013
• Expands
with
mulK-­‐
tenancy,
large
object
support,
metering
and
Amazon
S3
API
• Enables
public
and
private
cloud
storage
• SoXware
released
to
open
source
on
March
20th

5. Overview

6. Riak CS
is...
enterprise cloud storage
Riak
S3-compatibility
multi-tenancy
per user reporting
built
on top
of
large object storage

7. Enabling you to host your own
PUBLIC
PRIVATE
&
CLOUDS
or….
Reliable Storage Behind Apps

8. Riak CS Use Cases
Public Cloud
Storage
S3 Without AWS Cloud Drive
(General Content Storage)
Backup-as-a-
Service
Archival and
Preservation
Storage for
Internal
Clients / Depts.

9. Riak CS
Riak
Stanchion

10. Riak CS node
for every
node of Riak
1

11. How It
Works

12. Riak
Node
Riak
Node
Riak
Node
Riak
Node
Riak
Node
Large Object
Riak CS
S3
API
Reporting
API
Riak CS
S3
API
Reporting
API
Riak CS
S3
API
Reporting
API
Riak CS
S3
API
Reporting
API
Riak CS
S3
API
Reporting
API
1. User uploads an
object
1 MB
2. Riak CS
breaks object
into 1 MB chunks
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
1 MB
3. Riak CS
streams chunks
to Riak nodes
4. Riak replicates
and stores chunks

13. Under The
Hood

14. Riak
Riak is a distributed NoSQL key-value store
Store anything: text, images
More Flexible

15. Riak
Remains flexible in the long run, because of design

16. Riak
High availability
Low-latency
Horizontal scalability
Fault-tolerance
Ops friendliness

17. Riak
Masterless
• No master/slave or different roles
• All nodes are equal
• Write availability and scalability
• All nodes can accept/route requests

18. Riak
No Sharding
• Consistent hashing
• Prevents “hot spots”
• Lowers operational burden of scale
• Data rebalanced automatically

19. Riak
Availability and Fault-Tolerance
• Automatically replicates data
• Read and write data during
hardware failure and network
partition
• Hinted handoff

20. USERS
multi-tenancy:
Riak CS will track
individual usage/stats
access_key
secret_key
users identified by
users authenticated by

21. BUCKETS
users create buckets.
buckets are like folders.
store objects in buckets.
names are globally unique.

22. OBJECTS
stored in buckets.
store any file type.

23. Basic Operations
S3 API Commands
s3cmd mb s3://pavan-test
s3cmd ls s3://pavan-test
s3cmd put README.txt
s3://pavan-test s3cmd get s3://pavan-test/README.txt

24. Features

25. Riak CS
Large Object Support
• Started with 5GB / object
• Now have multipart upload
• Content agnostic

26. Riak CS
S3-Compatible API
• Use existing S3 libraries and tools
• RESTful operations
• Multipart upload
• S3-style ACLs for object/bucket
permissions
• S3 authentication scheme

27. Riak CS
Administration and Users
• Interface for user creation,
deletion, and credentials
• Configure so only admins can
create users

28. Riak CS
New Stuff in Riak 1.3
• Multipart upload: parts between
5MB and 5GB
• Restrict access to buckets based on
source IP

29. Multi-Datacenter Replication
• For active backups, availability
zones, disaster recovery, global
traffic
• Real-time or full-sync
• 24/7 support
• Per-node or storage-based pricing
“[deploying Riak CS] reduces the risk of using AWS and allows
customers to store their data in their own data centers, on their
own terms.” - Alex Williams, TechCrunch

30. Riak CS

31. Operations

32. OPERATIONAL STATS
exposed via HTTP resource: /riak-cs/stats
block
bucket
object
GET, PUT, DELETE
LIST KEYS, CREATE,
DELETE, GET/PUT ACL
GET, PUT, DELETE
HEAD, GET/PUT ACL
HISTOGRAMS COUNTERS

33. TRACK INDIVIDUAL USER’S
THE
“USAGE”
BUCKET
ACCESS
STORAGE

34. QUERY USAGE STATS
Storage and access statistics tracked
on per-user basis, as rollups for slices
of time
• Operations, Count, BytesIn,
BytesOut, + system and user error
• Objects, Bytes

35. Partnerships

36. Citrix CloudStack
• Combined platform for compute
and storage
• Summer ‘13: integrated
authentication support
• Videos on our blog

37. Roadmap

38. • Open stack integration
• API
• Keystone authentication
• Expand S3 API support
• Object versioning support
• Cloudstack integration work

39. QA
http://docs.basho.com/riakcs/latest/
http://docs.basho.com/riakcs/latest/riakcs-
tutorials/fast-track/
riak-users riak-users@lists.basho.com