Ceph Day Portland Jessie Williamson, SUSE http://ceph.com/cephdays/ceph-day-portland/

1. librados
Adventure! Excitement! Octopus-wrangling!
Jesse Williamson, SUSE (“jwilliamson@suse.de”)

2. 2
Disclaimer:
The views expressed here are my own and do not necessarily reflect those of my
employer, SUSE, nor of the Ceph authors and contributors.
(Whew!)

3. 3
● RADOS: What the Heck Is It?
...in which I try to explain a programming
tool while showing very little code.

4. 4
● RADOS: What the Heck Is It?
● Reliable, Autonomic Distributed Object Store
“RADOS: A Scalable Reliable Storage Service for Petabyte-scale Storage Clusters”. Weil, Leung, et. al., 2007.
● “...[provides] applications with the illusion of a single logical object store with well-defined safety semantics and
strong consistency guarantees.”
● Distributes data and workload across dynamic storage cluster
● “...[leverages] device intelligence to distribute the complexity surrounding consistent data access,
redundant storage, failure detection, and failure recovery in clusters consisting of many thousands of storage
devices.”

5. 5
● RADOS: What the Heck Is It?
● Reliable, Autonomic Distributed Object Store
“RADOS: A Scalable Reliable Storage Service for Petabyte-scale Storage Clusters”. Weil, Leung, et. al., 2007.
● “...[provides] applications with the illusion of a single logical object store with well-defined safety semantics and
● strong consistency guarantees.”
● Distributes data and workload across dynamic storage cluster
● “...[leverages] device intelligence to distribute the complexity surrounding consistent data access,
redundant storage, failure detection, and failure recovery in clusters consisting of many thousands of storage
devices.”
● Integral to Ceph!
● cluster management, device state, etc. (CRUSH maps)
● data distribution (“placement”) policies
● “smart” propagation of rules; data replication, consistency, etc.

6. 6
● RADOS: What the Heck Is It?
● RADOS “the idea”
● papers
● RADOS implementation “the thing”:
● Ceph:
- mons, OSDs, storage “smarts”
● Interfaces to the implementation:
● Command-line interface:
- the “rados” tool
● Service interface:
- RGW (RADOS Gateway); filesystems, ...

7. 7
● RADOS: What the Heck Is It?
● RADOS “the idea”
● papers
● RADOS implementation “the thing”:
● Ceph:
- mons, OSDs, storage “smarts”
● Interfaces to the implementation:
● Command-line interface:
- the “rados” tool
● Service interface:
- RGW (RADOS Gateway); filesystems, ...
● Programmatic interface:
- librados

8. 8
● librados:
● What is it, and what does it do?
● What uses it?
● What can you make with it?

9. 9
● librados:
● What is it?
Librados is a programming-language level interface to Ceph.
Bindings available at-least for (varying levels of “official” standing):
C++, C, Erlang, Go, PHP, Java, Python, Ruby, Haskell

10. 10
● librados:
● What is it?
Librados is a programming-language level interface to Ceph.
Bindings available at-least for (varying levels of “official” standing):
C++, C, Erlang, Go, PHP, Java, Python, Ruby, Haskell
● Lower (librados.h) and higher-level (librados.hpp, RadosClient)
behaviors

11. 11
● librados:
● What is it?
Librados is a programming-language level interface to Ceph.
Bindings available at-least for (varying levels of “official” standing):
C++, C, Erlang, Go, PHP, Java, Python, Ruby, Haskell
● Lower (librados.h) and higher-level (librados.hpp, RadosClient)
behaviors
● ...librados is “batteries included”-- just link in!

12. 12
● librados:
● What is it?
Librados is a programming-language level interface to Ceph.
Bindings available at-least for (varying levels of “official” standing):
C++, C, Erlang, Go, PHP, Java, Python, Ruby, Haskell
● Lower (librados.h) and higher-level (librados.hpp, RadosClient)
interfaces
● ...librados is “batteries included”-- just link in!!
● librados is a platform you can build your application on top of--
a client interface to Ceph.

13. 13
● librados:
● What is it?
Librados is a programming-language level interface to Ceph.
Bindings available at-least for (varying levels of “official” standing):
C++, C, Erlang, Go, PHP, Java, Python, Ruby, Haskell
● Lower (librados.h) and higher-level (librados.hpp, RadosClient)
interfaces
● ...librados is “batteries included”-- just link in!
● librados is a platform you can build your application on top of--
a client interface to Ceph
● Hides complexity: networking, replication, scaling, dealing
with filesystems...

14. 14
● librados:
● What is it?
librados (C++, C, Python, ...)
RADOS (Ceph infrastructure-- distributed object store)
RGW
(Web services)
CephFS
(filesystem)
Your application! :-)

15. 15
● librados:
● What is it?
import rados, sys
cluster = rados.Rados(conffile='ceph.conf')
cluster.connect()
print "Cluster ID: " + cluster.get_fsid()
cluster_stats = cluster.get_cluster_stats()
for key, value in cluster_stats.iteritems():
print key, value
ioctx = cluster.open_ioctx('data')
ioctx.write_full("hi_there", "Hello World!")
ioctx.close()

16. 16
● librados:
● What uses it? What can you make with it?

17. 17
● librados:
● What uses it? What can you make with it?
Ceph components and tools:
RADOS gateway (rgw);
“rados” command-line tool;
rdb-mirror
libradosstriper
librdb
...among others...

18. 18
● librados:
● What uses it? What can you make with it?
Ceph components and tools:
RADOS gateway (rgw);
“rados” command-line tool;
rdb-mirror
libradosstriper
librdb
...among others…
Potential project using librados to provide a backend for the Dovecot mail system:
http://permalink.gmane.org/gmane.comp.file-systems.ceph.devel/24092

19. 19
● librados:
● What uses it? What can you make with it?
Ceph components and tools:
RADOS gateway (rgw);
“rados” command-line tool;
rdb-mirror
libradosstriper
librdb
...among others…
Potential project using librados to provide a backend for the Dovecot mail system:
http://permalink.gmane.org/gmane.comp.file-systems.ceph.devel/24092
RadosFS: A Ceph RADOS API for Hadoop (“https://github.com/rootfs/RadosFs”)
Zlog: a distributed shared log for Ceph (“https://github.com/noahdesu/zlog”)
Apache and nginx modules
Docker and Vagrant goodies
...many more examples on github.

20. 20
● librados:
● What uses it? What can you make with it?
Ceph components and tools:
RADOS gateway (rgw);
“rados” command-line tool;
rdb-mirror
libradosstriper
librdb
...among others…
Potential project using librados to provide a backend for the Dovecot mail system:
http://permalink.gmane.org/gmane.comp.file-systems.ceph.devel/24092
Librados is a powerful Ceph interface-- build
anything you can imagine! :-)

21. 21
● librados:
● What does it do?
Some major functional areas:
Configuration: handle environment variables, config files, command-line options in
a consistent way;

22. 22
● librados:
● What does it do?
Some major functional areas:
Configuration: handle environment variables, config files, command-line options in
a consistent way;
Connections: abstract away networking, connection management, some
threading;
Pools: manage namespaces, iterate over objects in them

23. 23
● librados:
● What does it do?
Some major functional areas:
Configuration: handle environment variables, config files, command-line options in
a consistent way;
Connections: abstract away networking, connection management, some
threading;
Pools: manage namespaces, iterate over objects in them
I/O: Synchronous and Asynchronous read/write/update on objects

24. 24
● librados:
● What does it do?
Some major functional areas:
Configuration: handle environment variables, config files, command-line options in
a consistent way;
Connections: abstract away networking, connection management, some
threading;
Pools: manage namespaces, iterate over objects in them
I/O: Synchronous and Asynchronous read/write/update on objects
More:
Extended attributes, omaps, client watch/notify events, osd commands, statistics,
snapshots, ...all “baked in”.

25. 25
● librados:
Resources:
Ceph documentation (C, C++, Python, Java examples):
http://docs.ceph.com/docs/master/rados/api/
Other Bindings (Erlang, Go, Haskell, PHP, Ruby-- many more (Common Lisp, anyone?)):
https://github.com/renzhi/erlrados
https://github.com/mrkvm/rados.go
https://hackage.haskell.org/package/rados-haskell
https://github.com/ceph/ceph-ruby
https://github.com/ceph/phprados
Source code:
https://github.com/ceph/ceph/tree/master/src/include/rados
A more in-depth presentation from Sage Weil:
http://events.linuxfoundation.org/sites/events/files/slides/20150311%20vault15%20librados.pdf
Further reading:
Weil, Leung, et. al.. “RADOS: A Scalable, Reliable Storage Service for Petabyte-scale Storage Clusters”.
Weil, Brandt, et. al.. “CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data”.

26. Questions? :-)