Contenu connexe Plus de MapR Technologies (20) C APIs for Apache HBase by Aditya Kishore1. © 2014 MapR Technologies© 2014 MapR Technologies
C APIs for Apache HBase
Aditya Kishore (aditya@mapr.com)
HBase Meetup @ Ancestry.com
2. © 2014 MapR Technologies
About Me
• Apache HBase contributor since 2012.
• Software Engineer at MapR Technologies.
• Working on Apache HBase, M7 and Apache Drill.
• https://github.com/adityakishore/
3. © 2014 MapR Technologies
Agenda
• Introducing libHBase
• Design goals.
• API structures, functions and callbacks.
• Client for HBase 0.94.x
• Building libHBase and using in your application.
• A short example code using the APIs.
• Roadmap for the future.
• Demo (if time permits).
4. © 2014 MapR Technologies
Introducing libHBase
• Frequent requests from our customers (and on HBase mailing list).
• Long open JIRA HBASE-1015 (created: 21/Nov/08)
• “Mandatory for a database approaching 1.0 release” - Nick Dimiduk.
• Thrift / REST bindings are the only available solution.
• Need for stable, future ready, native APIs.
• “C” extern linkage enables support for other languages too.
• Work in progress to define standard C APIs and develop the client library.
• Join the conversation at HBASE-9835, HBASE-9977, HBASE-10168.
5. © 2014 MapR Technologies
• Parity with Java APIs.
• With native RPCs to HBase servers without any intermediate gateway.
• Uniform APIs across HBase releases to ensure binary compatibility.
• Thread-safe, non-blocking, asynchronous client.
• Synchronous wrappers available over the asynchronous client.
• No support for explicit batching in asynchronous APIs.
• Application managed buffers with destructors for library created resources.
Design Goals
6. © 2014 MapR Technologies
Structures, Functions and Callbacks
• The APIs use opaque structures to encapsulate implementation.
• Manipulated using create(), set_xyz(), and destroy() methods.
• Exception to this rule is hb_cell_t type which has static members.
• Asynchronous APIs all take a callback function as parameter.
• Invoking an async APIs always return success, error in callback.
• Callbacks can be triggered on any thread.
• Application are expected to not block in the callback routine.
7. © 2014 MapR Technologies
Available now: Client for HBase 0.94.x
• Download from MapR’s Github repository http://goo.gl/gs0uMs.
• Applied on top of HBase 0.94.17 code base.
• Patches available soon on JIRAs and Review Board.
• JNI implementation of the proposed APIs using AsyncHBase 1.5.
• Requires modified version of AsyncHBase http://goo.gl/aikZDn.
• Supports asynchronous Put, Get, Delete and Scan APIs.
• Synchronous Admin APIs wraps HBaseAdmin.
8. © 2014 MapR Technologies
Building libHBase and using in your Application
• Integrated with the HBase maven build using ‘libhbase’ profile.
• The HBase native library is built using Gradle.
• Header files in ‘/libhbase/include’, link with ‘-lhbase’.
• libhbase.so and libjvm.so need to be in LD_LIBRARY_PATH.
• Specify JVM classpath using ‘CLASSPATH’ or ‘HBASE_LIB_DIR’.
• Other JVM options can be specified using ‘LIBHBASE_OPTS’.
• Find the example code under ‘/libhbase/example/async’.
• A rudimentary performance test ‘/bin/perftest.sh’ is included.
9. © 2014 MapR Technologies
#include <hbase/hbase.h>
int main(int argc, char **argv) {
hb_connection_t connection = NULL;
hb_connection_create("localhost:2181",
"/hbase", &connection);
hb_client_t client = NULL;
hb_client_create(connection, &client);
hb_put_t put = NULL;
char *row_key = strdup("row_001");
size_t row_key_len = strlen(row_key);
hb_put_create((byte_t*) row_key,
row_key_len, &put);
hb_cell_t *cell = (hb_cell_t*)
calloc(1, sizeof(hb_cell_t));
cell->row = (byte_t*)row_key;
cell->row_len = row_key_len;
{...} // assemble the rest of the cell
hb_put_add_cell(put, cell);
hb_mutation_send(client, put,
mutation_callback, cell);
{...} // wait for callback to complete
hb_client_destroy(client, NULL, NULL);
return 0;
}
void mutation_callback(int32_t err,
hb_client_t client, hb_mutation_t mutation,
hb_result_t result, void *extra) {
hb_cell_t *cell = (hb_cell_t*) extra;
free(cell->row);
free(cell);
if (result) {
hb_result_destroy(result);
}
hb_mutation_destroy(mutation);
{...} // notify waiters
}
Example Code using the APIs
ZK Connection
Create the client
Prepare the Put
Assemble a Cell
Add Cell to Put
Submit to Client
Free the buffers
Free the Objects
HBase header
Free the Client
10. © 2014 MapR Technologies
Roadmap for Future
• Support for remaining APIs (increment, append, etc...).
• Performance test for Get / Scan / Mix workload.
• Support for Scan / Get Filters.
• Optimization of JNI / native call transitions.
• Support for secure RPCs.
• Native implementation for HBase 0.96+.
11. © 2014 MapR Technologies© 2014 MapR Technologies
DEMO
12. © 2014 MapR Technologies
Q & A
@mapr maprtech
aditya [at] mapr.com
Engage with us!
MapR
maprtech
mapr-technologies
13. © 2014 MapR Technologies© 2014 MapR Technologies
We are hiring!!!
http://www.mapr.com/company/careers