MongoDB is the trusted document store we turn to when we have tough data store problems to solve. For this talk we are going to go a little bit off the path and explore what other roles we can fit MongoDB into. Others have discussed how to turn MongoDB’s capped collections into a publish/subscribe server. We stretch that a little further and turn MongoDB into a full fledged broker with both publish/subscribe and queue semantics, and a the ability to mix them. We will provide code and a running demo of the queue producers and consumers. Next we will turn to coordination services: We will explore the fundamental features and show how to implement them using MongoDB as the storage engine. Again we will show the code and demo the coordination of multiple applications.

1. Fun Teaching MongoDB
New Tricks
Robert J. Moore
President
Allanbank Consulting, Inc.
Robert.J.Moore@allanbank.com

2. Agenda
 Inspiration
 A Little Fun: MongoDB as
 ... a Coordination Service?
 ... a Message Broker?
 https://github.com/allanbank/mongodb-tricks

3. Disclaimer
 Don't try these at home
 They are hacks in the best sense
 Professional developer on an air gap
laptop
 Enough! On with the fun

4. Inspiration - Topics
 Publish / Subscribe
 One writer, many readers
 MongoDB Capped Collection
 Tailable Cursor
 Can also have selectors

5. Inspiration - Topics
 Create a Capped Collection
 Acts like a ring buffer
 Tailable Cursor returns documents as
they are added in the future
...
Cursor
Document

6. Asynchronous Java Driver
 Website
 http://www.allanbank.com/mongodb-async-driver
 Focus on performance and usability
 Lower latency, higher throughput even under
heavy thread contention
 Benefit when using the synchronous
interface
 Greater benefit using the asynchronous
interface

7. Asynchronous Java Driver
Performance
http://www.allanbank.com/mongodb-async-driver/performance/ycsb.html

8. Coordination Service
 ZooKeeper
 Notification that something of interest
changed
 Watches
 Track entities in a group
 Group Management

9. Watches
 Replica Set Oplog
 All Insert, Update, Delete operations
 Capped – Tailable cursors again
 Want to convert from the Oplog
 Different fields contain the _id of the
document

10. Oplog Documents
 Insert
{ "ts" : { "t" : 1362958492000, "i" : 1 },
"h" : NumberLong("5915409566571821368"), "v" : 2,
"op" : "i",
"ns" : "test.test",
"o" : { "_id" : "513d189c8544eb2b5e000001" } }
 Delete
{ ... "op" : "d", ..., "b" : true,
"o" : { "_id" : "513d189c8544eb2b5e000001" } }
 Update
{ ... "op" : "u", ...,
"o2" : { "_id" : "513d189c8544eb2b5e000001" },
"o" : { "$set" : { "i" : 1 } } }


11. Initialize a Watcher - Code
final DocumentAssignable wantQuery =
or(
where("ns").equals(ns)
.and("op").in(constant("i"), constant("d"))
.and("o._id").matches(myContext),
where("ns").equals(ns)
.and("op").equals("u")
.and("o2._id").matches(myContext));
final Find.Builder builder = new Find.Builder();
if (myLastTs != null) {
final DocumentBuilder tsQuery = BuilderFactory.start();
tsQuery.push(myLastTs.getName())
.add(myLastTs.withName(ComparisonOperator.GT.getToken()));
builder.setQuery(and(tsQuery, wantQuery));
}
else {
builder.setQuery(wantQuery);
}
builder.tailable();
MongoDatabase localDb = myMongoClient.getDatabase("local");
MongoCollection oplog = localDb.getCollection("oplog.rs");
myControls = oplog.streamingFind(new OpLogNotification(), builder.build());

12. Watcher - Code
Watcher watcher = new Watcher(client, collection,
Pattern.compile(".*"), new WatchListener() {
@Override
public void changed(Operation op, String context,
Document document) {
if (op == Operation.DELETE) {
System.out.println(context + ": " + op);
}
else {
System.out.println(context + ": " + op + ": "
+ document);
}
}
});
watcher.start();

13. Watcher Demo

14. Group Management
 Notification of:
 Adding a Group Member
 Removing a Group Member
 Or Group Member Disappears
 Combination of:
 Watcher
 Scheduled Task for Heartbeat
 Expire old members

15. Group Manager Client - Code
GroupManager manager = new GroupManager(executor, client, collection, rootContext);
manager.addListener(new GroupListener() {
@Override
public void memberRemoved(String context) {
System.out.println(context + " - Removed");
}
@Override
public void memberAdded(String context) {
System.out.println(context + " - Added");
}
});
manager.start();
final GroupMember member = manager.addMember();
// Faster cleanup, if we can.
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
member.remove();
}
});

16. Group Demo

17. Queues
 Divide and Conquer
 One writer, one reader
 MongoDB Capped Collection
 Want to “share” a Cursor

18. Restartable Cursors
 What if we had multiple applications passing the
same “getmore” requests?
 “getmore” message references a cursor
struct {
MsgHeader header; // standard message header
int32 ZERO; // 0 - reserved for future use
cstring fullCollectionName; // "dbname.collectionname"
int32 numberToReturn; // number of documents to return
int64 cursorID; // cursorID from the OP_REPLY
}
http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/#op-get-more
 We need a way to save and restore the cursor
 Lets call them Restartable Cursors, but they don't
exist, yet

19. Queues
 Create a Capped Collection
 Initialize a Tailable Cursor
 Share with multiple processes
...
Cursor
Application 1 Application 2 Application 3 Document

20. Restartable Cursors
 Added the ability to:
 Persist an iterator or stream as a doument
 Halt or stop the iterator stream (after
exhausting the already requested
documents).
 For restart add the ability to restart the
iterator or stream
 Allows conversion from an iterator to a
stream and back.

21. Queues – Initialization Code
 Initialize a Tailable Cursor
 Share with multiple processes
Find.Builder builder = new Find.Builder(BuilderFactory.start());
builder.setTailable(true);
builder.setAwaitData(false); // Sigh.
MongoIterator<Document> cursor = collection.find(builder.build());
// Graceful shutdown of the iterator locally but not on the server.
cursor.stop();
while (cursor.hasNext()) {
System.out.println(cursor.next());
}
collection = db.getCollection("lookup");
collection.delete(BuilderFactory.start().add("_id", collectionName));
collection.insert(BuilderFactory.start().add("_id", collectionName)
.add("cursor", cursor.asDocument()));

22. Queues – Consumer Code
MongoCollection index = client.getDatabase(args[1]).getCollection(
"lookup");
Document queueLookupDoc = index.findOne(BuilderFactory.start().add(
"_id", args[2]));
DocumentElement cursorElement = queueLookupDoc.get(
DocumentElement.class, "cursor");
MongoIterator<Document> iter = client.restart(cursorElement
.getDocument());
long lastCount = 0;
while (iter.hasNext()) {
Document doc = iter.next();
// Do stuff
}

23. Queues Demo

24. Restartable Cursors - Gotcha
 Remember this: builder.setAwaitData(false); // Sigh.
 That masks a small issue...
Pin( long long cursorid ) :
_cursorid( INVALID_CURSOR_ID ) {
recursive_scoped_lock lock( ccmutex );
ClientCursor *cursor = ClientCursor::find_inlock( cursorid, true );
if ( cursor ) {
uassert( 12051, "clientcursor already in use? driver problem?",
cursor->_pinValue < 100 );
cursor->_pinValue += 100;
_cursorid = cursorid;
}
}
 Closes the cursor on the client
 https://jira.mongodb.org/browse/SERVER-8602
 https://github.com/allanbank/mongo/tree/concurrent_cursor_support

25. Final Thoughts
 Coordination Service
 Watchers
 Group Management
 Message Broker
 Topics
 Queues
 Remember the Disclaimer

26. Questions?
 Contact Information:
Robert.J.Moore@allanbank.com