1. Building your first Java app with MongoDB

2. Today’s webinar
• Introduction to MongoDB.
• Use a location-based surf report app as an
example.
• Basic data modelling, queries (inc
geospatial) and aggregation.
• MongoDB JS shell and Java code examples.

3. What is MongoDB?
• MongoDB is a scalable, high-performance,
open source NoSQL database.
• Document-oriented storage
• Full index support - including geospatial
• Querying
• Fast in-place updates
• Map-Reduce and Aggregation
• Replication and high availability
• Horizontal scaling with auto-sharding
• GridFS

4. Open source
• MongoDB is an open source project
• On GitHub
• Licensed under the AGPL
• Started & sponsored by 10gen
• Commercial licenses available
• Contributions welcome

5. High performance
• Written in C++
• Extensive use of memory-mapped files
i.e. read-through write-through memory
caching.
• Runs nearly everywhere
• Data serialized as BSON (fast parsing)
• Full support for primary & secondary indexes
• Document model = less work

6. Terminology

7. Example document
{
"_id" :
ObjectId("4c4ba5c0672c685e5e8aabf3"),
"reporter" : "Hergé",
"date" : ISODate("2012-02-
02T11:52:27.442Z"),
"location" : {
"name" : "Watergate Bay",
"state" : "Cornwall",
"country" : "UK”
}

8. Let’s build a location-based surf reporting app!

9. Let’s build a location-based surf reporting app!
• Report current conditions

10. Let’s build a location-based surf reporting app!
• Report current conditions
• Get current local conditions

11. Let’s build a location-based surf reporting app!
• Report current conditions
• Get current local conditions
• Determine best conditions per beach

12. Document structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"),
"reporter" : "matt",
"location" : {
"coordinates" : [
-50.46667,
5.05
],
"name" : "Watergate Bay",
"county" : "Cornwall"
},
"conditions" : {
"height" : 3,
"period" : 13,
"rating" : 5
},
"date" : ISODate("2011-11-16T20:17:17.277Z")
}

13. Document structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key,
"reporter" : "matt",
Unique,
"location" : {
"coordinates" : [ Auto-indexed
-50.46667,
5.05
],
"name" : "Watergate Bay",
"county" : "Cornwall"
},
"conditions" : {
"height" : 3,
"period" : 13,
"rating" : 5
},
"date" : ISODate("2011-11-16T20:17:17.277Z")
}

14. Document structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key,
"reporter" : "matt",
Unique,
"location" : {
"coordinates" : [ Auto-indexed
-50.46667,
5.05
], Geospatial
"name" : "Watergate Bay",
Compound Index
"county" : "Cornwall"
},
"conditions" : {
"height" : 3,
"period" : 13,
"rating" : 5
},
"date" : ISODate("2011-11-16T20:17:17.277Z")
}

15. Document structure
{
"_id" : ObjectId("504ceb3d30042d707af96fef"), Primary Key,
"reporter" : "matt",
Unique,
"location" : {
"coordinates" : [ Auto-indexed
-50.46667,
5.05
], Geospatial
"name" : "Watergate Bay",
Compound Index
"county" : "Cornwall"
},
"conditions" : {
"height" : 3,
"period" : 13,
"rating" : 5 Indexed for
}, Time-To-Live
"date" : ISODate("2011-11-16T20:17:17.277Z")
}

16. Java: DBObjects
BasicDBObject report = new BasicDBObject();
report.put("reporter", "matt");
report.put("date", new Date());
BasicDBObject location = new BasicDBObject();
location.put("name", "Watergate Bay");
location.put("county", "Cornwall");
location.put("country", "UK");
location.put("coordinates", new double[] {-50.46667, 5.05});
report .put("location", location);
BasicDBObject conditions = new BasicDBObject();
conditions .put("height", 3);
conditions .put("period", 13);
conditions.put("rating", 5);
report .put("conditions", conditions);

17. Connecting to MongoDB
Mongo m = new Mongo(Arrays.asList(
new ServerAddress("localhost", 27017),
new ServerAddress("localhost", 27018),
new ServerAddress("localhost", 27019)));
DB db = m.getDB( ”SurfReports” );
DBCollection coll = db. getCollection(“reports”);

18. Inserting and finding a document
coll.insert(report);
DBObject report2 = coll.findOne();
System.out.println(report2);
{ "_id" : ObjectId("504ceb3d30042d707af96fef"), "reporter" : "matt”, "location" : {
"coordinates" : [-50.46667, 5.05], "name" : "Watergate Bay”, "county" : "Cornwall" },
"conditions" : { "height" : 3, "period" : 13, "rating" : 5}, "date" : ISODate("2011-11-
16T20:17:17.277Z”)}

19. Get local surf conditions - shell
> db.reports.find(
{
"location.coordinates" : { $near : [50.41, -5.08] ,
$maxDistance : 0.5},
date : { $gte : new Date(2012, 11, 21)}
},
{"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})

20. Get local surf conditions - shell
> db.reports.find(
{
"location.coordinates" : { $near : [50.41, -5.08] ,
$maxDistance : 0.5},
date : { $gte : new Date(2012, 11, 21)}
},
{"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports

21. Get local surf conditions - shell
> db.reports.find(
{
"location.coordinates" : { $near : [50.41, -5.08] ,
$maxDistance : 0.5},
date : { $gte : new Date(2012, 11, 21)}
},
{"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports
• Get today’s reports

22. Get local surf conditions - shell
> db.reports.find(
{
"location.coordinates" : { $near : [50.41, -5.08] ,
$maxDistance : 0.5},
date : { $gte : new Date(2012, 11, 21)}
},
{"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports
• Get today’s reports
• Return only the relevant info

23. Get local surf conditions - shell
> db.reports.find(
{
"location.coordinates" : { $near : [50.41, -5.08] ,
$maxDistance : 0.5},
date : { $gte : new Date(2012, 11, 21)}
},
{"location.name" :1, _id : 0, "conditions" :1}
).sort({"conditions.rating" : -1})
• Get local reports
• Get today’s reports
• Return only the relevant info
• Show me the best surf first

24. Java: Building the query

25. Local surf conditions results
{ "location" : { "name" : ”Watergate Bay" }, "conditions" : { "height" : 3, "period" : 13, "rating" : 5 } }
{ "location" : { "name" : "Watergate Bay" }, "conditions" : { "height" : 2.5, "period" : 9, "rating" :4 } }
{ "location" : { "name" : ”Fistral North" }, "conditions" : { "height" : 3, "period" : 15, "rating" : 3 } }
{ "location" : { "name" : "Fistral North" }, "conditions" : { "height" : 3, "period" : 16, "rating" : 2 } }
{ "location" : { "name" : "Fistral North" }, "conditions" : { "height" : 0, "period" : 8, "rating" : 1 } }
{ "location" : { "name" : ”Little Fistral" }, "conditions" : { "height" : 3, "period" : 10, "rating" : 1 } }
{ "location" : { "name" : "Little Fistral" }, "conditions" : { "height" : 1, "period" : 15, "rating" : 1 } }
{ "location" : { "name" : ”Cribbar" }, "conditions" : { "height" : 5, "period" : 6, "rating" : 1 } }
{ "location" : { "name" : "Cribbar" }, "conditions" : { "height" : 1, "period" : 6, "rating" : 1 } }
{ "location" : { "name" : "Cribbar" }, "conditions" : { "height" : 0, "period" : 10, "rating" : 1 } }

26. Analysis feature:
Aggregation framework
What are the best conditions for my beach?

27. Aggregation framework
• Process a stream of documents
• Original input is a collection
• Final output is a result document
• Series of operators
• Filter or transform data
• Input/output chain
ps ax | grep mongod | head -n 1

28. Pipelining operations
$match Match “Fistral North”
$project Only interested in conditions
Group by rating, average
$group
wave height and wave period
$sort Order by best conditions

29. Aggregation framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : ”Fistral North"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}

30. Aggregation framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : ”Fistral North"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Match “Fistral North”

31. Aggregation framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : ”Fistral North"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Only interested in conditions

32. Aggregation framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : ”Fistral North"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Group by rating and average conditions

33. Aggregation framework
{ "aggregate" : "reports" ,
"pipeline" : [
{ "$match" : { "location.name" : ”Fistral North"}} ,
{ "$project" : { "conditions" : 1}} ,
{ "$group" : {
"_id" : "$conditions.rating" ,
"average height" : { "$avg" : "$conditions.height"} ,
"average period" : { "$avg" : "$conditions.period"}}} ,
{ "$sort" : { "_id" : -1}}
]
}
Show the best conditions first

34. Java: Aggregation helper

35. High availability: Replica sets
• Initialize -> Election
• Primary + data replication from primary to secondary
Node 1 Node 2
Secondary Heartbeat Secondary
Node 3
Primary Replication
Replication

36. Replica Set - failure
• Primary down/network failure
• Automatic election of new primary if majority exists
Primary Election
Node 1 Node 2
Secondary Heartbeat Secondary
Node 3
Primary

37. Replica Set - failover
• New primary elected
• Replication established from new primary
Node 1 Node 2
Secondary Heartbeat Primary
Node 3
Primary

38. Durability
• WriteConcern.SAFE
• WriteConcern.JOURNAL_SAFE
• WriteConcern.FSYNC_SAFE
• WriteConcern.REPLICAS_SAFE
• WriteConcern.MAJORITY
• WriteConcern.NORMAL
• WriteConcern.NONE
m.setWriteConcern(WriteConcern.SAFE);
coll.save(dbObj,WriteConcern.SAFE);

39. Read preferences
• ReadPreference.primary()
• ReadPreference.primaryPreferred()
• ReadPreference.secondary()
• ReadPreference.secondaryPreferred()
• ReadPreference.nearest()
ReadPreference preference = ReadPreference.primaryPreferred();
DBCursor cur = new DBCursor(collection, query, null, preference);

40. Sharding
• Automatic partitioning and management
• Range based
• Convert to sharded system with no downtime
• Fully consistent
• No code changes required

41. Scaling MongoDB
MongoDB
Single Instance
or
Replica Set
Client
Application

42. Scaling MongoDB
Client
Application

43. Mechanism of sharding
Complete data set
Define shard key on location name
Cribbar Fistral Little Sennen Watergate
North Fistral Bay

44. Mechanism of sharding
Chunk Chunk
Define shard key on location name
Cribbar Fistral Little Sennen Watergate
North Fistral Bay

45. Mechanism of sharding
Chunk Chunk Chunk Chunk
Cribbar Fistral Little Sennen Watergate
North Fistral Bay

46. Mechanism of sharding
Chunk Chunk Chunk Chunk
Shard 1 Shard 2 Shard 3 Shard 4

47. Mechanism of sharding
Chu Chu
nkc nkc
Chu Chu Chu Chu Chu Chu Chu Chu
nkc nkc nkc nkc nkc nkc nkc nkc
Shard 1 Shard 2 Shard 3 Shard 4

48. Mechanism of sharding
Query: Watergate Bay
Client
Application
config
config
config
mongos
Chu Chu
nkc nkc
Chu Chu Chu Chu Chu Chu Chu Chu
nkc nkc nkc nkc nkc nkc nkc nkc
Shard 1 Shard 2 Shard 3 Shard 4
48

49. Mechanism of sharding
Query: Cribbar
Client
Application
config
config
config
mongos
Chu Chu
nkc nkc
Chu Chu Chu Chu Chu Chu Chu Chu
nkc nkc nkc nkc nkc nkc nkc nkc
Shard 1 Shard 2 Shard 3 Shard 4
49

50. More information
Resource Location
MongoDB Downloads www.mongodb.org/download
Free Online Training education.10gen.com
Webinars and Events www.10gen.com/events
White Papers www.10gen.com/white-papers
Customer Case Studies www.10gen.com/customers
Presentations www.10gen.com/presentations
Documentation docs.mongodb.org
Additional Info info@10gen.com