During this session we will cover the best practices for implementing a real-time inventory with MongoDB. This includes properly model quantities and stores to avoid large numbers of documents being indexed, how to efficiently use geo-indexing to find the closest store with a specific item available and how to run aggregation to gather interesting inventory stats. We will also cover operational considerations, like how to make inventory queries and updates from anywhere be low-latency and resilient to network partitions via tag-aware sharding.

1. Senior Consulting Engineer, MongoDB
Norman Graham
#MongoDBWorld
Retail Reference Architecture:
Real-Time, Geo-Distributed
Inventory

2. Inventory
Inventory
MongoDB
External Inventory
Internal Inventory
Regional Inventory
Purchase Orders
Fulfillment
Promotions

3. Inventory – Traditional Architecture
Relational DB
System of Records
Nightly
Batches
Analytics,
Aggregations,
Reports
Caching
Layer
Field Inventory
Internal &
External Apps
Point-in-time
Loads

4. Inventory – Opportunities Missed
• Can’t reliably detect availability
• Can't redirect purchasers to in-store pickup
• Can’t do intra-day replenishment
• Degraded customer experience
• Higher internal expense

5. Inventory – Principles
• Single view of the inventory
• Used by most services and channels
• Read-dominated workload
• Local, real-time writes
• Bulk writes for refresh
• Geographically distributed
• Horizontally scalable

6. Requirement Challenge MongoDB
Single view of
inventory
Ensure availability of
inventory information on
all channels and
services
Developer-friendly,
document-oriented
storage
High volume,
low latency reads
Anytime, anywhere
access to inventory
data without
overloading the system
of record
Fast, indexed reads
Local reads
Horizontal scaling
Bulk updates,
intra-day deltas
Provide window-in-time
consistency for highly
available services
Bulk writes
Fast, in-place updates
Horizontal scaling
Rapid application
development cycles
Deliver new services
rapidly to capture new
opportunities
Flexible schema
Rich query language
Agile-friendly iterations
Inventory – Requirements

7. Inventory – Target Architecture
Relational DB
System of Records
Analytics,
Aggregations,
Reports
Field Inventory
Internal &
External Apps
Inventory
Assortments
Shipments
Audits
Products
Stores
Point-in-time
Loads
Nightly
Refresh
Real-time
Updates

8. Horizontal Scaling
Inventory – Technical Decisions
Store
Inventory
Schema
Indexing

9. Inventory – Collections
Stores Inventory
Products
Audits AssortmentsShipments

10. > db.stores.findOne()
{
"_id" : ObjectId("53549fd3e4b0aaf5d6d07f35"),
"className" : "catalog.Store",
"storeId" : "store0",
"name" : "Bessemer store",
"address" : {
"addr1" : "1st Main St",
"city" : "Bessemer",
"state" : "AL",
"zip" : "12345",
"country" : "US"
},
"location" : [ -86.95444, 33.40178 ],
...
}
Stores – Sample Document

11. Stores – Sample Queries
• Get a store by storeId
db.stores.find({ "storeId" : "store0" })
• Get a store by zip code
db.stores.find({ "address.zip" : "12345" })

12. What’s near me?

13. Stores – Sample Geo Queries
• Get nearby stores sorted by distance
db.runCommand({
geoNear : "stores",
near : {
type : "Point",
coordinates : [-82.8006, 40.0908] },
maxDistance : 10000.0,
spherical : true
})

14. Stores – Sample Geo Queries
• Get the five nearest stores within 10 km
db.stores.find({
location : {
$near : {
$geometry : {
type : "Point",
coordinates : [-82.80, 40.09] },
$maxDistance : 10000.0 } }
}).limit(5)

15. Stores – Indices
• { "storeId" : 1 }, { "unique" : true }
• { "name" : 1 }
• { "address.zip" : 1 }
• { "location" : "2dsphere" }

16. > db.inventory.findOne()
{
"_id": "5354869f300487d20b2b011d",
"storeId": "store0",
"location": [-86.95444, 33.40178],
"productId": "p0",
"vars": [
{ "sku": "sku1", "q": 14 },
{ "sku": "sku3", "q": 7 },
{ "sku": "sku7", "q": 32 },
{ "sku": "sku14", "q": 65 },
...
]
}
Inventory – Sample Document

17. Inventory – Sample Queries
• Get all items in a store
db.inventory.find({ storeId : "store100" })
• Get quantity for an item at a store
db.inventory.find({
"storeId" : "store100",
"productId" : "p200"
})

18. Inventory – Sample Queries
• Get quantity for a sku at a store
db.inventory.find(
{
"storeId" : "store100",
"productId" : "p200",
"vars.sku" : "sku11736"
},
{ "vars.$" : 1 }
)

19. Inventory – Sample Update
• Increment / decrement inventory for an item at a
store
db.inventory.update(
{
"storeId" : "store100",
"productId" : "p200",
"vars.sku" : "sku11736"
},
{ "$inc" : { "vars.$.q" : 20 } }
)

20. Inventory – Sample Aggregations
• Aggregate total quantity for a product
db.inventory.aggregate( [
{ $match : { productId : "p200" } },
{ $unwind : "$vars" },
{ $group : {
_id : "result",
count : { $sum : "$vars.q" } } } ] )
{ "_id" : "result", "count" : 101752 }

21. Inventory – Sample Aggregations
• Aggregate total quantity for a store
db.inventory.aggregate( [
{ $match : { storeId : "store100" } },
{ $unwind : "$vars" },
{ $match : { "vars.q" : { $gt : 0 } } },
{ $group : {
_id : "result",
count : { $sum : 1 } } } ] )
{ "_id" : "result", "count" : 29347 }

22. Inventory – Sample Aggregations
• Aggregate total quantity for a store
db.inventory.aggregate( [
{ $match : { storeId : "store100" } },
{ $unwind : "$vars" },
{ $group : {
_id : "result",
count : { $sum : "$vars.q" } } } ] )
{ "_id" : "result", "count" : 29347 }

24. Inventory – Sample Geo-Query
• Get inventory for an item near a point
db.runCommand( {
geoNear : "inventory",
near : {
type : "Point",
coordinates : [-82.8006, 40.0908] },
maxDistance : 10000.0,
spherical : true,
limit : 10,
query : { "productId" : "p200",
"vars.sku" : "sku11736" } } )

25. Inventory – Sample Geo-Query
• Get closest store with available sku
db.runCommand( {
geoNear : "inventory",
near : {
type : "Point",
coordinates : [-82.800672, 40.090844] },
maxDistance : 10000.0,
spherical : true,
limit : 1,
query : {
productId : "p200",
vars : {
$elemMatch : { sku : "sku11736",
q : { $gt : 0 } } } } } )

26. Inventory – Sample Geo-
Aggregation
• Get count of inventory for an item near a point
db.inventory.aggregate([
{ $geoNear: {
near : { type : "Point",
coordinates : [-82.800672, 40.090844] },
distanceField: "distance",
maxDistance: 10000.0, spherical : true,
query: { productId: "p200",
vars : { $elemMatch : { sku : "sku11736",
q : {$gt : 0} } } },
includeLocs: "dist.location",
num: 5 } },
{ $unwind: "$vars" },
{ $match: { "vars.sku" : "sku11736" } },
{ $group: { _id: "result", count: {$sum: "$vars.q"} } }])

27. Inventory – Sample Indices
• { storeId : 1 }
• { productId : 1, storeId : 1 }
• { productId : 1, location : "2dsphere" }
• Why not "vars.sku"?
– { productId : 1, storeId : 1, "vars.sku" : 1 }

28. Horizontal Scaling
Inventory – Technical Decisions
Store
Inventory
Schema
Indexing

29. Shard
East
Shard
Central
Shard
West
East DC
Inventory – Sharding Topology
West DC Central DC
Legacy
Inventory
Primary
Primary
Primary

30. Inventory – Shard Key
• Choose shard key
– { storeId : 1 } ?
– { productId : 1, storeId : 1 }?
– { storeId : 1, productId : 1 } ?
• Set up sharding
– sh.enableSharding("inventoryDB")
– sh.shardCollection(
"inventoryDB.inventory",
{ storeId : 1, productId : 1 })

31. Inventory – Shard Tags
• Set up shard tags
– sh.addShardTag("shard0000", "west")
– sh.addShardTag("shard0001", "central")
– sh.addShardTag("shard0002", "east")
• Set up tag ranges
– sh.addTagRange("inventoryDB.inventory",
{ storeId : 0 }, { storeId : 100}, "west" )
– sh.addTagRange("inventoryDB.inventory",
{ storeId : 100 }, { storeId : 200 }, "central" )
– sh.addTagRange("inventoryDB.inventory",
{ storeId : 200 }, { storeId : 300 }, "east" )

32. Senior Consulting Engineer, MongoDB
Norman Graham
#MongoDBWorld
Thank You