The document provides an overview of MongoDB, including: - MongoDB uses a document-based data model and stores data in JSON-like documents using BSON format. - The main MongoDB processes are mongod, which stores the data, mongos for routing queries, and config servers for metadata. - MongoDB supports scaling out by adding mongod processes and sharding data. - The document also discusses Amadeus' architecture using MongoDB, with applications connecting to mongos processes which route to MongoDB clusters.

1. Confidential
RESTRICTED
Confidential
RESTRICTED
MongoDB
Monitoring & Operability
©2014AmadeusITGroupSA
Training for App Mgmt and
Operations
Nicolas Motte, 18/11/2014

2. Confidential
RESTRICTED
Page 2
Agenda
_ Overview of MongoDB
_ Amadeus Architecture around NoSQL
_ MongoDB concepts
_ Workflows
_ Operability & Monitoring
4
14
21
37
41
©2014AmadeusITGroupSA

3. Confidential
RESTRICTED
Overview of MongoDB
1
©2014AmadeusITGroupSA

4. Confidential
RESTRICTED
Page 4
Overview of MongoDB
_ Document store
©2014AmadeusITGroupSA
High-level

5. Confidential
RESTRICTED
Page 5
Overview of MongoDB
_ Document store
• Stores JSON-style documents
©2014AmadeusITGroupSA
High-level

6. Confidential
RESTRICTED
Page 6
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
©2014AmadeusITGroupSA
High-level

7. Confidential
RESTRICTED
Page 7
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
_ Several binaries written in C++
©2014AmadeusITGroupSA
High-level

8. Confidential
RESTRICTED
Page 8
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
_ Several binaries written in C++
• Mongod : The actual database process, owning the data
©2014AmadeusITGroupSA
High-level

9. Confidential
RESTRICTED
Page 9
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
_ Several binaries written in C++
• Mongod : The actual database process, owning the data
• Mongos : A connection multiplexer
©2014AmadeusITGroupSA
High-level

10. Confidential
RESTRICTED
Page 10
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
_ Several binaries written in C++
• Mongod : The actual database process, owning the data
• Mongos : A connection multiplexer
• Config servers : Holds cluster topology
©2014AmadeusITGroupSA
High-level

11. Confidential
RESTRICTED
Page 11
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
_ Several binaries written in C++
• Mongod : The actual database process, owning the data
• Mongos : A connection multiplexer
• Config servers : Holds cluster topology
• Agent : Monitoring process
©2014AmadeusITGroupSA
High-level

12. Confidential
RESTRICTED
Page 12
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
_ Several binaries written in C++
• Mongod : The actual database process, owning the data
• Mongos : A connection multiplexer
• Config servers : Holds cluster topology
• Agent : Monitoring process
• MMS : Monitoring UI, connected to the Agents
©2014AmadeusITGroupSA
High-level

13. Confidential
RESTRICTED
Page 13
Overview of MongoDB
_ Document store
• Stores JSON-style documents
• Uses BSON format for storage
_ Several binaries written in C++
• Mongod : The actual database process, owning the data
• Mongos : A connection multiplexer
• Config servers : Holds cluster topology
• Agent : Monitoring process
• MMS : Monitoring UI, connected to the Agents
_ Opensource and Enterprise versions
©2014AmadeusITGroupSA
High-level

14. Confidential
RESTRICTED
Page 14
Overview of MongoDB
©2014AmadeusITGroupSA
Mongod
Mongod
Mongod
Architecture

15. Confidential
RESTRICTED
Page 15
Overview of MongoDB
©2014AmadeusITGroupSA
Mongod
Mongod
Mongod
Mongos
Architecture

16. Confidential
RESTRICTED
Page 16
Overview of MongoDB
©2014AmadeusITGroupSA
Mongod
Mongod
Mongod
Mongos
Config
Architecture

17. Confidential
RESTRICTED
Page 17
Overview of MongoDB
©2014AmadeusITGroupSA
Mongod
Mongod
Mongod
Mongos
Config
Client
Client
Client
Architecture

18. Confidential
RESTRICTED
Page 18
Overview of MongoDB
©2014AmadeusITGroupSA
Mongod
Mongod
Mongod
Mongos
Config
Client
Client
Client
Agent
Architecture

19. Confidential
RESTRICTED
Page 19
Overview of MongoDB
©2014AmadeusITGroupSA
Mongod
Mongod
Mongod
Mongos
Config
Client
Client
Client
Agent
MMS
Architecture

20. Confidential
RESTRICTED
Page 20
Overview of MongoDB
_ Why do we use MongoDB ?
©2014AmadeusITGroupSA
Couchbase
Memcached
MongoDB
Oracle
Comparison

21. Confidential
RESTRICTED
Page 21
Overview of MongoDB
Memcached Couchbase MongoDB Oracle
Database Model Key-Value Key-Value Document Table
Data scheme Schema-free Schema-free Schema-free Schema-full
Indexes
Persistency
Data Integrity
Transaction
©2014AmadeusITGroupSA
Comparison

22. Confidential
RESTRICTED
Page 22
Overview of MongoDB
Memcached Couchbase MongoDB Oracle
Database Model Key-Value Key-Value Document Table
Data scheme Schema-free Schema-free Schema-free Schema-full
Indexes
Persistency
Data Integrity
Transaction
©2014AmadeusITGroupSA
Comparison

23. Confidential
RESTRICTED
Page 23
Overview of MongoDB
Memcached Couchbase MongoDB Oracle
Database Model Key-Value Key-Value Document Table
Data scheme Schema-free Schema-free Schema-free Schema-full
Indexes
Persistency
Data Integrity
Transaction
©2014AmadeusITGroupSA
Comparison

24. Confidential
RESTRICTED
Page 24
Overview of MongoDB
Memcached Couchbase MongoDB Oracle
Database Model Key-Value Key-Value Document Table
Data scheme Schema-free Schema-free Schema-free Schema-full
Indexes
Persistency
Data Integrity
Transaction
©2014AmadeusITGroupSA
Comparison

25. Confidential
RESTRICTED
Page 25
Overview of MongoDB
Memcached Couchbase MongoDB Oracle
Database Model Key-Value Key-Value Document Table
Data scheme Schema-free Schema-free Schema-free Schema-full
Indexes
Persistency
Data Integrity
Transaction
©2014AmadeusITGroupSA
Comparison

26. Confidential
RESTRICTED
Page 26
Overview of MongoDB
Memcached Couchbase MongoDB Oracle
Database Model Key-Value Key-Value Document Table
Data scheme Schema-free Schema-free Schema-free Schema-full
Indexes
Persistency
Data Integrity
Transaction
©2014AmadeusITGroupSA
Comparison

27. Confidential
RESTRICTED
Page 27
Overview of MongoDB
Memcached Couchbase MongoDB Oracle
Database Model Key-Value Key-Value Document Table
Data scheme Schema-free Schema-free Schema-free Schema-full
Indexes
Persistency
Data Integrity
Transaction
©2014AmadeusITGroupSA
Comparison

28. Confidential
RESTRICTED
Page 28
Overview of MongoDB
_ Mongod is the process storing the data
Mongod

29. Confidential
RESTRICTED
Page 29
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
©2014AmadeusITGroupSA
Mongod

30. Confidential
RESTRICTED
Page 30
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
Mongod

31. Confidential
RESTRICTED
Page 31
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
• The primary is the only process that
receives write operations
©2014AmadeusITGroupSA
Mongod

32. Confidential
RESTRICTED
Page 32
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
• The primary is the only process that
receives write operations
• SECONDARY
©2014AmadeusITGroupSA
Mongod

33. Confidential
RESTRICTED
Page 33
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
• The primary is the only process that
receives write operations
• SECONDARY
• A secondary maintains a copy of the
primary’s data set
©2014AmadeusITGroupSA
Mongod

34. Confidential
RESTRICTED
Page 34
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
• The primary is the only process that
receives write operations
• SECONDARY
• A secondary maintains a copy of the
primary’s data set
• ARBITER
©2014AmadeusITGroupSA
Mongod

35. Confidential
RESTRICTED
Page 35
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
• The primary is the only process that
receives write operations
• SECONDARY
• A secondary maintains a copy of the
primary’s data set
• ARBITER
• An arbiter does not have a copy of
data set and cannot become a primary
©2014AmadeusITGroupSA
Mongod

36. Confidential
RESTRICTED
Page 36
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
• The primary is the only process that
receives write operations
• SECONDARY
• A secondary maintains a copy of the
primary’s data set
• ARBITER
• An arbiter does not have a copy of
data set and cannot become a primary
©2014AmadeusITGroupSA
Mongod

37. Confidential
RESTRICTED
Page 37
Overview of MongoDB
_ Mongod is the process storing the data
_ 3 different types
• PRIMARY
• The primary is the only process that
receives write operations
• SECONDARY
• A secondary maintains a copy of the
primary’s data set
• ARBITER
• An arbiter does not have a copy of
data set and cannot become a primary
_ Heartbeats are sent every 10s
between mongods
©2014AmadeusITGroupSA
Mongod
Heartbeat

38. Confidential
RESTRICTED
Page 38
Overview of MongoDB
_ Config servers are special mongod instances that store
the metadata for a cluster (like topology)
©2014AmadeusITGroupSA
Config Server

39. Confidential
RESTRICTED
Page 39
Overview of MongoDB
_ Config servers are special mongod instances that store
the metadata for a cluster (like topology)
_ A production cluster has exactly three config servers
©2014AmadeusITGroupSA
Config Server

40. Confidential
RESTRICTED
Page 40
Overview of MongoDB
_ Mongos is a
connection
multiplexer
_ Heartbeat are
sent every 10s
_ Map/Reduce
queries
©2014AmadeusITGroupSA
Mongos
Hundredsofclients
A single connection/server
Clients are not aware of the
topology
Mongos hides all the
complexity and standardizes
the configuration
Mongos
Mongod
Mongod
Mongod
Config
Heartbeat

41. Confidential
RESTRICTED
Page 41
Overview of MongoDB
_ Mongos is a
connection
multiplexer
©2014AmadeusITGroupSA
Mongos
Mongos

42. Confidential
RESTRICTED
Page 42
Overview of MongoDB
_ Mongos is a
connection
multiplexer
©2014AmadeusITGroupSA
Mongos
Hundredsofclients
Mongos
Mongod
Mongod
Mongod
Config

43. Confidential
RESTRICTED
Page 43
Overview of MongoDB
_ Mongos is a
connection
multiplexer
©2014AmadeusITGroupSA
Mongos
Hundredsofclients
A single connection/server
Mongos
Mongod
Mongod
Mongod
Config

44. Confidential
RESTRICTED
Page 44
Overview of MongoDB
_ Mongos is a
connection
multiplexer
©2014AmadeusITGroupSA
Mongos
Hundredsofclients
A single connection/server
Clients are not aware of the
topology
Mongos
Mongod
Mongod
Mongod
Config

45. Confidential
RESTRICTED
Page 45
Overview of MongoDB
_ Mongos is a
connection
multiplexer
©2014AmadeusITGroupSA
Mongos
Hundredsofclients
A single connection/server
Clients are not aware of the
topology
Mongos hides all the
complexity and standardizes
the configuration
Mongos
Mongod
Mongod
Mongod
Config

46. Confidential
RESTRICTED
Page 46
Overview of MongoDB
_ Mongos is a
connection
multiplexer
_ Heartbeat are
sent every 10s
©2014AmadeusITGroupSA
Mongos
Hundredsofclients
A single connection/server
Clients are not aware of the
topology
Mongos hides all the
complexity and standardizes
the configuration
Mongos
Mongod
Mongod
Mongod
Config
Heartbeat

47. Confidential
RESTRICTED
Page 47
Overview of MongoDB
_ Mongos is a
connection
multiplexer
_ Heartbeat are
sent every 10s
_ Map/Reduce
queries
©2014AmadeusITGroupSA
Mongos
Hundredsofclients
A single connection/server
Clients are not aware of the
topology
Mongos hides all the
complexity and standardizes
the configuration
Mongos
Mongod
Mongod
Mongod
Config
Heartbeat

48. Confidential
RESTRICTED
Page 48
Overview of MongoDB
_ Monitoring process
©2014AmadeusITGroupSA
Agent
Agent

49. Confidential
RESTRICTED
Page 49
Overview of MongoDB
_ Monitoring process
_ 1 active/1 passive
per cluster
©2014AmadeusITGroupSA
Agent
Agent

50. Confidential
RESTRICTED
Page 50
Overview of MongoDB
_ Monitoring process
_ 1 active/1 passive
per cluster
_ Retrieves the
config servers from
MMS
©2014AmadeusITGroupSA
Agent
Agent
MMS

51. Confidential
RESTRICTED
Page 51
Overview of MongoDB
_ Monitoring process
_ 1 active/1 passive
per cluster
_ Retrieves the
config servers from
MMS
_ Retrieves the list of
processes from a
config server
©2014AmadeusITGroupSA
Agent
Config
Agent

52. Confidential
RESTRICTED
Page 52
Overview of MongoDB
_ Monitoring process
_ 1 active/1 passive
per cluster
_ Retrieves the
config servers from
MMS
_ Retrieves the list of
processes from a
config server
_ Connects to all
processes
©2014AmadeusITGroupSA
Agent
Mongod
Mongod
Mongod
Mongos
Config
Agent
MMS

53. Confidential
RESTRICTED
Page 53
_ Datastore
• Pros
• Structured data
• Flexible (Schema-less)
• Indexes
• Scaling is “easy”
• Easy to deploy
• Cons
• Transactions are not yet supported
• Each command is atomic but locking is at database level in 2.6 (moving to
document level in 2.8)
_ Analytical search
• MSX
Use Cases
©2014AmadeusITGroupSA
Overview of MongoDB

54. Confidential
RESTRICTED
Amadeus Architecture Around NoSQL
2
©2014AmadeusITGroupSA

55. Confidential
RESTRICTED
Page 55
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
Amadeus Architecture Around NoSQL

56. Confidential
RESTRICTED
Page 56
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
Amadeus Architecture Around NoSQL

57. Confidential
RESTRICTED
Page 57
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
_ Deployed on each AS
mongos
Amadeus Architecture Around NoSQL

58. Confidential
RESTRICTED
Page 58
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
_ Deployed on each AS
_ BEs are connected via the
standard Mongo C++
client
BE BE BEBE
mongos
Amadeus Architecture Around NoSQL

59. Confidential
RESTRICTED
Page 59
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
_ Deployed on each AS
_ BEs are connected via the
standard Mongo C++
client
_ Mongos is connected to all
the MongoDB servers
BE BE BEBE
mongos
Amadeus Architecture Around NoSQL

60. Confidential
RESTRICTED
Page 60
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
_ Deployed on each AS
_ BEs are connected via the
standard Mongo C++
client
_ Mongos is connected to all
the MongoDB servers
_ Duwrapped process
monitored by the local
Agent
SI Agent
BE BE BEBE
mongos
dwi
Amadeus Architecture Around NoSQL

61. Confidential
RESTRICTED
Page 61
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
_ Deployed on each AS
_ BEs are connected via the
standard Mongo C++
client
_ Mongos is connected to all
the MongoDB servers
_ Duwrapped process
monitored by the local
Agent
_ Equivalent of ProxyD for
Memcached and
Couchbase
SI Agent
BE BE BEBE
mongos
dwi
Amadeus Architecture Around NoSQL

62. Confidential
RESTRICTED
Page 62
Mongos
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongos ?
_ Deployed on each AS
_ BEs are connected via the
standard Mongo C++
client
_ Mongos is connected to all
the MongoDB servers
_ Duwrapped process
monitored by the local
Agent
_ Equivalent of ProxyD for
Memcached and
Couchbase
_ Also running on NoSQL
Tier for administration
tasks
SI Agent
BE BE BEBE
mongos
dwi
mongos
Amadeus Architecture Around NoSQL

63. Confidential
RESTRICTED
Page 63
Mongod
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongod ?
Amadeus Architecture Around NoSQL

64. Confidential
RESTRICTED
Page 64
Mongod
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongod ?
Amadeus Architecture Around NoSQL

65. Confidential
RESTRICTED
Page 65
Mongod
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongod ?
_ Deployed on cluster
nodes
Amadeus Architecture Around NoSQL
mongod
mongod
mongod
mongod

66. Confidential
RESTRICTED
Page 66
Mongod
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongod ?
_ Deployed on cluster
nodes
_ Accepts connections
from Mongos on App
Tier and NoSQL Tier
mongod
mongod
mongod
mongod
mongos
Amadeus Architecture Around NoSQL

67. Confidential
RESTRICTED
Page 67
Mongod
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Mongod ?
_ Deployed on cluster
nodes
_ Accepts connections
from Mongos on App
Tier and NoSQL Tier
_ A mongod can be a
primary, a secondary
or an arbiter
mongod
mongod
mongod
mongod
mongos
Amadeus Architecture Around NoSQL

68. Confidential
RESTRICTED
Page 68
Agent
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Agent?
mongod
mongod
mongod
mongod
mongos
Amadeus Architecture Around NoSQL

69. Confidential
RESTRICTED
Page 69
Agent
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Agent?
_ Deployed on 2
cluster nodes, 1
active / 1 passive
mongod
mongod
mongod
mongod
mongos
agent
Amadeus Architecture Around NoSQL

70. Confidential
RESTRICTED
Page 70
Agent
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Agent?
_ Deployed on 2
cluster nodes, 1
active / 1 passive
_ Connects to Mongos
to monitor all
processes
mongod
mongod
mongod
mongod
mongos
agent
Amadeus Architecture Around NoSQL

71. Confidential
RESTRICTED
Page 71
Config Server
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Config servers?
mongod
mongod
mongod
mongod
mongos
agent
Amadeus Architecture Around NoSQL

72. Confidential
RESTRICTED
Page 72
Config Server
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Config servers?
_ Running on
dedicated nodes
mongod
mongod
mongod
mongod
mongos
agent
config
Amadeus Architecture Around NoSQL

73. Confidential
RESTRICTED
Page 73
Config Server
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Config servers?
_ Running on
dedicated nodes
_ Accepts connections
from mongos
mongod
mongod
mongod
mongod
mongos
agent
config
mongos
Amadeus Architecture Around NoSQL

74. Confidential
RESTRICTED
Page 74
Config Server
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
(GOS, DZB)
_ Config servers?
_ Running on
dedicated nodes
_ Accepts connections
from mongos
_ These nodes are
treated like any
other cluster nodes
in our tools
mongod
mongod
mongod
mongod
mongos
agent
config
mongos
Amadeus Architecture Around NoSQL

75. Confidential
RESTRICTED
Page 75
Amadeus Architecture Around NoSQL
Configuration
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
MongoS port
retrieved from
the MAG
dwi
Duwrapper Conf
(listen port)
+
Configuration files
Keyfile_xxx
Mongos_xxx.yml
xxx.pem
_ BE Configuration
_ Mongos
Configuration
_ MongoDB
Configuration

76. Confidential
RESTRICTED
MongoDB concepts
3
©2014AmadeusITGroupSA

77. Confidential
RESTRICTED
Page 77
MongoDB Concepts
_ Replica Set or Shard
MongoDB
©2014AmadeusITGroupSA

78. Confidential
RESTRICTED
Page 78
MongoDB Concepts
_ Replica Set or Shard
• A replica set is a group of mongod
instances that host the same data
set
MongoDB
©2014AmadeusITGroupSA

79. Confidential
RESTRICTED
Page 79
MongoDB Concepts
_ Replica Set or Shard
• A replica set is a group of mongod
instances that host the same data
set
MongoDB
©2014AmadeusITGroupSA

80. Confidential
RESTRICTED
Page 80
MongoDB Concepts
_ Shard Key
• Determines the distribution of the documents among the cluster’s shards
• The shard key is an indexed field that exists in every document
MongoDB
©2014AmadeusITGroupSA

81. Confidential
RESTRICTED
Page 81
MongoDB Concepts
_ Shard Map
• MongoDB partitions data in
“chunks”, chunks are associated
to a replica set
MongoDB
©2014AmadeusITGroupSA
Shard Map
Shard 1 Shard 2 Shard 3

82. Confidential
RESTRICTED
Page 82
MongoDB Concepts
_ Shard Map
• MongoDB partitions data in
“chunks”, chunks are associated
to a replica set
• When a chunk grows over the
chunk size (64 MB), MongoDB
splits the chunk in smaller
chunks. The (arbitrary) limit has
been chosen to avoid moving too
much data during a balancing.
MongoDB
©2014AmadeusITGroupSA
Shard Map
Shard 1 Shard 2 Shard 3

83. Confidential
RESTRICTED
Page 83
MongoDB Concepts
_ Shard Map
• MongoDB partitions data in
“chunks”, chunks are associated
to a replica set
• When a chunk grows over the
chunk size (64 MB), MongoDB
splits the chunk in smaller
chunks. The (arbitrary) limit has
been chosen to avoid moving too
much data during a balancing.
• The config server provides a
shard map with at least one
mongod process per shard
MongoDB
©2014AmadeusITGroupSA
Shard Map
Shard 1 Shard 2 Shard 3

84. Confidential
RESTRICTED
Page 84
MongoDB Concepts
_ MongoDB functional architecture
MongoDB
©2014AmadeusITGroupSA

85. Confidential
RESTRICTED
Page 85
MongoDB Concepts
_ MongoDB functional architecture
• A MongoDB cluster contains several databases
MongoDB
©2014AmadeusITGroupSA

86. Confidential
RESTRICTED
Page 86
MongoDB Concepts
_ MongoDB functional architecture
• A MongoDB cluster contains several databases
• Each database contains several collections (equivalent of Oracle schemas)
MongoDB
©2014AmadeusITGroupSA

87. Confidential
RESTRICTED
Page 87
MongoDB Concepts
_ MongoDB functional architecture
• A MongoDB cluster contains several databases
• Each database contains several collections (equivalent of Oracle schemas)
• A collection contains a set of JSON documents (equivalent of Oracle tables)
MongoDB
©2014AmadeusITGroupSA

88. Confidential
RESTRICTED
Page 88
MongoDB Concepts
_ MongoDB functional architecture
• A MongoDB cluster contains several databases
• Each database contains several collections (equivalent of Oracle schemas)
• A collection contains a set of JSON documents (equivalent of Oracle tables)
• Documents are distributed across all the shards
• Each sharded collection need a value to shard on (=shard key)
MongoDB
©2014AmadeusITGroupSA

89. Confidential
RESTRICTED
Page 89
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}

90. Confidential
RESTRICTED
Page 90
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos

91. Confidential
RESTRICTED
Page 91
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos
SHARDSHARD SHARD SHARD

92. Confidential
RESTRICTED
Page 92
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos
SHARDSHARD SHARD SHARD
INSERT in my_collection

93. Confidential
RESTRICTED
Page 93
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos
INSERT in my_collection
SHARDSHARD SHARD SHARD
Chunk = HASH(doc.name)

94. Confidential
RESTRICTED
Page 94
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos
INSERT in my_collection
SHARDSHARD SHARD SHARD
Chunk = HASH(doc.name)

95. Confidential
RESTRICTED
Page 95
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos
INSERT in my_collection
SHARDSHARD SHARD SHARD
Chunk = HASH(doc.name)

96. Confidential
RESTRICTED
Page 96
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos
INSERT in my_collection
SHARDSHARD SHARD SHARD
Chunk = HASH(doc.name)

97. Confidential
RESTRICTED
Page 97
_ Example
MongoDB Concepts
MongoDB
©2014AmadeusITGroupSA
{
“name”: “Winky”
}
mongos
INSERT in my_collection
SHARDSHARD SHARD SHARD
_ Remember, a shard is a
replica set, it is spread
over different machinesChunk = HASH(doc.name)

98. Confidential
RESTRICTED
Page 98
MongoDB Concepts
_ Data Replication
MongoDB
©2014AmadeusITGroupSA

99. Confidential
RESTRICTED
Page 99
MongoDB Concepts
_ Data Replication
• A write goes first to the primary of a shard
MongoDB
©2014AmadeusITGroupSA
NODE 1
Mongod
(primary)
Replica Set/Shard
write
NODE 2
Mongod
(secondary)

100. Confidential
RESTRICTED
Page 100
MongoDB Concepts
_ Data Replication
• A write goes first to the primary of a shard
• The primary writes the operation in a special collection called oplog
MongoDB
©2014AmadeusITGroupSA
NODE 1
Mongod
(primary)
Replica Set/Shard
oplog
write
NODE 2
Mongod
(secondary)

101. Confidential
RESTRICTED
Page 101
MongoDB Concepts
_ Data Replication
• A write goes first to the primary of a shard
• The primary writes the operation in a special collection called oplog
• Secondaries periodically poll the oplog to get the latest updates
MongoDB
©2014AmadeusITGroupSA
NODE 1
Mongod
(primary)
Replica Set/Shard
oplog
write
NODE 2
Mongod
(secondary)

102. Confidential
RESTRICTED
Page 102
MongoDB Concepts
_ Data Replication
• Oplog window
• Time difference between the first and the last entry in the oplog collection
• Replication Lag
• Time between the latest entry on the primary and the latest entry on a secondary
• Replication headroom
• Difference between Oplog Window and Replication Lag
MongoDB
©2014AmadeusITGroupSA

103. Confidential
RESTRICTED
Page 103
MongoDB Concepts
_ Journaling
• When running with journaling, MongoDB stores and applies write
operations in memory and in the on-disk journal before the changes are
present in the data files on disk
• MongoDB journals operations by batch (called group commits), the size is
configurable
• A group commit must block all writers during the commit
• In case of a crash, mongod uses the journal to warm-up
MongoDB
©2014AmadeusITGroupSA

104. Confidential
RESTRICTED
Page 104
MongoDB Concepts
_ Chunk splitting
• Each mongos knows how much it writes to each chunk
• When a mongos thinks a chunk needs to be split (=reached the max chunk
size), it queries the related mongod and check the chunk size. If needed, it
will start the balancer (if enabled)
MongoDB
©2014AmadeusITGroupSA

105. Confidential
RESTRICTED
Page 105
MongoDB Concepts
_ When do we move a chunk ?
MongoDB
©2014AmadeusITGroupSA

106. Confidential
RESTRICTED
Page 106
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
MongoDB
©2014AmadeusITGroupSA

107. Confidential
RESTRICTED
Page 107
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA

108. Confidential
RESTRICTED
Page 108
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2

109. Confidential
RESTRICTED
Page 109
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
“I want to move
this chunk”

110. Confidential
RESTRICTED
Page 110
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
Global lock, no other
chunk can be moved

111. Confidential
RESTRICTED
Page 111
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
“Please move chunk
4 to shard 2”

112. Confidential
RESTRICTED
Page 112
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
Please poll this list
of documents {..}

113. Confidential
RESTRICTED
Page 113
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
Polling..

114. Confidential
RESTRICTED
Page 114
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
Block read/writes

115. Confidential
RESTRICTED
Page 115
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
Block read/writes
Update chunk map

116. Confidential
RESTRICTED
Page 116
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
Connects with
outdated map

117. Confidential
RESTRICTED
Page 117
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2
Get new map
2

118. Confidential
RESTRICTED
Page 118
MongoDB Concepts
_ When do we move a chunk ?
• A shards holds too many chunks
• A new shard is added
MongoDB
©2014AmadeusITGroupSA
mongos
SHARD
2
SHARD
1config
mongos
1 1 1 1 2 21 1 1 1 2 2 2

119. Confidential
RESTRICTED
Page 119
MongoDB Concepts
_ Scalability
• If we need more nodes, we add more shards (done through one of the
mongos running on NoSQL tier)
• When you add a shard to a sharded cluster, you affect the balance of
chunks among the shards. The balancer will begin migrating chunks so
that the cluster will achieve balance
• The balancer has an impact on performance
• It moves one chunk at a time
MongoDB
©2014AmadeusITGroupSA

120. Confidential
RESTRICTED
Page 120
MongoDB Concepts
_ Automatic Recovery
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
mongos

121. Confidential
RESTRICTED
Page 121
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
mongos

122. Confidential
RESTRICTED
Page 122
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
PrimarySecondary
mongos

123. Confidential
RESTRICTED
Page 123
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
PrimarySecondary
mongos

124. Confidential
RESTRICTED
Page 124
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
PrimarySecondary
mongos
Write outage
(read depends on configuration)

125. Confidential
RESTRICTED
Page 125
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
PrimarySecondary
mongos
Write outage
(read depends on configuration)
Election

126. Confidential
RESTRICTED
Page 126
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
PrimarySecondary
mongos
Write outage
(read depends on configuration)
Primary

127. Confidential
RESTRICTED
Page 127
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
PrimarySecondary
mongos
Primary

128. Confidential
RESTRICTED
Page 128
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recoverymongos

129. Confidential
RESTRICTED
Page 129
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recovery
Secondary
mongos 1. Read from journal
Primary

130. Confidential
RESTRICTED
Page 130
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recovery
Secondary
mongos
Primary
Secondary

131. Confidential
RESTRICTED
Page 131
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
PrimarySecondary
mongos Sends its priority in
the heartbeat. In
case it is the
highest…
Secondary
Primary

132. Confidential
RESTRICTED
Page 132
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recovery
Primary
mongos
…self stepdown
Secondary

133. Confidential
RESTRICTED
Page 133
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
RecoveryPrimary
Secondary
mongos

134. Confidential
RESTRICTED
Page 134
MongoDB Concepts
_ Automatic Recovery
• Mongod-mongod
heartbeat checks if the
processes are up and
contains the priority of
the mongod
• Mongos-mongod
heartbeat checks the role
of the processes
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
RecoveryPrimary
Secondary
mongos
isMaster()

135. Confidential
RESTRICTED
Page 135
MongoDB Concepts
_ Rollback files
• A rollback reverts write operations on a former primary when the member
rejoins its replica set after a failover. A rollback is necessary only if the
primary had accepted write operations that the secondaries had
not successfully replicated before the primary stepped down
• When a rollback does occur, administrators must decide whether to apply
or ignore the rollback data.
• Note: Rollback files can be avoided if the application uses replica
acknowledged write concern (=write successful only when written on
replica)
MongoDB
©2014AmadeusITGroupSA

136. Confidential
RESTRICTED
Page 136
MongoDB Concepts
_ Full resync
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recoverymongos 1. Read from journal

137. Confidential
RESTRICTED
Page 137
MongoDB Concepts
_ Full resync
• In case the down time
is higher than the
oplog window, a full
resync is needed
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recoverymongos 1. Read from journal

138. Confidential
RESTRICTED
Page 138
MongoDB Concepts
_ Full resync
• In case the down time
is higher than the
oplog window, a full
resync is needed
• 1. Stop process
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recoverymongos 1. Read from journal

139. Confidential
RESTRICTED
Page 139
MongoDB Concepts
_ Full resync
• In case the down time
is higher than the
oplog window, a full
resync is needed
• 1. Stop process
• 2. wipe-out data
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recoverymongos 1. Read from journal

140. Confidential
RESTRICTED
Page 140
MongoDB Concepts
_ Full resync
• In case the down time
is higher than the
oplog window, a full
resync is needed
• 1. Stop process
• 2. wipe-out data
• 3. start process
MongoDB
©2014AmadeusITGroupSA
Primary
Secondary Arbiter
Heartbeat
Primary
Recoverymongos 1. Read from journal

141. Confidential
RESTRICTED
Page 141
Training MongoDB
_ MongoDB processes
• mongod (primary/secondary/arbiter), mongos, conf, agent, MMS
_ Replica Set / Shard, Shard Key, Shard Map
_ Database, Collection, Document
_ Replication (oplog)
_ Journaling
_ Capture/Restore
_ Chunk Move
_ Automatic Recovery / Election
Recap – What you should know by now
©2014AmadeusITGroupSA

142. Confidential
RESTRICTED
Workflows
4
©2014AmadeusITGroupSA

143. Confidential
RESTRICTED
Page 143
Workflows – discover topology
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
_ Mongos startsAgent
BE BE BEBE
mongos

144. Confidential
RESTRICTED
Page 144
Workflows – discover topology
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
_ Mongos starts
_ Reads Mongos.yml file
and connects to the
config server
Agent
BE BE BEBE
mongos

145. Confidential
RESTRICTED
Page 145
Workflows – discover topology
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
_ Mongos starts
_ Reads Mongos.yml file
and connects to the
config server
_ Retrieves the shard map
Agent
BE BE BEBE
mongos

146. Confidential
RESTRICTED
Page 146
Workflows – discover topology
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
_ Mongos starts
_ Reads Mongos.yml file
and connects to the
config server
_ Retrieves the shard map
_ Connects to all the
mongods
Agent
BE BE BEBE
mongos

147. Confidential
RESTRICTED
Page 147
Workflows – discover topology
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
_ Mongos starts
_ Reads Mongos.yml file
and connects to the
config server
_ Retrieves the shard map
_ Connects to all the
mongods
_ Retrieves the complete
list of processes for
each shard from the
mongods
Agent
BE BE BEBE
mongos

148. Confidential
RESTRICTED
Page 148
Workflows – discover topology
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
_ Mongos starts
_ Reads Mongos.yml file
and connects to the
config server
_ Retrieves the shard map
_ Connects to all the
mongods
_ Retrieves the complete
list of processes for
each shard from the
mongods
_ Retrieves the role of
each process is the
command isMaster()
Agent
BE BE BEBE
mongos

149. Confidential
RESTRICTED
Page 149
Workflows - write
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to store a
document
{..}

150. Confidential
RESTRICTED
Page 150
Workflows - write
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to store a
document
_ The document is routed
according to the shard key{..}

151. Confidential
RESTRICTED
Page 151
Workflows - write
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to store a
document
_ The document is routed
according to the shard key
_ Written only on primary
primary
{..}

152. Confidential
RESTRICTED
Page 152
Workflows - write
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to store a
document
_ The document is routed
according to the shard key
_ Written only on primary
_ And replicated to the
secondary asynchronously
primary secondary
{..}{..}

153. Confidential
RESTRICTED
Page 153
Workflows - write
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to store a
document
_ The document is routed
according to the shard key
_ Written only on primary
_ And replicated to the
secondary asynchronously
_ The client can decide to
wait for a successful write
in memory, on disk or on
the secondaries
primary secondary
{..}{..}

154. Confidential
RESTRICTED
Page 154
Workflows – read
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to read
documents
{..}

155. Confidential
RESTRICTED
Page 155
Workflows – read
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to read
documents
_ The query is routed
according to the shard key{..}
primary

156. Confidential
RESTRICTED
Page 156
Workflows – read
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to read
documents
_ The query is routed
according to the shard key
_ Depending on the query,
several mongods can be
targeted
primary primary
{..}{..}

157. Confidential
RESTRICTED
Page 157
Workflows – read
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to read
documents
_ The query is routed
according to the shard key
_ Depending on the query,
several mongods can be
targeted
_ Depending on the
configuration, mongos
could read on a primary or
a secondary
primary primary secondary
{..}{..} {..}

158. Confidential
RESTRICTED
Page 158
Workflows – read
MongoDB
©2014AmadeusITGroupSA
AS
OBE
(GOS, App Support)
App
Tier
NoSQL
Tier
MongoDB
Agent
BE BE BEBE
mongos
_ The BE wants to read
documents
_ The query is routed
according to the shard key
_ Depending on the query,
several mongods can be
targeted
_ Depending on the
configuration, mongos
could read on a primary or
a secondary
_ Mongos will process the
documents returned and
reply to the BE
primary primary secondary
{..}

159. Confidential
RESTRICTED
Operability & Monitoring
5
©2014AmadeusITGroupSA

160. Confidential
RESTRICTED
Page 160
Operability & Monitoring
_ Let’s check the new stuffs in the life cycle of a software in ADP
• Deployment
• ACO
• Logs
• Monitoring
• Automatic Alerts
• Operations
MongoDB
©2014AmadeusITGroupSA

161. Confidential
RESTRICTED
Page 161
Operability & Monitoring
_ New Pega flow defined for MongoDB clusters
_ Uses Puppet for installation automation
Deployment
©2014AmadeusITGroupSA

162. Confidential
RESTRICTED
Page 162
Operability & Monitoring
_ ULAM: mongos is deployed as a component of the OBE using it.
_ Later, MongoDB upgrades will be done by ULAM too (not initial
installation)
Deployment
©2014AmadeusITGroupSA

163. Confidential
RESTRICTED
Page 163
Operability & Monitoring
_ Called by ULAM during a load
_ New check to be created for Mongos
Application Checkout (ACO)
©2014AmadeusITGroupSA

164. Confidential
RESTRICTED
Page 164
Operability & Monitoring
_ Current design (might evolve)
• Mongos sends its logs a binary using Tracer, this binary sends all the logs
(Error + Info) to the application logserver in “mongos_link”
Logs
©2014AmadeusITGroupSA

165. Confidential
RESTRICTED
Page 165
Operability & Monitoring
_ New section in Infrastructure
Sentinel
©2014AmadeusITGroupSA

166. Confidential
RESTRICTED
Page 166
Operability & Monitoring
Sentinel
©2014AmadeusITGroupSA

167. Confidential
RESTRICTED
Page 167
Operability & Monitoring
_ Uniping
• New pinger for mongo (check the status of mongod, mongos and config
processes on database side)
_ Scheck
• Nothing new, it checks nodes via ssh
_ Topx
• Nothing new, it monitors memory
_ YALB
• Monitors logs and raises specialized IRs for mongos on application side
MongoDB
©2014AmadeusITGroupSA

168. Confidential
RESTRICTED
Page 168
Operability & Monitoring
_ Trends
• Tracks IRs for Ops
MongoDB
©2014AmadeusITGroupSA

169. Confidential
RESTRICTED
Page 169
Operability & Monitoring
_ New links to MongoDB OSAGs/OPROCs
MongoDB
©2014AmadeusITGroupSA

170. Confidential
RESTRICTED
Page 170
Operability & Monitoring
_ OSAG
• Operators and System Administrators Guide
• It contains OPROCs, POCs, escalation procedures, Impact & Recovery of
configuration changes, Setup procedures and much more!
MongoDB
©2014AmadeusITGroupSA

171. Confidential
RESTRICTED
Page 171
Operability & Monitoring
_ For GOS
• Escalation Procedure
• The link will be in each IR
• We don’t expect any action from GOS on MongoDB side, recovery
procedures are fully automated. The actions available in Workbench can be
used for test purposes
MongoDB
©2014AmadeusITGroupSA

172. Confidential
RESTRICTED
Page 172
Operability & Monitoring
_ Workbench
MongoDB
©2014AmadeusITGroupSA

173. Confidential
RESTRICTED
Page 173
Operability & Monitoring
MongoDB
©2014AmadeusITGroupSA

174. Confidential
RESTRICTED
Page 174
Operability & Monitoring
MongoDB
©2014AmadeusITGroupSA
_ NoSQL tab

175. Confidential
RESTRICTED
Page 175
Operability & Monitoring
MongoDB
©2014AmadeusITGroupSA
_ NoSQL tab
_ Cluster status

176. Confidential
RESTRICTED
Page 176
Operability & Monitoring
MongoDB
©2014AmadeusITGroupSA
_ NoSQL tab
_ Cluster status
_ Node status

177. Confidential
RESTRICTED
Page 177
Operability & Monitoring
MongoDB
©2014AmadeusITGroupSA
_ NoSQL tab
_ Cluster status
_ Node status
_ Process Status

178. Confidential
RESTRICTED
Page 178
Operability & Monitoring
MongoDB
©2014AmadeusITGroupSA
_ NoSQL tab
_ Cluster status
_ Node status
_ Process Status
_ Shard
distribution

179. Confidential
RESTRICTED
Page 179
Operability & Monitoring
_ Start/stop of a Secondary: Right click on the mongod process
_ Stepdown a primary: Right click on the mongod process
MongoDB
©2014AmadeusITGroupSA

180. Confidential
RESTRICTED
Page 180
Operability & Monitoring
_ Shard distribution
MongoDB
©2014AmadeusITGroupSA

181. Confidential
RESTRICTED
Page 181
Operability & Monitoring
_ Shard distribution
MongoDB
©2014AmadeusITGroupSA
Number of primaries

182. Confidential
RESTRICTED
Page 182
Operability & Monitoring
_ Shard distribution
MongoDB
©2014AmadeusITGroupSA
Number of primaries
Primary

183. Confidential
RESTRICTED
Page 183
Operability & Monitoring
_ Shard distribution
MongoDB
©2014AmadeusITGroupSA
Number of primaries
Primary Secondary

184. Confidential
RESTRICTED
Page 184
Operability & Monitoring
_ Shard distribution
_ Highlight of the node hosting secondary processes for the
selected node
MongoDB
©2014AmadeusITGroupSA
Number of primaries
Primary Secondary

185. Confidential
RESTRICTED
You can follow us on:
AmadeusITGroup
amadeus.com/blog
amadeus.com
Thank you
©2014AmadeusITGroupSA