Triggers in MongoDB

Triggers in MongoDB
Antonios Giannopoulos and Jason Terpko
DBA’s @ Rackspace/ObjectRocket
linkedin.com/in/antonis/ | linkedin.com/in/jterpko/
1

Introduction
www.objectrocket.com
2
Antonios Giannopoulos Jason Terpko

Overview
• What is a Trigger
• Why is Useful
• Application Triggers
• Oplog Tailing
• 3.6+ Streams
• Use Cases
3

What is a trigger?
4
A trigger is a database object that is associated with a table.
It will be activated when a defined action is executed for the table.
Actions usually are:
- INSERT
- UPDATE
- DELETE
It can be invoked before or after the event.

MySQL trigger definition
5

Trigger example (BEFORE)
6
*For Deposits an AFTER event trigger is preferred J

Trigger example (BEFORE)
7

Trigger example (AFTER)
8

Trigger example (AFTER)
9

Foreign Keys
10
Foreign Keys may also considered as triggers, but less flexible

MongoDB
Application
Triggers
• Implementation Options
• Considerations
• Examples
11

Application Based Triggers
12
• Maintained internally
• Compatibility
• Flexibility
• Added Development Cycles
Self-Implemented
Open-Source Package
• Publically maintained
• Contributions and tested by a larger user base
• Potentially restricted to specific versions, client and server

Self-Implemented (Before)
13

Self-Implemented (After)
14

Python (mongotriggers)
15

Node.js (mongo-oplog)
16

Oplog Tailing
• The oplog
• Tailable cursors
• Replica Set
• Sharded cluster
• Advantages/Disadvantages/Co
nsiderations
17

The oplog
18
Capped collection (FIFO)
Fixed Size - oplogSizeMB (5% of free space is the default)
Located under local.oplog.rs
Holds every CRUD operation (Insert/Update/Delete/Commands)
MongoDB Replication
Idempotent by design

Anatomy of the local.oplog.rs
19

20

21
ts: timestamp of the oplog entry
t: election "term"
h: unique hash
v: version of the oplog
op: Type of operation (insert/update/delete/commands)
ns: Database & collection affected
o: The new state of the document after performing the change
o2: Contain the _id field of the affected document
ui: Collection’s UUID
wall: timestamp of the oplog entry
fromMigrate : Sparse field, is true when the operation comes from balancing

Tailable cursor
22
Equivalent to the tail Unix command with the -f option
Remains open after the client exhausts the results in the initial cursor
Ideal for capped collections (where indexes are not practical)
MongoDB replication uses tailable cursors to read the oplog
Initial scan is expensive – It scans the entire collection
Available on the vast majority of drivers

Trigger implementation
23

Trigger implementation
24

Using Replica Set
25
Primary’s and Secondary's oplog are identical
A tailable cursor is enough
Trigger action must loop through the primary

Using Sharded Cluster
26
s1 s2
Oplog is not visible thought the
mongos
A tailable cursor per shard
Adjust to topology changes
Must filter balancer events
Security issues
Trigger action must loop through
the mongos

Considerations
27
Only AFTER INSERT/UPDATE/DELETE supported
Rollback an operation
Handle nodes rollbacks (Replica set state)
Avoid replay the same operations
Keep up with the replication pace (Dedicated members using replica
set tags)

Considerations
28
Apply before commit/commit errors (w:majority or w:n, n>1)
Preserve the order of operations – sharded clusters only
Filter out migration events (fromMigrate) – sharded clusters only
Filter out update events on Orphans – sharded clusters only

Change
Streams • Topology
• Change Events
• Examples
• Sharded Clusters
• Considerations
29

What is a Change Stream?
30
s1 s2
3) Resumable
2) Driver Supported
1) Real-time
4) Secure
5) Synchronized
6) Flexible
(1)
(2)
(3)
(4)
(5)
(6)
X X

Event Document (Insert)
31

Event Document (Update)
32

Event Document (Update - Full)
33

Event Document (Replace)
34

Event Document (Remove)
35

Event Document (Invalidate)
36

Python Stream – watch()
37

Flexible (Pipeline Stages)
38
$match, $addFields
$match, $project

Sharded Clusters
39
s1 s2
3) Cold Shard(s)
2) Rate of Change
1) Coordination
4) Geographical Dist.
5) Orphans
(1)
(2)
(3)
(4)
(5)

Additional Considerations
40
• Majority Read Concern / Engine
• Replica Member Availability / Arbiters
• Oplog Size
• Collection Dropped or Renamed (Invalidation)
• Large Documents and Maximum Document Size

Use Case
• Shard Key Analysis
41

Use Cases
42
Auditing
Event Scheduler
Caching
Selective Replication
Disaster Recovery
…

Mongo to Mongo connector
43
Reads the oplog on source
and replays on destination
source dest
Tailable cursor
Use Cases
• Selective replication
• Multisource replication
• Disaster recovery
• Heterogeneous
Replication

Use Case: Shard Key Analysis
44
Define a shard key is challenging:
• Can break your application
• Not an easy task to revert it (requires downtime)
Shard Key limitations (two out of many):
• Shard Key Value in a Document is Immutable.
• NULL values are not allowed

Use case: Shard Key Analysis
45
Query system.profile
• Requires extra room as default is 1MiB
• Adds overhead (extra writes)
• Snapshot of operations
Oplog or Streams
• Already present
• No extra overhead
• Covers bigger duration

Shard Key Analysis - Oplog
46

Shard Key Analysis - Streams
47

Questions?
48

Rate My Session
49

50
We’re Hiring!
Looking to join a dynamic & innovative
team?
https://www.objectrocket.com/careers/
or email careers@objectrocket.com

Thank you!
Address:
401 Congress Ave Suite 1950
Austin, TX 78701
Support:
1-800-961-4454
Sales:
1-888-440-3242
51

Triggers in MongoDB

Recommandé

Recommandé

Contenu connexe

Similaire à Triggers in MongoDB

Similaire à Triggers in MongoDB (20)

Plus de Antonios Giannopoulos

Plus de Antonios Giannopoulos (12)

Dernier

Dernier (20)

Triggers in MongoDB