Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Triggers In MongoDB
Similaire à Triggers In MongoDB (20)
Dernier
Dernier (20)
Triggers In MongoDB
- 1. Triggers in MongoDB Antonios Giannopoulos and Jason Terpko DBA’s @ Rackspace/ObjectRocket linkedin.com/in/antonis/ | linkedin.com/in/jterpko/ 1
- 3. Overview • What is a Trigger • Why is Useful • Application Triggers • Oplog Tailing • 3.6+ Streams • Use Cases www.objectrocket.com 3
- 4. What is a trigger? www.objectrocket.com 4 A trigger is a database object that is associated with a table. It will be activated when a defined action is executed for the table. Actions usually are: - INSERT - UPDATE - DELETE It can be invoked before or after the event.
- 6. Trigger example (BEFORE) www.objectrocket.com 6 *For Deposits an AFTER event trigger is preferred
- 10. Foreign Keys www.objectrocket.com 10 Foreign Keys may also considered as triggers, but less flexible
- 11. MongoDB Application Triggers • Implementation Options • Considerations • Examples www.objectrocket.com 11
- 12. Application Based Triggers www.objectrocket.com 12 • Maintained internally • Compatibility • Flexibility • Added Development Cycles Self-Implemented Open-Source Package • Publically maintained • Contributions and tested by a larger user base • Potentially restricted to specific versions, client and server
- 17. Oplog Tailing • The oplog • Tailable cursors • Replica Set • Sharded cluster • Advantages/Disadvantages/Co nsiderations www.objectrocket.com 17
- 18. The oplog www.objectrocket.com 18 Capped collection (FIFO) Fixed Size - oplogSizeMB (5% of free space is the default) Located under local.oplog.rs Holds every CRUD operation (Insert/Update/Delete/Commands) MongoDB Replication Idempotent by design
- 19. Anatomy of the local.oplog.rs www.objectrocket.com 19
- 20. Anatomy of the local.oplog.rs www.objectrocket.com 20
- 21. Anatomy of the local.oplog.rs www.objectrocket.com 21 ts: timestamp of the oplog entry t: election "term" h: unique hash v: version of the oplog op: Type of operation (insert/update/delete/commands) ns: Database & collection affected o: The new state of the document after performing the change o2: Contain the _id field of the affected document ui: Collection’s UUID wall: timestamp of the oplog entry fromMigrate : Sparse field, is true when the operation comes from balancing
- 22. Tailable cursor www.objectrocket.com 22 Equivalent to the tail Unix command with the -f option Remains open after the client exhausts the results in the initial cursor Ideal for capped collections (where indexes are not practical) MongoDB replication uses tailable cursors to read the oplog Initial scan is expensive – It scans the entire collection Available on the vast majority of drivers
- 25. Using Replica Set www.objectrocket.com 25 Primary’s and Secondary's oplog are identical A tailable cursor is enough Trigger action must loop through the primary
- 26. Using Sharded Cluster www.objectrocket.com 26 s1 s2 Oplog is not visible thought the mongos A tailable cursor per shard Adjust to topology changes Must filter balancer events Security issues Trigger action must loop through the mongos
- 27. Considerations www.objectrocket.com 27 Only AFTER INSERT/UPDATE/DELETE supported Rollback an operation Handle nodes rollbacks (Replica set state) Avoid replay the same operations Keep up with the replication pace (Dedicated members using replica set tags)
- 28. Considerations www.objectrocket.com 28 Apply before commit/commit errors (w:majority or w:n, n>1) Preserve the order of operations – sharded clusters only Filter out migration events (fromMigrate) – sharded clusters only Filter out update events on Orphans – sharded clusters only
- 29. Change Streams • Topology • Change Events • Examples • Sharded Clusters • Considerations www.objectrocket.com 29
- 30. What is a Change Stream? www.objectrocket.com 30 s1 s2 3) Resumable 2) Driver Supported 1) Real-time 4) Secure 5) Synchronized 6) Flexible (1) (2) (3) (4) (5) (6) X X
- 33. Event Document (Update - Full) www.objectrocket.com 33
- 37. Python Stream – watch() www.objectrocket.com 37
- 38. Flexible (Pipeline Stages) www.objectrocket.com 38 $match, $addFields $match, $project
- 39. Sharded Clusters www.objectrocket.com 39 s1 s2 3) Cold Shard(s) 2) Rate of Change 1) Coordination 4) Geographical Dist. 5) Orphans (1) (2) (3) (4) (5)
- 40. Additional Considerations www.objectrocket.com 40 • Majority Read Concern / Engine • Replica Member Availability / Arbiters • Oplog Size • Collection Dropped or Renamed (Invalidation) • Large Documents and Maximum Document Size
- 41. Use Case • Shard Key Analysis www.objectrocket.com 41
- 42. Use Cases www.objectrocket.com 42 Auditing Event Scheduler Caching Selective Replication Disaster Recovery …
- 43. Mongo to Mongo connector www.objectrocket.com 43 Reads the oplog on source and replays on destination source dest Tailable cursor Use Cases • Selective replication • Multisource replication • Disaster recovery • Heterogeneous Replication
- 44. Use Case: Shard Key Analysis www.objectrocket.com 44 Define a shard key is challenging: • Can break your application • Not an easy task to revert it (requires downtime) Shard Key limitations (two out of many): • Shard Key Value in a Document is Immutable. • NULL values are not allowed
- 45. Use case: Shard Key Analysis www.objectrocket.com 45 Query system.profile • Requires extra room as default is 1MiB • Adds overhead (extra writes) • Snapshot of operations Oplog or Streams • Already present • No extra overhead • Covers bigger duration
- 46. Shard Key Analysis - Oplog www.objectrocket.com 46
- 47. Shard Key Analysis - Streams www.objectrocket.com 47
- 50. www.objectrocket.com 50 We’re Hiring! Looking to join a dynamic & innovative team? https://www.objectrocket.com/careers/ or email careers@objectrocket.com
- 51. Thank you! Address: 401 Congress Ave Suite 1950 Austin, TX 78701 Support: 1-800-961-4454 Sales: 1-888-440-3242 www.objectrocket.com 51