High level overview of Mongoose and lessons learned from a year managing a popular NodeJS module.

1. Lessons in Open Source from the
MongooseJS ODM
Valeri Karpov
Software Engineer, MongoDB
www.thecodebarbarian.com
www.slideshare.net/vkarpov15
github.com/vkarpov15
@code_barbarian

2. *
Who Am I?
•NodeJS Engineer at MongoDB
•Maintainer of mongoose ODM
•Former CTO, LevelUp
•Hacker in Residence at BookaLokal
•Author of Professional AngularJS

3. *
Talk Overview
•How I got into mongoose
•High-level overview of how mongoose works
•Recent new features
•What I’m working on now
•Lessons learned from my first year

4. *
What is Mongoose?
•MongoDB ODM for NodeJS
•Schema validation for schema-less database
•Neat syntax for building MongoDB operations
•Very active module:
• 400k downloads/month
• 7000 GitHub stars
• 5-20 GitHub issue notifications/day

5. *
How I got into Mongoose
•Aug 2012: work with janky C++ event loop in HFT
•Nov 2012: start working on fashion startup
•Dec 2012: discover mongoose, cut SLOC in half
•Jan 2013: Leave HFT shop
•Apr 2013: Write original MEAN stack blog post
•Aug 2013: Leave startup land and go to MongoDB

6. *
How I got into Mongoose
•Dec 2013: Aaron Heckmann leaves MongoDB
•Apr 2014: Handoff over Twitter

7. *
How does Mongoose Work?
•Core features:
• Schema validation
• Syntactic sugar for persisting documents
• Hooks

8. *
Part 1: Mongoose Schema Validation
•MongoDB is schema-less...
•But you don’t want end users to store arbitrary data

9. *
Mongoose Schema Validation

10. *
Part 2: Syntactic Sugar for Updates
•MongoDB updates look like this:
•
•Do I have to manually build the update part?

11. *
Object.defineProperty()
•Current JavaScript spec lets you hook into property
access

12. *
Syntactic Sugar for Updates
•General idea: define mongoose schema
● When creating doc, loop over all schema paths
● For each path, use Object.defineProperty()
● Object.defineProperty() setter converts:
● Into:
● save() applies all updates

13. *
Part 3: Hooks / middleware
•Schema-level business logic
•Suppose you want to:
• Add a timestamp every time you save()
• Add your own caching layer to queries
• Log every query based on config file setting

14. *
Hooks / middleware
•Schema-level business logic

15. *
Hooks => Plugins
•Higher-level concept of “plugins”
•External hooks

16. *
Recent Additions
•Mongoose 4.0 released March 25
•Sweet new features:
• Query middleware
• Schema validation in the browser
• Validators on update()

17. *
What I’m Working On Now
•Promises: OOP for async operations
•Single biggest source of mongoose feature requests

18. *
Promises Explained
•General idea: users don’t like nested callbacks
•See Callback Hell is a Myth

19. *
Promises Explained
•Promises provide flatter structure

20. *
This Sounds Great But...
•The promises space is wildly fragmented
•Ask 10 users, get 10 different favorite promises
libraries - Bluebird, Q, RSVP, When, native ES6
•Mongoose promises feature requests are usually
• Use promises library X natively
• Support promises library Y’s feature Z

21. *
General Solution
•Mongoose 4 uses mpromise natively
•mpromise designed for ease rather than features or
performance, minimal adoption outside of mongoose
•Considering deprecating native promise support
•For now, planned solution is to allow users to specify
promise library of choice

22. *
Challenges
•Hooks and promises are confusing
•Promises need to be resolved
•Should they be resolved before or after post hooks?
•If after, hooks library needs to handle resolving!

23. *
Final Bit: Lessons Learned
•Linus’ Law:
•“Given a large enough beta-tester and co-developer
base, almost every problem will be characterized
quickly and the fix will be obvious to someone”
•Open source code is great because of the combined
input of the entire user base

24. *
Final Bit: Lessons Learned
•When using mongoose, you get contributions from a
lot of smart people
● The value of an OS module is (usually) proportional
to its user base

25. *
Final Bit: Lessons Learned
•Putting your code in the open makes you go the extra
mile to make it awesome
•And sometimes you have to eat some humble pie

26. *
First Rule in Running an Open Source
Module
•Always be responsive
•Nothing is worse than opening a github issue and not
having anybody touch it for a month
•Worse: somebody responds and says this library is no
longer maintained

27. *
Second Rule: Lead with a Clear
Vision
•Not necessarily in conflict with the “get people
involved”
•Easy for a module to become bloated
•One-off contributors aren’t as invested in the module
•Leads to unclear interfaces
•For instance, yeoman Ionic generator

28. *
ABD: Always Be Dogfooding
•Why I work on BookaLokal, mongoose plugins, etc.
•“Nobody should start to undertake a large project. You
start with a small trivial project, and you should never
expect it to get large... If it doesn't solve some fairly
immediate need, it's almost certainly over-designed... You
need to get something half-way useful first, and then
others will say "hey, that almost works for me", and they'll
get involved in the project.” - Linus Torvalds

29. *
Thanks for Listening!
•Slides on:
• Twitter: @code_barbarian
• Slideshare: slideshare.net/vkarpov15
•Mongoose on Github