Introduce yourself - MongoDB Java driver - Who knows MongoDB? - Actually backwards compatibility is not the subject of this talk - this talk is about design
http://homepages.cwi.nl/~paulk/patents/isnot/node4.html - When getting any formal training in Software Delivery Lifecycle, might see this - get the impression that design is about turning requirements into tech specs - then it’s over and you get on to the fun coding
Whitepaper that spawned waterfall - “as much as possible” - the output of the stage is a set of documents - this is BUFD, and we hate it - did design die when waterfall died?
Did design die when waterfall became unpopular? - Evolutionary design - Minimum viable product
- Design is something YOU do every day - Seen this work at LMAX - no large documents, just well designed code - And when you’re doing library/API work, you MUST do design - I would use all the things I learnt at LMAX to improve the Java driver - This is something you’re all doing, but I want to expose it
- This is a journey you’ve all been on - Sharing the pain via the example of the MongoDB Java driver
- 9 months ago, joined 10gen Not only going to maintain / improve Java driver, actually want to re-write it what developer doesn’t want that? we have an existing driver, we have a server spec
- we’re going to re-write it with generics - fluent API - brave new world, Java 8 - Async - Modern, Intuitive, Performant, Easy to Maintain
- Java versions - first obvious challenge - we don’t want to support two code bases - we don’t want a big bang upgrade - we have a lot of users - we don’t know who they are - we need them to be able to continue without pain - how do we have new shiny API-ness with the old api intact as well? - Can you use both?
- what are our requirements? - what do we not like about the current driver? - what are we hoping to achieve? - encoders vs writers - migration - documentation - test coverage - lack of consistency - regression bugs - Lots of conversations with the C# guys - make it easy for ODMs
- Save Point - wrote down everything we were thinking about - what was on the table, what was off - what was important - what was bothering us
- (no need to translate the contents of the document) - this is our baseline - starting point - and now we had it, we could use it to measure X against Y - we could point people at it when they asked us what we were doing - we could use it to have conversations with other driver developers - turned it into a more succinct version on the wiki - series of talks - this is now out of date, it was a marker in time
- between drivers & shell, different terminology - within the driver: naming, ordering, formatting - distracting - Ubiquitous language
- Save Point - easier than expected - knew we wanted some - enforced in build, in jenkins - means we have bigger conversations than just the little things - can also highlight areas where the design is poor, e.g. long methods, complex if statements
Started off with over 6000 Checkstyle errors 200 FindBugs Nearly 100 PMD Errors Now we have zero YAY
One down! we could actually have conversations about the design, not about tabs vs spaces Ubiquitous language
- who are they? - what do they want? - how do they work? - novice or advanced
- this is hard because our software is free and open source
1. public api 2. need to be able to wrap 3. like me, make code pretty, design should be documented in the code, HABITABLE
- You’re not writing the new Java driver for you - talk to them - involve them - community - Tutorial / hack session - MUG - JUG - This talk
- did you notice I didn’t kill that monster earlier?
- Now we have design goals - And identified users - ...much easier to talk about architecture
- generated from our code - allows us to have conversations with other people - i.e. other drivers - different colours, different responsibilities
We’d been thinking the driver was “simple” Actually it turned out that we have a tiered architecture and that our different users will interact with it at different layers
- So for example the Scala driver would reuse the connection pooling - the protocol layer - and provide a Scala-looking API
Talking about architecture Focussing on Domain We’ve talked about creating a cleaner design Validated by the fact that it works for our users
Let’s talk about backwards compatibility
Given architecture Can develop strategy for backwards compatibility
- Could wrap the API - I think this will let the IDE help you migrate - And you can use both at the same time
- or we could treat the old API like any other third party library - this actually validates the design
- but we don’t know that we’ve killed this monster yet - how do we know we’ve succeeded?
- Our old unit tests - Our new unit/functional tests - Our acceptance tests - Scala tests - Morphia Tests - Other friendly test cases
- In order to kill it, we had to understand our design
So that was backwards compatibility, the title of this talk. Turned out it was relatively easy in our case: - defined design goals - understood users - outlined an architecture
- internal tools - run it against friendly code - ...and finally, run it in the wild
- we’ve found a way to be backwards compatible while moving forwards with the API - what should that look like? - everyone has an opinion, tastes differ - and actually it can be the last thing to change - ...except you need feedback - what’s wrong with the old one?
Since backwards compatibility affects the API, lets talk about the new API
Feedback is welcome, but would be great if you sent it to us
- looks the same
-take an example with more params - overloading is common in the driver - be more explicit
-take an example with more params - overloading is common in the driver - be more explicit
-take an example with more params - overloading is common in the driver - be more explicit
Take another example that has changed Filter then remove Can you see we’re starting to get the idea of terminators?
Let’s look at Find and Modify - filter again - why do you have to change the names?
We want the API to be consistent across operations
Stole this from Collections Stream from Java 8 Idea is to chain together operations bit of a shift in mindset for some Java people but DSLs & Java 8 will help move us this way
Lack of consistency: Well and truly dead
- Fluent - Explicit - Auto complete is important - Consistency
Public API - lots of decisions to take - need to get agreement across drivers - need feedback from users (old & new, real developers & driver developers) - Things will change - API is both the thing that cannot change very often but should be allowed to evolve - Last responsible moment? - Hack days, blogs, friendly users...
- Save Point - forced me to write against the API - changed it - got feedback
- This talk is part of the process - Get feedback from all three types of our users
- Design is Hard. but it is your job - You do it every day - Working Software over Comprehensive Documentation - Design is a journey, not a destination - YOu are not typists Enjoy yourself What would I do differently?
If no-one, ask them what they’re doing -are they here for design? - directly? - ODMs? - Spring Data?
Conclusion - it’s not about the destination, it’s the journey - it’s not about producing a document, or a tonne of documents, it’s about the questions you ask, the conversations you have - Design is Hard. But it is your job.