The perfect representation for data manipulation is based on collections, lists or maps but most applications are designed based on an object domain model. We solved this issue by implementing a KVM (Key Value Mapping) at the core level of our application. Data manipulation became very easy after implementing the bi-directional mapping of our entities to a key/value model. It took significant time before being able to leverage this new ecosystem and it started with a wide refactoring of the entities in order to fix our model key space.

1. DomainModel.stream()

2. DOMAINMODEL.STREAM()
https://github.com/lesfurets/model­map

3. WELCOME TO THE FURETS !

4. @GDIGUGLI – GILLES DI GUGLIELMO
Designer of sweet cooked software since 1999
Software Architect at LesFurets.com
@JUBAUDRY – JULIEN BAUDRY
Java Developer since 2007
Software Architect at LesFurets.com

8. 1 website, 5 Insurance Products : Car, Health, Home, Bike, Loan
1 codebase, 450k lines of code, 60k unit tests, 150 selenium tests
22 Developers, 2 DevOps, 3 Architects
10+ production servers
1 release per day
7 years of code history
2.5M quotes/year, 31% of market share

9. DOMAIN MODEL, WHAT DO YOU MEAN ?

10. MOVING THE ARCHITECTURE

11. ARCHITECTURE BUZZ
Copy on write
Immutable data
NoSQL
Clustered Data Base
Data streaming
Micro batching
Lambda architecture
Data on demand

12. WHAT YOU HAVE ...

13. WHAT YOU WANT …

14. YOU ARE IN TROUBLE ...

15. FRONT APP EVOLUTIONS
Your domain model is cloned using two
different languages.
Governance of the domain model and
change requests are difficult to
implement.
Development owners of the server and
front application are different.
UI Spaghetti code.

16. FRONT STORAGE EVOLUTIONS
Mapping entities in Cassandra is painful.
Don’t migrate your Entity/Relation model
using cut & paste!
You should probably flat your domain
model to fit a single row.
Infinite storage.

17. BACKOFFICE EVOLUTIONS
A lot of rework to implement.
Spark and ML­lib love data
vectors.
Spark read raw data, aggregate,
write them in microbatches.
No more (very long) nightly
batches.
Full history is available for
custom computations.

18. WHAT WE DO NOW ?

19. ADD KEYS EVERYWHERE !

20. A BIT OF DECORATION

21. LET'S START WITH THIS MODEL...

22. ... AND TRANSFORM IT TO A KEY­VALUE MODEL

23. USER MODEL

24. USER MODEL

25. USER MODEL

26. ACCOUNT MODEL

27. ACCOUNT MODEL

28. ACCOUNT MODEL

29. ACCOUNT MODEL

30. ACCOUNT MODEL

31. FROM DOMAIN MODEL TO KEY­VALUE MODEL
Step 1 : create one Enumeration with all keys
Step 2 : annotate your domain model
Step 3 : generate the key value model

32. LET'S SEE THE DOMAIN MODEL & THE KEY­VALUE MODEL

33. STEP 1 : ONE ENUMERATION WITH ALL KEYS
one literal for each key
position (optional) : used when collections are denormalized
tag (optional) : used to add meta­information

34. STEP 2 : ONE ANNOTATION TO ANNOTATE THE DOMAIN
one method to retrieve the FieldId
one method to retrieve the Constraint

35. STEP 3 : ANNOTATE YOUR DOMAIN MODEL
attribute, getter or setter can be annotated
getter and setter must follow Java naming conventions

36. STEP 4 : GENERATE THE KEY VALUE MODEL

37. ... JUST ONE LITTLE ISSUE.

38. ACCOUNT & USER ID
sampleModel.getUser().getId()

39. ACCOUNT & USER ID
sampleModel.getAccount().getId()

40. MANY KEY FOR A SINGLE ATTRIBUTE
Each key must have a unique path to retrieve its value
How to retrieve the right value for each field?
getAccount().getId()? getUser().getId()?

41. SOLUTION : CONSTRAINTS
One for each field, specify which getter to use

42. ADD A CONSTRAINT FOR EACH ANNOTATION
the code generator only use the path with the specified getter
each field is now mapped correctly

43. LET THE MAGIC BEGIN ...

44. CODE GENERATOR UNDER THE HOOD
Execution is done using a maven plugin
Annotated properties are processed using Java Reflection
Generate handwritten style code, easy to read
Enable JIT optimisations by combining lambdas and enumerations
Lightweight pattern, designed for performance

45. GENERATION OUTPUT (1/3)

46. GENERATION OUTPUT (2/3)
An Enumeration containing metadate : type, generics type

47. GENERATION OUTPUT (3/3)
a CSV containing each field, path, and value type

48. USAGES ARE UNLIMITED

49. YOU CAN USE JAVA 8 STREAM ON YOUR DOMAIN MODEL !
FieldModel#stream() : a stream of key/value pair (FieldId/Object)
FieldModel#parallelStream() : parallel stream and better performance
Easy use a filter/map/reduce operations on a domain model
Could you write a domain model diff tool now ?

50. WHERE WE USE IT AT LESFURETS.COM
Simple UI binding by replacing JSON model by a Dictionary (80% less code)
Use JSON Dictionary for client/server RPC
Batch and ETL with an email provider (90% less code)
Persistence in cassandra (from scratch with 200 lines of code for a whole model)
Injecting user data in selenium (40% less code)
Mocking our domain model in thousand of unit tests
... everything in iterative fashion during the past 2 years.

51. FUTURE AND IDEAS
Push as OSS our Cassandra serialisation code
Push some Spark samples
JEP 276: Dynamic Linking of Language­Defined Object Models

52. MODEL­MAP AVAILABLE ON GITHUB
http://github.com/lesfurets/model­map
Framework and examples
Apache Licence
Try it and contribute !

53. THANK YOU!