This document discusses using MongoDB for lightweight business intelligence (BI). It begins by introducing the author and their experience with MongoDB. Traditional SQL databases are described as the "wrong hammer" for agile BI needs. The document then outlines an approach using MongoDB for BI that is more iterative and adaptable compared to traditional enterprise BI systems. Examples from Trunk Club are provided that demonstrate using MongoDB to power applications for real-time analytics and decision support.

1. LIGHTWEIGHT
BUSINESS INTELLIGENCE
WITH MONGODB
COREY EHMKE
SENIOR ENGINEER, TRUNK CLUB
NOVEMBER 2012

2. WHO AM I?
A DEVELOPER WITH NEARLY 20 YEARS OF EXPERIENCE
AN ACTIVE OPEN SOURCE CONTRIBUTOR
A VETERAN OF THE WEB APPLICATION PLATFORM WARS
SERVING AS SENIOR ENGINEER AT TRUNK CLUB

3. MY INTRODUCTION
TO MONGODB
BUILDING A PPC LANDING PAGE HOSTING PLATFORM
INTERACTION-LEVEL ANALYTICS
A/B AND MULTIVARIATE TESTING
DYNAMIC PAGE CONTENT
REQUIREMENT FOR REAL-TIME REPORTING

4. DOES THIS SOUND FAMILIAR?
MODELS CONSTANTLY EVOLVING IN RESPONSE TO ILL-
DEFINED REQUIREMENTS AND CHANGING NEEDS
POLYMORPHIC REPRESENTATIONS OF USER
INTERACTIONS
VERSIONING TO SUPPORT MULTIVARIATE TESTING
REQUIREMENTS
NEED TO QUICKLY SCALE UP IN RESPONSE TO DEMAND

5. TRADITIONAL SQL:
THE WRONG HAMMER

6. TRADITIONAL SQL:
THE WRONG HAMMER
BIG DATA = PAINFUL MIGRATIONS
SCRIPTS TO MANAGE DATA TRANSFORMATIONS
LOGIC EMBEDDED IN YOUR DATA STORE
SQL TRICKS TO WORK AROUND RDBMS LIMITATIONS
CHALLENGE OF SCALING UP A SQL DATABASE

7. JAMIE ZAWINSKI
“YOU NEVER KNOW WHAT THE DESIGN IS
UNTIL THE PROGRAM IS DONE.”

8. IN SHORT...
TRADITONAL ORMS AND SQL
JUST DON’T FEEL AGILE.

9. MONGODB TO THE RESCUE
DOCUMENT MODEL
EMBEDDED COLLECTIONS
AGGREGATION FRAMEWORK
BUILT-IN VERSIONING
MAP REDUCE
SCALABILITY WITHOUT TEARS

10. BUSINESS INTELLIGENCE

11. WHAT IS BUSINESS INTELLIGENCE?
COLLECTION, MAINTENANCE, AND ORGANIZATION OF
MISSION-CRITICAL KNOWLEDGE
HISTORICAL VIEW OF BUSINESS OPERATIONS
TOOLS TO SUPPORT DECISION MAKING

12. STEP ONE:
THE NAÏVE APPROACH
REPORTING OUT OF THE TRANSACTIONAL DATABASE
RAW SQL EMBEDDED IN YOUR CODEBASE
GRANTING DB ACCESS TO STAKEHOLDERS

13. THE NAÏVE APPROACH:
SHORTCOMINGS
FIGHTING THE SCHEMA
POOR PERFORMANCE
IMPACT ON PRODUCTION RESOURCES

14. STEP TWO:
THE ENTERPRISE APPROACH
BI DATABASE DISTINCT FROM TRANSACTIONAL DB
NIGHTLY EXTRACT, TRANSFORM, LOAD OPERATION (ETL)
SCHEMA DESIGNED FOR REPORTING
SEPARATE HARDWARE AND SOFTWARE STACK
COMBINATION OF STATIC AND DYNAMIC REPORTS

15. THE ENTERPRISE APPROACH:
SHORTCOMINGS
24 HOUR DELAY IN INFORMATION
EXPENSIVE TO CONFIGURE AND MAINTAIN
REQUIRES HIGHLY SPECIALIZED RESOURCES
WATERFALL DEVELOPMENT APPROACH
MAKES IT HARD TO CHANGE YOUR MIND OR ADAPT
ENTERPRISEY

16. LIGHTWEIGHT
BUSINESS INTELLIGENCE

17. REDEFINING THE BI APPROACH
PROVIDE NEAR-REAL-TIME DATA TO SUPPORT DECISIONS
LEVERAGE EXISTING INFRASTRUCTURE
SUPPORT ITERATIVE, AGILE METHODOLOGIES
USE EXISTING SOFTWARE DEVELOPMENT RESOURCES

18. INHERENT ADVANTAGES
OF MONGODB IN BI
SUPPORTS AN AGILE APPROACH TO DEVELOPMENT
FLEXIBLE AND DYNAMIC SCHEMAS
SUPPORT FOR NATIVE DATATYPES
POWERFUL QUERYING AND AGGREGATION
FAST AND PERFORMANT
EASY TO SCALE UP

19. START BY ASKING QUESTIONS
INVOLVE STAKEHOLDERS
DETERMINE KPI’S
DEFINE THE QUESTIONS
FIND THE ANSWERS IN YOUR DATA

20. TURN INFERENCES INTO FACTS
WHAT FACTS ARE REQUIRED TO ANSWER BI QUESTIONS?
DETERMINE YOUR PROCESS FOR FACT EXTRACTION
DESIGN YOUR SCHEMA ACCORDINGLY
DE-NORMALIZE LIKE A BOSS
PROVIDE A CENTRAL, SINGLE SOURCE OF TRUTH

21. PRESENT ANSWERS
VISUALIZATION IS KEY
DASHBOARD DESIGN IS HARD
EMBRACE AGILITY
FOCUS ON CONTINUOUS COLLABORATION

22. PARALLEL DB DEPLOYMENT
MODERN FRAMEWORKS SUPPORT MULTIPLE ORMS
PEACEFUL COEXISTENCE WITH TRADITIONAL RDBMS
LOW-FRICTION, LOW-COST IMPLEMENTATION
SINGLE APPLICATION = CONSOLIDATED BUSINESS LOGIC

23. STATISTICAL MODELS
REPRESENT CONSOLIDATED DATA POINTS
BREAK OUT OF THE ACTIVE RECORD PATTERN
USE FACT TABLES THAT STRETCH THE TRADITIONAL
OBJECT RELATIONAL APPROACH
DENORMALIZED AND OPTIMIZED FOR REPORTING

24. STREAMING ETL
EVENT-TRIGGERED, CONTINUOUS DATA EXTRACTION
ALGORITHMS DEFINED IN CODE RATHER THAN SQL OR
ETL SCRIPTS
ALLOW RESOURCE-INTENSIVE CALCULATIONS TO
HAPPEN IN THE BACKGROUND
PROVIDE NEAR-REAL-TIME DATA
DELIVER ON THE PROMISE OF DECISION SUPPORT

25. LIGHTWEIGHT
BUSINESS INTELLIGENCE:
EXAMPLES

26. TRUNK CLUB
SERVICE-ORIENTED
STARTUP
SOFTWARE SYSTEMS
DESIGNED TO OPTIMIZE
AND STREAMLINE
BUSINESS PROCESSES
TECHNOLOGY IS A KEY
DIFFERENTIATOR
ENGINEERING PROVIDES
LEVERAGE FOR SCALING
THE BUSINESS

27. OUR ENGINEERING PHILOSOPHY
STARTUPS MAKE CRITICAL DECISIONS ON A DAILY BASIS
BETTER DATA LEADS TO BETTER DECISION-MAKING
OUR MISSION IS .: TO PROVIDE THIS DATA IN A TIMELY
AND USEFUL FORM

28. SOME CRITICAL DATA POINTS
MARKETING CAMPAIGN PERFORMANCE
MEMBER ON-BOARDING FUNNEL
TRUNK LIFECYCLE
STYLIST INTERACTIONS
INVENTORY PERFORMANCE

29. LEVIATHAN:
SWALLOW ALL THE THINGS

30. LEVIATHAN
RECORDS EVENTS FROM ALL APPLICATIONS
COLLECTS AND DISPLAYS REAL-TIME DATA
BROWSE, SEARCH, & DRILL INTERFACE
LONGITUDINAL ANALYSIS WITH DYNAMIC COHORTS

31. LEVIATHAN
Member Sales Sales
iOS App Web App iOS App
Member Ops
Website Web App
Leviathan
MongoDB

32. LEVIATHAN

33. LEVIATHAN

34. LEVIATHAN

35. LEVIATHAN

36. LEVIATHAN

37. PRODUCT INFORMANT:
REVEALING HIDDEN FACTS

38. PRODUCT INFORMANT
INVENTORY PERFORMANCE METRICS
EXTRACTS FACTS FROM A DOZEN RDBMS TABLES
DELIVERS ON DECISION SUPPORT

39. PRODUCT INFORMANT

40. PRODUCT INFORMANT

41. QUESTIONS & DISCUSSION

42. KEEP IN TOUCH!
COREY@TRUNKCLUB.COM
@BANTIK ON TWITTER/APP.NET
GITHUB.COM/BANTIK
HAPPY HOUR @ TRUNK CLUB
CHICAGO RUBY MEET-UPS