Presentation by Christophe Bogaert to Measurecamp London September 2016. Christophe discussed what makes consuming and analysing event-streams difficult, and outlined a number of techniques for overcoming those obstacles.

1. EVENT DATA MODELING
MEASURECAMP LONDON ‘16

2. MEASURECAMP LONDON ‘16
WHO’S CAPTURING ATOMIC DATA?
Who’s using GA Premium, Adobe, Snowplow, Segment, … to capture
atomic or event-level data?
How is the data made available, consumed, turned into insights?

3. MEASURECAMP LONDON ‘16
WE ALL LIKE ATOMIC DATA…
With current technologies, we can record all user interactions, across
all channels, store it in our own data warehouse, and join it with all
other datasets we have.
… BUT IT REMAINS HARD TO CONSUME

4. MEASURECAMP LONDON ‘16
EXAMPLE 1
Event stream:
‣ Pre-roll loaded, clicked, skipped, …
‣ Main video loaded, paused, …
‣ Interactions within the video
‣ Subscribe, like, share, comment, …
‣ Much, much more

5. MEASURECAMP LONDON ‘16
EXAMPLE 2
Event stream:
‣ Tutorial start, tutorial finish
‣ Start game, change difficulty
‣ Level up
‣ Purchase
‣ Invite friends
‣ Much, much more

6. MEASURECAMP LONDON ‘16
WHY IS IT HARD TO CONSUME?
Events need to be looked at in context, and in the right order, to
become valuable.
End users cannot be expected to do the complex transformations that
are required to draw insights from the atomic data.

7. “EVENT DATA MODELING IS THE PROCESS OF USING BUSINESS
LOGIC TO AGGREGATE AND TRANSFORM EVENT-LEVEL DATA TO
PRODUCE MODELED DATA THAT IS SIMPLER TO CONSUME”
DEFINITION

8. MEASURECAMP LONDON ‘16
EVENT DATA MODELING
BEFORE DATA MODELING
DATA IS IMMUTABLE
AND UN-OPINIONATED
AFTER DATA MODELING
DATA IS MUTABLE
AND OPINIONATED

9. MEASURECAMP LONDON ‘16
EVENT DATA MODELING
▸ ID stitching
▸ Macro events
▸ Units of work
▸ Sessions
▸ Users

10. THOUGHTS OR QUESTIONS?
WE’RE HIRING
JUNIOR DATA
ANALYSTS

11. MEASURECAMP LONDON ‘16
EVENT DATA PIPELINE
PROCESSINGCOLLECTION
REAL-TIME APPS
REAL-TIME DASHBOARDS
DATA EXPLORATION
PREDICTIVE MODELING
DATA
WAREHOUSE
WEB
APPS
SERVERS
3RD PARTY
IOT

12. MEASURECAMP LONDON ‘16
EVENT DATA PIPELINE
PROCESSINGCOLLECTION
REAL-TIME APPS
REAL-TIME DASHBOARDS
DATA EXPLORATION
PREDICTIVE MODELING
DATA
WAREHOUSE
WEB
APPS
SERVERS
3RD PARTY
IOT
MANY SOURCES

13. MEASURECAMP LONDON ‘16
EVENT DATA PIPELINE
PROCESSINGCOLLECTION
REAL-TIME APPS
REAL-TIME DASHBOARDS
DATA EXPLORATION
PREDICTIVE MODELING
DATA
WAREHOUSE
WEB
APPS
SERVERS
3RD PARTY
IOT
ONE PIPELINE
UNIFIED LOG, NO SILOS

14. MEASURECAMP LONDON ‘16
EVENT DATA PIPELINE
PROCESSINGCOLLECTION
REAL-TIME APPS
REAL-TIME DASHBOARDS
DATA EXPLORATION
PREDICTIVE MODELING
DATA
WAREHOUSE
WEB
APPS
SERVERS
3RD PARTY
IOT
VALIDATION ENRICHMENT DATA MODELING
ONE PIPELINE
UNIFIED LOG, NO SILOS

15. MEASURECAMP LONDON ‘16
EVENT DATA PIPELINE
PROCESSINGCOLLECTION
REAL-TIME APPS
REAL-TIME DASHBOARDS
DATA EXPLORATION
PREDICTIVE MODELING
DATA
WAREHOUSE
WEB
APPS
SERVERS
3RD PARTY
IOT
MANY CONSUMERS