This document outlines a 6-layer data model for collecting, storing, analyzing, and gaining insights from data. The 6 layers are: 1) collect and monitor raw data from both internal and external sources, 2) move, transform, and store the data, 3) discover and explore the data through search and visualization tools, 4) aggregate the data through feature stores and metrics, 5) build models and run experiments using machine learning, and 6) generate insights to inform actions.

1. Data Model
Chu-Cheng Hsieh

2. L6:Insight
L5: Model
&Experiments
L4: Aggregate
L3: Discover & Explore
L2: Move, Transform, Store
L1: Collect & Monitoring

3. L1: Collect &
Monitoring
Connect to data
source
Log & record
internal and
external events
Data quality
checks
Missing values
Abnormal values
(schema
evaluation)
Inconsistency
(versioning)

4. L2: Move, Transform & Store
Controlling cost: data
Storage, computation (ETLs)
From Unstructured to
Structured data
Metadata Generation

5. L3: Discover & Explore
DATA SEARCH &
DISCOVERY PLATFORM
VALIDATION VISUALIZATION
(DASHBOARD)

6. L4: Aggregate Layer
FEATURE STORES
(OFFLINE & STREAMING)
METRICS AGGREGATION
FUNCTION (REPORTS)

7. L5: Model &Experiments
MACHINE LEARNING
(FORECASTING, OUTLIER
DETECTION, ETC.)
A/B TESTS (EXPERIMENTS) AI APPLICATIONS
(PERSONALIZED
MARKETING, ETC.)

8. L6: Insight (& Act)
EXPLAIN ANALYTICS ACTIONS BASED ON
INSIGHTS