Leboncoin, qui rassemble aujourd'hui 30 millions d'utilisateurs uniques chaque mois, a dû faire considérablement évoluer ses pratiques “data” depuis son lancement en 2006 pour accompagner la croissance vertigineuse de sa plateforme et ses équipes. Entre galères, zombies, data engineering et data mesh, ce ne fut pas de tout repos !

1. D'un hack au datamesh
L'évolution du Data Engineering chez leboncoin

2. Parcours d'intégration
Lancement du site
leboncoin
2006
Basé sur le modèle Blocket,
notre “grande soeur” Suédoise
En seulement 2 ans,
leboncoin est rentable
2008
10 Millions d’annonces en ligne
Ouverture de notre premier
centre de télévente à
Montceau-les-Mines
leboncoin devient le
leader de la petite
annonce en ligne (1)
Élue 3ème marque préférée des Français
(1)
2012
LBC entre au top 10 des sites français
en termes d’audience (MAU) (2)
2014
73% des Français
utilisent leboncoin(3)
1 milliard
d’annonces
postées sur leboncoin
2017
Lancement
de l’app iOS
leboncoin élu
2016
2018
leboncoin est
entièrement
responsive
2010
4ème entreprise la plus
utile en France(4)
Lancement du
paiement P2P
2020
#2 au classement
Great Place To Work (5)
Lancement de la
livraison
Employé
# 1400

3. ● Lead Architect
● Leboncoin depuis 2012
● Dev BI => Data engineer => Architect
Simon Maurin,
@MaurinSimon
● Lead Data Engineer
● Leboncoin en 2013-2017, et depuis 2020
● Chef de projets BI => Responsable Data Eng
=> Lead Data engineer
Stéphanie Baltus-Bergamo
@steph_baltus

4. Chapter I :
The Beginning
(2012)

7. ETL
BI
SOLUTION
DWH
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
CACHE

8. ETL
BI
SOLUTION
DWH
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
CACHE

9. ETL
BI
SOLUTION
DWH
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
CACHE

10. Can’t Scale Enough !
(And more are coming)

12. Chapter II :
Data Platform
(2015-2018)

13. Plateforme
(ré-utilisabilité)
Dépasser les problèmes
d’échelle
Augmenter la
Résilience du système
La Data Platform en 3 Objectifs :

14. Plateforme
(ré-utilisabilité)
Dépasser les problèmes
d’échelle
Augmenter la
Résilience du système
La Data Platform en 3 Objectifs :
➔ Scaling Horizontal
➔ Elasticité

15. Plateforme
(ré-utilisabilité)
Dépasser les problèmes
d’échelle
Augmenter la
Résilience du système
La Data Platform en 3 Objectifs :
➔ Scaling Horizontal
➔ Elasticité

16. Build
Data Driven Products

17. Plateforme
(ré-utilisabilité)
Dépasser les problèmes
d’échelle
Augmenter la
Résilience du système
La Data Platform en 3 Objectifs :
➔ Scaling Horizontal
➔ Elasticité
➔ Découplage
➔ Data Lake (RAW data)

18. Plateforme
(ré-utilisabilité)
Dépasser les problèmes
d’échelle
Augmenter la
Résilience du système
La Data Platform en 3 Objectifs :
➔ Scaling Horizontal
➔ Elasticité
➔ Découplage
➔ Data Lake (RAW data)

19. Pipeline orchestrator : Airflow

20. Observabilité & Alerting

21. Eventual Consistency

22. Idempotence
A + A = A

23. Idempotence & Commutativité
A + A = A
B + A = A + B

24. Idempotence & Commutativité => Rejouabilité
A + A = A
A + A + B + A + B + A = A + B
B + A = A + B

25. Idempotence & Commutativité : How To ?
https://gist.github.com/sbaltus/25d599b8ea52142f5691bb98aaf27e36

26. Idempotence & Commutativité : DEMO
https://gist.github.com/sbaltus/25d599b8ea52142f5691bb98aaf27e36
A + A + B = A + B
B + A = A + B

27. https://gist.github.com/sbaltus/25d599b8ea52142f5691bb98aaf27e36

28. Plateforme
(ré-utilisabilité)
Dépasser les problèmes
d’échelle
Augmenter la
Résilience du système
La Data Platform en 3 Objectifs :
➔ Scaling Horizontal
➔ Elasticité
➔ Découplage
➔ Data Lake (RAW data)
➔ Admin UI & Auto-Healing
➔ Observabilitité
➔ Replayability
➔ Eventual Consistency

29. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
EL
EL
EL

30. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
EL
EL
EL

31. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
EL
EL
EL

32. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
EL
EL
EL

33. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
EL
EL
EL

34. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
EL
EL
EL

36. NOPE !

37. Feature Teams
& Microservices
(2017)

38. UI
ALL BUSINESS
LOGIC
DATA ACCESS
LAYER
UI
MICROSERVICE
DOMAIN A
MICROSERVICE
DOMAIN B
MICROSERVICE
DOMAIN C
MONOLITH MICRO-SERVICES
EVENT BUS

39. UI
ALL BUSINESS
LOGIC
DATA ACCESS
LAYER
UI
MICROSERVICE
DOMAIN A
MICROSERVICE
DOMAIN B
MICROSERVICE
DOMAIN C
MONOLITH MICRO-SERVICES
EVENT BUS

40. Entente Cordiale
(et RFCs)

41. Event Stream Boy Scout Rules @Leboncoin :

42. Event Stream Boy Scout Rules @Leboncoin :
➔ Producer Ownership ! Domain owner :
◆ Runs producer code
◆ Chooses scope (public | private)

43. Event Stream Boy Scout Rules @Leboncoin :
➔ Producer Ownership ! Domain owner :
◆ Runs producer code
◆ Chooses scope (public | private)
➔ Event Bus as API & schema as contract

44. Event Stream Boy Scout Rules @Leboncoin :
➔ Producer Ownership ! Domain owner :
◆ Runs producer code
◆ Chooses scope (public | private)
➔ Event Bus as API & schema as contract
➔ For all public event streams :
● Avro
● Backward compatibility
● Common skeleton
● Maximized-minimal content
● Documentation

45. Event Stream Boy Scout Rules @Leboncoin :
➔ Producer Ownership ! Domain owner :
◆ Runs producer code
◆ Chooses scope (public | private)
➔ Event Bus as API & schema as contract
➔ For all public event streams :
● Avro
● Backward compatibility
● Common skeleton
● Maximized-minimal content
● Documentation
➔ Entity naming & typing normalization
➔ Kafka configuration standards

46. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
leboncoin.fr
PHP / C
MONOLITH
ERP
PHP
MONOLITH
EL
EL
EL

47. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
EVENT BUS
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
ARCHIVER

49. NOPE !

50. I Want My Money Back !
(2018)

51. Offline
Data
Online
Data
Automated ML Pipeline
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
Data
Extraction ML
Model
Manual Experiment Steps
Data
Preparation
Model
Training
Model
Evaluation
Model
Analysis
Continuous Deployment
Model
Serving
Business
Monitoring
Data
Analysis
Data
Extraction
Data Engineering
51
1 2
3 4
Staging/Preprod/Prod Staging/Preprod/Prod

52. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
EVENT BUS
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
ARCHIVER

53. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
EVENT BUS
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
ARCHIVER
ML PIPELINE

54. DATALAKE
ETL
CRM
WEB
ANALYTICS
DATA API
EL INVENTORY
BI
SOLUTION
DWH
ORCHESTRATOR
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
MICRO
SERVICES
ARCHIVER
DISCOVERY
SERVICE
CACHE
ML PIPELINE
EVENT BUS

57. NOPE !

59. Build
Data Driven Products
at Company Scale

60. Chapter III :
DATA MESH
(Since 2019)

61. ORIGINES
How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh (Zhamak Dehghani)

64. Data Boy Scout Rules :
➔ Producer Ownership ! Domain owner :
◆ Runs producer code
◆ Chooses scope (public | private)
➔ Event Bus as API
➔ Datalake as API
➔ Schema as contract
➔ For all public event streams :
● Avro
● Backward compatibility
● Common skeleton
● Maximized-minimal content
● Documentation
➔ Entity naming & typing normalization
➔ Kafka configuration standards
➔ Privacy by design
◆ Criticity Tagging
◆ Data Access
◆ Data Forget
➔ Security

68. Data Discovery
Self-Serve
Data Infra
Data Quality
Monitoring
Developer tooling :
MLOPS

69. Data Discovery
Self-Serve
Data Infra
Data Quality
Monitoring
Developer tooling :
MLOPS

70. data
.conf
DATA INFRA PROVISIONER
(CI/CD)

71. data
.conf
DATA INFRA PROVISIONER
(CI/CD)
❏ Topology
❏ T-shirt size
❏ Scope
❏ Encoding & schema
❏ Privacy informations
❏ Domain & team
❏ Producers/consumers
data
.conf

72. data
.conf
VALIDATOR
data
.conf
DATA INFRA PROVISIONER
(CI/CD)

73. data
.conf
SCHEMA
REGISTRY
ARCHIVER
PROVISIONER
VALIDATOR
data
.conf
DATA INFRA PROVISIONER
(CI/CD)

74. Data Discovery
Self-Serve
Data Infra
Data Quality
Monitoring
Developer tooling :
➔ Configuration descriptive
➔ Abstraction
➔ Auto-provisioning
➔ Respect des
standards
MLOPS

75. Data Discovery
Self-Serve
Data Infra
Data Quality
Monitoring
Developer tooling :
➔ Configuration
descriptive
➔ Abstraction
➔ Auto-provisioning
➔ Respect des
standards
➔ Toutes sources
➔ Documentation
➔ Ownership
➔ Lineage/usages
MLOPS

77. Data Discovery
Self-Serve
Data Infra
Data Quality
Monitoring
Developer tooling :
➔ Configuration
descriptive
➔ Abstraction
➔ Auto-provisioning
➔ Respect des
standards
➔ Toutes sources
➔ Documentation
➔ Ownership
➔ Lineage/usages
➔ Règles métiers
➔ Metrics & SLO
➔ As a service
MLOPS

79. Data Discovery
Self-Serve
Data Infra
Data Quality
Monitoring
Developer tooling :
➔ Configuration
descriptive
➔ Abstraction
➔ Auto-provisioning
➔ Respect des
standards
➔ Toutes sources
➔ Documentation
➔ Ownership
➔ Lineage/usages
➔ Règles métiers
➔ Metrics & SLO
➔ As a service
MLOPS

80. Offline
Data
Online
Data
Automated ML Pipeline
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
Data
Extraction ML
Model
Manual Experiment Steps
Data
Preparation
Model
Training
Model
Evaluation
Model
Analysis
Continuous Deployment
Model
Serving
Business
Monitoring
Data
Analysis
Data
Extraction
Data Engineering
80
1 2
3 4
Staging/Preprod/Prod
Staging/Preprod/Prod

81. Model
Analysis
Data
Analysis
Iterate
Orchestrated ML Experiment
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
ML Metadata
Store
Automated ML Pipeline
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
Data
Extraction ML
Model
Continuous Deployment
Model
Serving
Business
Monitoring
Feature Store
Staging/Preprod/Prod
Experimentation/Development/Test
1
2
3
Staging/Preprod/Prod

82. Model
Analysis
Data
Analysis
Iterate
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
ML Metadata
Store
Automated ML Pipeline
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
Data
Extraction ML
Model
Continuous Deployment
Model
Serving
Business
Monitoring
Feature Store
Staging/Preprod/Prod
Experimentation/Development/Test
1
2
3
Staging/Preprod/Prod
Orchestrated ML Experiment

83. Model
Analysis
Data
Analysis
Iterate
Orchestrated ML Experiment
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
ML Metadata
Store
Automated ML Pipeline
Data
Preparation
Model
Training
Model
Evaluation
Data
Validation
Model
Validation
Data
Extraction ML
Model
Continuous Deployment
Model
Serving
Business
Monitoring
Feature Store
Staging/Preprod/Prod
Experimentation/Development/Test
1
2
3
Staging/Preprod/Prod

84. Data Discovery
Self-Serve
Data Infra
Data Quality
Monitoring
Developer tooling :
➔ Configuration
descriptive
➔ Abstraction
➔ Auto-provisioning
➔ Respect des
standards
➔ Toutes sources
➔ Documentation
➔ Ownership
➔ Lineage/usages
➔ Règles métiers
➔ Metrics & SLO
➔ As a service
MLOPS
➔ Experimentation
Pipelines sur étagère
➔ Adaptation CI/CD
Modèles & dataset
➔ Feature store

85. We Are Here
(And it’s not over)

86. Comment passer les
compétences à l’échelle ?

87. ➔ Recrutements
Comment passer les
compétences à l’échelle ?

88. ➔ Recrutements
➔ Convergence software & data engineering
Comment passer les
compétences à l’échelle ?

89. Design
Pratiques
de développement
Data Engine
Awareness
Pollinisation des compétences :
➔ CI/CD for everyone
➔ Packaging
➔ Pair Programming
➔ TDD
➔ Clean archi
➔ Refactoring (Databases)
➔ Distributed Systems Patterns
➔ SQL is a must-have
➔ Modélisation
➔ Comment choisir ma DB

90. ➔ Recrutements
➔ Convergence software & data engineering
◆ Communautés de pratiques
Comment passer les
compétences à l’échelle ?

91. ➔ Recrutements
➔ Convergence software & data engineering
◆ Communautés de pratiques
◆ Formations en interne
Comment passer les
compétences à l’échelle ?
gmail

92. 2023 ?

93. Merci !
Des Questions ?
Retrouvez-nous 👉
👇
https://medium.com/leboncoin-engineering-blog
Comprendre le comportement des français,
c'est possible grâce à la data chez Leboncoin
Demain, de 15h30 à 16h15