2. Memory Matters
Drift Detection with a Low Memory Footprint
for ML Models on Kafka Streams
Alessandro Conflitti Head of Data Science

3. Concept drift
It refers to a drift or shift in the
statistical properties of the target or
dependent variable(s), i.e. the
relationship between the input features
X and the target outcomes Y has changed
over time.
Pt1
(Y|X) ≠ Pt2
(Y|X)
Data drift
It refers to a distribution change associated
with the inputs of a model.
This means there is a shift in the statistical
properties of the independent variables,
and therefore the training set is no longer
representative of the live input variables.
Pt1
(X) ≠ Pt2
(X)
Drift: an overview

4. Source: arxiv.org/abs/1010.4784
Reoccurring
context
Different types of (concept) drift
Incremental
drift
Gradual
drift
Sudden
drift

5. ● In (batch) ML Models
● In streaming
● In Kafka topics (with no
associated ML models)
Drift is
everywhere

6. Practical example:
readings from several weather stations.
Drift is
everywhere

7. What is
Helicon?
Helicon is the enterprise-grade
real-time ML platform designed
to combine streaming event
analysis and AI, simplifying
and accelerating developments
in Real Time Analytics projects
and Machine Learning-enabled
Decision Support Systems.

8. Drift architecture blueprint

10. Very low memory footprint!
● Especially important in applications where
we receive hundreds of messages per
second.
● Successfully tested for edge computing, see
tesi.univpm.it/handle/20.500.12075/12010.
Improvement on known
algorithm
ADWIN compares the average of two
subwindows.
To this we add a check on the variance,
which improves the detection rates.
Under the hood

11. Bells &
Whistles
drift is detected
Alerting and Webhooks
webhook sends alerts to a
channel of choice, e.g. Slack
a dedicated Kafka topic
is populated

12. Online Machine Learning
Batch (offline) ML: entire training data set must be made available prior to
the learning task and used as a whole. The training process is often done in
an offline environment due to the expensive training cost.
Online ML: learning from a sequence of data instances one by one at each
time, which arrive in a sequential order; a learner aims to learn and update
the best predictor for future data at every step.
Models are updated instantly and efficiently according to incoming stream
data and constantly learn new knowledge.
Far more efficient and scalable for large-scale machine learning tasks in
real-world data analytics applications where data are not only large in size,
but also arrive at a high velocity.
Sources: arxiv.org/abs/1802.02871 arxiv.org/abs/2201.01633
What is next

13. Alessandro Conflitti (Ph.D.)
alessandro.conflitti@radicalbit.io
linkedin.com/in/alessandroconflitti
Thanks!
radicalbit.io