Morichetta, A., Casas, P., & Mellia, M. (2019). EXPLAIN-IT: Towards explainable AI for unsupervised network traffic analysis. In Proceedings of the 3rd ACM CoNEXT Workshop on Big DAta, Machine Learning and Artificial Intelligence for Data Communication Networks (pp. 22–28).
1. EXPLAIN-IT: Towards Explainable
AI for Unsupervised Network
Traffic Analysisº
Andrea Morichetta★, Pedro Casas*, Marco Mellia★
Politecnico di Torino★, Austrian Institute of Technology*
3rd ACM CoNEXT Workshop on Big DAta, Machine Learning and Artificial
Intelligence for Data Communication Networks
2. The Gap
• Scenario: Rising popularity of ML applications for solving
specific problems in network traffic analysis.
• Ground truth is systematically missing – difficult to obtain
(structural complexity and big data volumes)
• Labeled datasets are frequently simplistic representation of
real-world phenomena, often also outdated.
2
3. Unsupervised learning to fill the gap
• Unsupervised techniques allow to have a better understanding of the
data, exploring its shape and patterns.
• However, it is difficult to analyze their results
• Typical solutions:
• manual inspection problem when there are too many or too complex data
• unsupervised quality metrics the why is missing
• supervised quality metrics not good if ground truth inherently wrong or
biased
3
4. Knowledge extraction from the clusters
Goal: have an interpretable representation of the features relevance in
the clusters
• For understanding the clusters content
• For better explanation of the data aggregation
4
5. Knowledge extraction – a supervised
approach
A possible solution: White box classifiers (white box techniques: e.g.,
linear regression and decision trees)
+Gives us also the opportunity to evaluate the cluster
attribution/assignment (via classification)
+Clear and algorithmically grounded
+Gives an “interpretation” available for the analysis
- It limits the set of applicable techniques
5
How to make this approach more general and extend the
set of algorithms?
6. Explainable AI - extend the supervised
approach
• EXPLAINABLE AI makes it easier to understand why certain decisions
or predictions have been made.
• Achieved by:
• Restricting the complexity of the machine learning model (intrinsic)
• Or by applying methods that analyze the model after training (post
hoc),
• e.g., LIME (Local Interpretable Model-agnostic Explanations)1 can
explain the predictions of any classifier or regressor, by
approximating it locally with an interpretable model.
6
1Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should i trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD
international conference on knowledge discovery and data mining. 2016.
8. Use case
• 10654 YouTube video sessions, coming from different sources, smartphone
(HTML player and YouTube app) and desktop (HTML player)
• Set of ~500 features:
• at the full video session level (e.g., session downlink throughout)
• as well as at different time resolutions with time slots of ∆t = [1, 5, 10] seconds.
• We focus on the average video quality (AVGQ) metric. We consider video
resolution as follows:
• 0: Low Definition (LD), with AVGQ < 480
• 1: Standard Definition (SD), with 480 ≤ AVGQ < 720
• 2: High Definition (HD), with AVGQ ≥ 720
8
9. Clustering phase
• Goal: We want to obtain 3 clusters in output:
a. Low Definition, LD
b. Standard definition, SD
c. High Definition, HD
• Algorithms used:
• Agglomerative (1) clustering with Ward Links (Ward minimizes the variance
of the clusters being merged)
• Agglomerative (2) clustering with Single Links (Single single uses the
minimum of the distances between all observations of the two sets)
• K-Means
• BIRCH - Balanced Iterative Reducing and Clustering using Hierarchies
9
11. Clustering results – label distribution
11
Label distribution after agglomerative Ward clustering
12. Clustering results – feature Inspection
12
Example of feature inspection inthe results of agglomerative Ward clustering
Cluster 0 Cluster 1 Cluster 2
13. Interpret with model – using Support Vector
Machines
• Hyperplane-based classifiers
• The SVM selects the maximum margin separating hyperplane
• Use of kernel function to map points on a high-dimensional space
• However, it is a black-box classifier
• Thus, Explainable AI can aid us
13
14. Interpret with model – using SVM
14
Agglomerative (1)
Results of SVM applied to Agglomerative with Ward
16. Conclusion and future work
• Interesting approach for improving the interpretation of clustering
results by relying on XAI principles
• Is explainable AI an advantage in the YouTube case, where features
are complex?
• Is LIME always good? Look at alternatives, e.g., SHAP
• Is it possible to avoid the classification step?
• Extend it to other scenarios
• Expand the research on different clustering algorithms
• Use different classification techniques
16
Notes de l'éditeur
Why our model predicted a specific label? E.g., if traffic is malicious or not?
LIME intuition is to look closer in the area of the predicted decision, and get easier boundaries
LIME is only based on inputs and outputs of the model
Random generating data points, by perturbation, in the neighborhood of our target data point
What we get, is a new dataset in the neighborhood of our target, that we can interpret with a white box model
Assign weights to the points closer to the target in order to get these rights when predicting with a local linear model
packet-level video traffic measurements
only information extracted from the network traffic for each of the captured packet are packet time and packet size. From these two values, we then derive a full set of 477 different features
Overall/full session traffic, downlink traffic and uplink traffic
Sampled empirical distributions of overall session traffic, downlink traffic and uplink traffic
extracted from the analyzed network video traffic packets into relevant Video Quality Metrics.
Six VQMs:
initial delay,
frequency of stallings,
number of stalling events,
number of quality switches,
average video quality (video vertical resolution, e.g., 480p, 720p, 1080p, etc.)
and average video bitrate.