This document discusses unsupervised learning techniques, including distances, clustering algorithms, and examples from Styria practice. It introduces common distance measures like Euclidean, Manhattan, and Mahalanobis distances. For clustering, it describes K-means clustering and provides a Spark example. It also discusses using convolutional neural network outputs as unsupervised learning features and shows examples of semi-manual photo clustering, T-SNE concept visualization, and automatic learned hierarchies from Styria projects.

1. Marko Velic PhD
Data Science Department
Styria Medijski Servisi d.o.o.
marko.velic@styria.hr
UNSUPERVISED LEARNING
(WITH SPARK)

2. CONTENTS
 Distances
• Eucledian
• Manhattan
• Mahalanobis
• Cosine Similarity
 Clustering
• K-Means
• Example (Spark)
 Examples from Styria practice (not Spark – for now)
10.03.2016 2

3. MACHINE LEARNING
10.03.2016 3

4. UNSUPERVISED LEARNING
 Opservations are not assigned to classes
 Computer program is not ‘supervised’
throughout the learning process
 Usually the task is to find ‘meaningful’
groups within data
 Decision is made based on distances i.e.
similarities among data points
10.03.2016 4

5. DISTANCES
10.03.2016 5
• To decide upon the groups we have to introduce
similarity measure or contrary – a distance measure
• Pitagora’s theorem – Euclidean distance
• dist((2, -1), (-2, 2))= √((2 - (-2))² + ((-1) - 2)²) = √((2 + 2)² + (-1 -
2)²) = √((4)² + (-3)²) = √(16 + 9) = √25 = 5

6. DISTANCES & APPROACHES
10.03.2016 6
Source:
http://en.wikipedia.org/wiki/Man
hattan_distance
 Manhattan/Cityblock/Taxicab
• dist((x, y), (a, b)) = |x - a| + |y - b|
 Normalization!
 Mahalanobis – considers variance
• “multidimensional z-score”
 Cosine similarity
 Autoencoders – ‘unsupervised’ neural nets
 Non-unsupervised but based on distances
• ReliefF measure, KNN classifier ... etc...

7. K-MEANS
7
Simplified:
1. Randomly place
centroids
2. Find the closest
3. Put centroid in the
middle
4. GOTO 2
Image source:
http://www.javabeat.net/2011/05/k-means-
clustering-algorithms-in-mahout/

8. DEMO (SPARK!)
 K-means clustering of photos (ie.
their vector representations)
 Convolutional neural network as
a supervised model and its
outputs as features for
unsupervised models
 Vector representations after the
pooling layers, after every
convolutional layer (Caffe)
 Clustering in Spark
8

9. T-SNE CLUSTER VISUALIZATION
9

10. SEMI-MANUAL CLUSTERING OF PHOTOS
10Gruping photos based in visual features, Enes Deumić, Styria Data Science Team

11. SEMI-MANUAL CLUSTERING OF PHOTOS
11Gruping photos based in visual features, Enes Deumić, Styria Data Science Team

12. NATURAL LANGUAGE PROCESSING
10.03.2016 12
T-sne concept visualization; vecernji.hr, Styria Data Science Team

13. AUTOMATIC (LEARNED) HIERARCHIES
13
Hierarchical clustering, Florijan Stamenković, Styria Data Science Team

14. VISUAL SEARCH EXAMPLE
14

15. CONCLUSION
 Distances
• Eucledian
• Manhattan
• Mahalanobis
• Cosine Similarity
 Clustering
• K-Means
 We can nicely combine supervised and unsupervised
features
 SparkNet: Training Deep Networks in Spark
http://arxiv.org/pdf/1511.06051v4.pdf
 https://news.developer.nvidia.com/caffe-on-spark-for-
deep-learning-from-yahoo/
10.03.2016 15

16. THANK YOU!

17. CONCLUSION
10.03.2016 17