1.
Link Prediction via
Subgraph Embedding-Based
Convex Matrix Completion
Link Prediction via
Subgraph Embedding-Based
Convex Matrix Completion
Zhu Cao
Tsinghua University
Linlin Wang
Tsinghua University
Gerard de Melo
http://gerard.demelo.org
Rutgers University

2.
Social NetworksSocial Networks
Image: CC-BY-SA by Tanja Cappell. https://www.flickr.com/photos/frauhoelle/8464661409

3.
Link PredictionLink Prediction

4.
Link PredictionLink Prediction
?

5.
Traditional
Neighbour Overlap Methods
Traditional
Neighbour Overlap Methods
?
Examples:
● Common Neighbours
● Adamic/Adar
(frequency weighted)
● Jaccard coefficient
based on neighbours

6.
Traditional
Neighbour Overlap Methods
Traditional
Neighbour Overlap Methods
?
Examples:
● Common Neighbours
● Adamic/Adar
(frequency weighted)
● Jaccard coefficient
based on neighbours

7.
Previous Work:
Connectivity-Based Methods
Previous Work:
Connectivity-Based Methods
?
Examples:
● PageRank
● SimRank
● Hitting Time
● Commute Time
(roundtrip)
● Katz

8.
Learning RepresentationsLearning Representations
Image: Adapted from Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations

9.
SVDSVD
E.g. Graph's
Adjacency Matrix
Image: Adapted from Tim Roughgarden & Gregory Valiant

10.
SVDSVD
Left Singular Vectors Right Singular Vectors
Singular Values
E.g. Graph's
Adjacency Matrix
Image: Adapted from Tim Roughgarden & Gregory Valiant

11.
Low-Rank Approximation
via SVD
Low-Rank Approximation
via SVD
≈
k
Left Singular Vectors
k
Right Singular Vectors
k Singular
ValuesE.g. Graph's
Adjacency Matrix
Image: Adapted from Tim Roughgarden & Gregory Valiant
k k k

12.
Word Vector Representations:
word2vec
Word Vector Representations:
word2vec
0.02
0.12
0.04
...
0.03
0.08
Prediction
…
the
local
population
of
sparrows
...
Large-Scale
Text
sparrows
population
local
...
word2vec Skip-Gram Model

13.
Previous Work:
DeepWalk
Previous Work:
DeepWalk
0.02
0.12
0.04
...
0.03
0.08
Prediction
…
n15
n382
n49
n729
n23
...
Social
Network
n49
n729
n382
...
word2vec Skip-Gram Model
Random Walk
Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations

14.
Previous Work:
node2vec
Previous Work:
node2vec
Grover & Leskovec (2016). node2vec: Scalable Feature Learning for Networks.
Biased Random Walks

15.
Our Approach:
SEMAC
Our Approach:
SEMAC
Subgraph
Embedding
+
Matrix Completion

16.
Our Approach:
SEMAC
Our Approach:
SEMAC

17.
Our Approach:
SEMAC
Our Approach:
SEMAC
Step 1:
Run Breadth-First Search for different depths d
from each node v
Step 1:
Run Breadth-First Search for different depths d
from each node v

18.
Our Approach:
SEMAC
Our Approach:
SEMAC

19.
Our Approach:
SEMAC
Our Approach:
SEMAC

20.
Our Approach:
SEMAC
Our Approach:
SEMAC

21.
Our Approach:
SEMAC
Our Approach:
SEMAC

22.
Our Approach:
SEMAC
Our Approach:
SEMAC
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs

23.
Our Approach:
SEMAC
Our Approach:
SEMAC
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs
Edges:
a) same node,
depth ± 1
b) same depth
neighbour node

24.
Our Approach:
SEMAC
Our Approach:
SEMAC
Edges:
a) same node,
depth ± 1
b) same depth,
neighbour node
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs

25.
Our Approach:
SEMAC
Our Approach:
SEMAC
Edges:
a) same node,
depth ± 1
b) same depth,
neighbour node
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs

26.
Our Approach:
SEMAC
Our Approach:
SEMAC
Step 3:
Learn subgraph embeddings
using G'
Step 3:
Learn subgraph embeddings
using G'

27.
Our Approach:
SEMAC
Our Approach:
SEMAC
Step 3:
Learn subgraph embeddings
using G'
Step 3:
Learn subgraph embeddings
using G'
Nuclear Norm
Nuclear Norm Minimization
Find W that
minimizes

28.
Our Approach:
SEMAC
Our Approach:
SEMAC
Step 3:
Learn subgraph embeddings
using G'
Step 3:
Learn subgraph embeddings
using G'
Nuclear Norm
Nuclear Norm Minimization
Find W that
minimizes
Frobenius NormCompare only
non-zero (observed)
entries (unlike SVD)

29.
Our Approach:
SEMAC
Our Approach:
SEMAC
Result of Step 3:
Embedding for
every subgraph
Result of Step 3:
Embedding for
every subgraph
0.32
...
0.27
0.81
...
0.12

30.
Our Approach:
SEMAC
Our Approach:
SEMAC
Step 4:
Create node embeddings
Concatenate embeddings
of subgraphs for different
depths d
Step 4:
Create node embeddings
Concatenate embeddings
of subgraphs for different
depths d
0.32
...
0.27
0.81
...
0.12
...
0.32
...
0.27
0.81
...
0.12

31.
Link PredictionLink Prediction
?
0.32
0.14
0.03
...
0.18
0.09
0.28
0.11
0.08
...
0.24
0.13
Step 5:
Link prediction
via Vector Cosine
Step 5:
Link prediction
via Vector Cosine

32.
ExperimentsExperiments
Image: CC-BY by Marc Smith with NodeXL. https://www.flickr.com/photos/marc_smith/6871711979

33.
ExperimentsExperiments
Facebook
(McAuley &
Leskovec 2012)

34.
ExperimentsExperiments
Facebook
(McAuley &
Leskovec 2012)
Small connected
component
subsets of Facebook

35.
ExperimentsExperiments
Wikipedia Coauthorship
(Leskovec & Krevl 2014)

36.
ExperimentsExperiments
Wikipedia Coauthorship
Protein-Protein Interactions
(Breitkreutz et al. 2008)
(Leskovec & Krevl 2014)

37.
ExperimentsExperiments
AUC based on 5-fold Cross-Validation

38.
SummarySummarySummarySummary
Goal: State-of-the-Art
Link Prediction
Consider Subgraphs
► Different depths
► Graph of subgraphs with links
to related subgraphs
Create Representations
► Nuclear-Norm Minimization
to better account for
unobserved links
► Concatenate, Cosine
Get in Touch!
http://gerard.demelo.org
gdm@demelo.org
Get in Touch!
http://gerard.demelo.org
gdm@demelo.org
Thank
you!
0.32
0.14
...
0.09
0.28
0.11
...
0.13

39.
AcknowledgmentsAcknowledgmentsAcknowledgmentsAcknowledgments
User Icons by
Freepik (CC-BY)
https://www.freepik.com
Title Image: CC-BY by
Chris Potter
https://www.flickr.com/photos/865
Thank
you!
0.32
0.14
...
0.09
0.28
0.11
...
0.13