SEMAC Graph Node Embeddings for Link Prediction

1. Link Prediction via Subgraph Embedding-Based Convex Matrix Completion Link Prediction via Subgraph Embedding-Based Convex Matrix Completion Zhu Cao Tsinghua University Linlin Wang Tsinghua University Gerard de Melo http://gerard.demelo.org Rutgers University

2. Social NetworksSocial Networks Image: CC-BY-SA by Tanja Cappell. https://www.flickr.com/photos/frauhoelle/8464661409

3. Link PredictionLink Prediction

4. Link PredictionLink Prediction ?

5. Traditional Neighbour Overlap Methods Traditional Neighbour Overlap Methods ? Examples: ● Common Neighbours ● Adamic/Adar (frequency weighted) ● Jaccard coefficient based on neighbours

6. Traditional Neighbour Overlap Methods Traditional Neighbour Overlap Methods ? Examples: ● Common Neighbours ● Adamic/Adar (frequency weighted) ● Jaccard coefficient based on neighbours

7. Previous Work: Connectivity-Based Methods Previous Work: Connectivity-Based Methods ? Examples: ● PageRank ● SimRank ● Hitting Time ● Commute Time (roundtrip) ● Katz

8. Learning RepresentationsLearning Representations Image: Adapted from Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations

9. SVDSVD E.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant

10. SVDSVD Left Singular Vectors Right Singular Vectors Singular Values E.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant

11. Low-Rank Approximation via SVD Low-Rank Approximation via SVD ≈ k Left Singular Vectors k Right Singular Vectors k Singular ValuesE.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant k k k

12. Word Vector Representations: word2vec Word Vector Representations: word2vec 0.02 0.12 0.04 ... 0.03 0.08 Prediction … the local population of sparrows ... Large-Scale Text sparrows population local ... word2vec Skip-Gram Model

13. Previous Work: DeepWalk Previous Work: DeepWalk 0.02 0.12 0.04 ... 0.03 0.08 Prediction … n15 n382 n49 n729 n23 ... Social Network n49 n729 n382 ... word2vec Skip-Gram Model Random Walk Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations

14. Previous Work: node2vec Previous Work: node2vec Grover & Leskovec (2016). node2vec: Scalable Feature Learning for Networks. Biased Random Walks

15. Our Approach: SEMAC Our Approach: SEMAC Subgraph Embedding + Matrix Completion

16. Our Approach: SEMAC Our Approach: SEMAC

17. Our Approach: SEMAC Our Approach: SEMAC Step 1: Run Breadth-First Search for different depths d from each node v Step 1: Run Breadth-First Search for different depths d from each node v

22. Our Approach: SEMAC Our Approach: SEMAC Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs

23. Our Approach: SEMAC Our Approach: SEMAC Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs Edges: a) same node, depth ± 1 b) same depth neighbour node

24. Our Approach: SEMAC Our Approach: SEMAC Edges: a) same node, depth ± 1 b) same depth, neighbour node Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs

25. Our Approach: SEMAC Our Approach: SEMAC Edges: a) same node, depth ± 1 b) same depth, neighbour node Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs

26. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G'

27. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G' Nuclear Norm Nuclear Norm Minimization Find W that minimizes

28. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G' Nuclear Norm Nuclear Norm Minimization Find W that minimizes Frobenius NormCompare only non-zero (observed) entries (unlike SVD)

29. Our Approach: SEMAC Our Approach: SEMAC Result of Step 3: Embedding for every subgraph Result of Step 3: Embedding for every subgraph 0.32 ... 0.27 0.81 ... 0.12

30. Our Approach: SEMAC Our Approach: SEMAC Step 4: Create node embeddings Concatenate embeddings of subgraphs for different depths d Step 4: Create node embeddings Concatenate embeddings of subgraphs for different depths d 0.32 ... 0.27 0.81 ... 0.12 ... 0.32 ... 0.27 0.81 ... 0.12

31. Link PredictionLink Prediction ? 0.32 0.14 0.03 ... 0.18 0.09 0.28 0.11 0.08 ... 0.24 0.13 Step 5: Link prediction via Vector Cosine Step 5: Link prediction via Vector Cosine

32. ExperimentsExperiments Image: CC-BY by Marc Smith with NodeXL. https://www.flickr.com/photos/marc_smith/6871711979

33. ExperimentsExperiments Facebook (McAuley & Leskovec 2012)

34. ExperimentsExperiments Facebook (McAuley & Leskovec 2012) Small connected component subsets of Facebook

35. ExperimentsExperiments Wikipedia Coauthorship (Leskovec & Krevl 2014)

36. ExperimentsExperiments Wikipedia Coauthorship Protein-Protein Interactions (Breitkreutz et al. 2008) (Leskovec & Krevl 2014)

37. ExperimentsExperiments AUC based on 5-fold Cross-Validation

38. SummarySummarySummarySummary Goal: State-of-the-Art Link Prediction Consider Subgraphs ► Different depths ► Graph of subgraphs with links to related subgraphs Create Representations ► Nuclear-Norm Minimization to better account for unobserved links ► Concatenate, Cosine Get in Touch! http://gerard.demelo.org gdm@demelo.org Get in Touch! http://gerard.demelo.org gdm@demelo.org Thank you! 0.32 0.14 ... 0.09 0.28 0.11 ... 0.13

39. AcknowledgmentsAcknowledgmentsAcknowledgmentsAcknowledgments User Icons by Freepik (CC-BY) https://www.freepik.com Title Image: CC-BY by Chris Potter https://www.flickr.com/photos/865 Thank you! 0.32 0.14 ... 0.09 0.28 0.11 ... 0.13