Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

SEMAC Graph Node Embeddings for Link Prediction

692 vues

Publié le

We present a new graph representation learning approach called SEMAC that jointly exploits fine-grained node features as well as the overall graph topology. In contrast to the SGNS or SVD methods espoused in previous representation-based studies, our model represents nodes in terms of subgraph embeddings acquired via a form of convex matrix completion to iteratively reduce the rank, and thereby, more effectively eliminate noise in the representation. Thus, subgraph embeddings and convex matrix completion are elegantly integrated into a novel link prediction framework.

Publié dans : Technologie
  • Identifiez-vous pour voir les commentaires

  • Soyez le premier à aimer ceci

SEMAC Graph Node Embeddings for Link Prediction

  1. 1. Link Prediction via Subgraph Embedding-Based Convex Matrix Completion Link Prediction via Subgraph Embedding-Based Convex Matrix Completion Zhu Cao Tsinghua University Linlin Wang Tsinghua University Gerard de Melo http://gerard.demelo.org Rutgers University
  2. 2. Social NetworksSocial Networks Image: CC-BY-SA by Tanja Cappell. https://www.flickr.com/photos/frauhoelle/8464661409
  3. 3. Link PredictionLink Prediction
  4. 4. Link PredictionLink Prediction ?
  5. 5. Traditional Neighbour Overlap Methods Traditional Neighbour Overlap Methods ? Examples: ● Common Neighbours ● Adamic/Adar (frequency weighted) ● Jaccard coefficient based on neighbours
  6. 6. Traditional Neighbour Overlap Methods Traditional Neighbour Overlap Methods ? Examples: ● Common Neighbours ● Adamic/Adar (frequency weighted) ● Jaccard coefficient based on neighbours
  7. 7. Previous Work: Connectivity-Based Methods Previous Work: Connectivity-Based Methods ? Examples: ● PageRank ● SimRank ● Hitting Time ● Commute Time (roundtrip) ● Katz
  8. 8. Learning RepresentationsLearning Representations Image: Adapted from Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations
  9. 9. SVDSVD E.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant
  10. 10. SVDSVD Left Singular Vectors Right Singular Vectors Singular Values E.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant
  11. 11. Low-Rank Approximation via SVD Low-Rank Approximation via SVD ≈ k Left Singular Vectors k Right Singular Vectors k Singular ValuesE.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant k k k
  12. 12. Word Vector Representations: word2vec Word Vector Representations: word2vec 0.02 0.12 0.04 ... 0.03 0.08 Prediction … the local population of sparrows ... Large-Scale Text sparrows population local ... word2vec Skip-Gram Model
  13. 13. Previous Work: DeepWalk Previous Work: DeepWalk 0.02 0.12 0.04 ... 0.03 0.08 Prediction … n15 n382 n49 n729 n23 ... Social Network n49 n729 n382 ... word2vec Skip-Gram Model Random Walk Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations
  14. 14. Previous Work: node2vec Previous Work: node2vec Grover & Leskovec (2016). node2vec: Scalable Feature Learning for Networks. Biased Random Walks
  15. 15. Our Approach: SEMAC Our Approach: SEMAC Subgraph Embedding + Matrix Completion
  16. 16. Our Approach: SEMAC Our Approach: SEMAC
  17. 17. Our Approach: SEMAC Our Approach: SEMAC Step 1: Run Breadth-First Search for different depths d from each node v Step 1: Run Breadth-First Search for different depths d from each node v
  18. 18. Our Approach: SEMAC Our Approach: SEMAC
  19. 19. Our Approach: SEMAC Our Approach: SEMAC
  20. 20. Our Approach: SEMAC Our Approach: SEMAC
  21. 21. Our Approach: SEMAC Our Approach: SEMAC
  22. 22. Our Approach: SEMAC Our Approach: SEMAC Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs
  23. 23. Our Approach: SEMAC Our Approach: SEMAC Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs Edges: a) same node, depth ± 1 b) same depth neighbour node
  24. 24. Our Approach: SEMAC Our Approach: SEMAC Edges: a) same node, depth ± 1 b) same depth, neighbour node Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs
  25. 25. Our Approach: SEMAC Our Approach: SEMAC Edges: a) same node, depth ± 1 b) same depth, neighbour node Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs
  26. 26. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G'
  27. 27. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G' Nuclear Norm Nuclear Norm Minimization Find W that minimizes
  28. 28. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G' Nuclear Norm Nuclear Norm Minimization Find W that minimizes Frobenius NormCompare only non-zero (observed) entries (unlike SVD)
  29. 29. Our Approach: SEMAC Our Approach: SEMAC Result of Step 3: Embedding for every subgraph Result of Step 3: Embedding for every subgraph 0.32 ... 0.27 0.81 ... 0.12
  30. 30. Our Approach: SEMAC Our Approach: SEMAC Step 4: Create node embeddings Concatenate embeddings of subgraphs for different depths d Step 4: Create node embeddings Concatenate embeddings of subgraphs for different depths d 0.32 ... 0.27 0.81 ... 0.12 ... 0.32 ... 0.27 0.81 ... 0.12
  31. 31. Link PredictionLink Prediction ? 0.32 0.14 0.03 ... 0.18 0.09 0.28 0.11 0.08 ... 0.24 0.13 Step 5: Link prediction via Vector Cosine Step 5: Link prediction via Vector Cosine
  32. 32. ExperimentsExperiments Image: CC-BY by Marc Smith with NodeXL. https://www.flickr.com/photos/marc_smith/6871711979
  33. 33. ExperimentsExperiments Facebook (McAuley & Leskovec 2012)
  34. 34. ExperimentsExperiments Facebook (McAuley & Leskovec 2012) Small connected component subsets of Facebook
  35. 35. ExperimentsExperiments Wikipedia Coauthorship (Leskovec & Krevl 2014)
  36. 36. ExperimentsExperiments Wikipedia Coauthorship Protein-Protein Interactions (Breitkreutz et al. 2008) (Leskovec & Krevl 2014)
  37. 37. ExperimentsExperiments AUC based on 5-fold Cross-Validation
  38. 38. SummarySummarySummarySummary Goal: State-of-the-Art Link Prediction Consider Subgraphs ► Different depths ► Graph of subgraphs with links to related subgraphs Create Representations ► Nuclear-Norm Minimization to better account for unobserved links ► Concatenate, Cosine Get in Touch! http://gerard.demelo.org gdm@demelo.org Get in Touch! http://gerard.demelo.org gdm@demelo.org Thank you! 0.32 0.14 ... 0.09 0.28 0.11 ... 0.13
  39. 39. AcknowledgmentsAcknowledgmentsAcknowledgmentsAcknowledgments User Icons by Freepik (CC-BY) https://www.freepik.com Title Image: CC-BY by Chris Potter https://www.flickr.com/photos/865 Thank you! 0.32 0.14 ... 0.09 0.28 0.11 ... 0.13

×