This document provides an overview of several network embedding methods, including DeepWalk, Node2vec, and LINE. DeepWalk learns latent representations of nodes in a network using random walks and language modeling techniques. Node2vec generalizes DeepWalk by allowing the random walks to be tuned to different graph exploration strategies. LINE embeds networks by preserving both first-order and second-order proximities between nodes. The document also briefly discusses other methods like PTE, heterogeneous network embedding, and applications of network embedding.

1. Network embedding
methods: a survey
So Yeon Kim
2018. 1. 31.

2. Outline
● Network embedding methods
○ DeepWalk
○ Node2vec
○ LINE
○ Others
● Conclusion

3. Deepwalk (Perozzi et al., KDD 2014)
● Network embedding?
○ We map each node in a network into a low dimensional space
○ Distributed representation for nodes
○ Similarity between nodes indicate the link strength
○ Encode network information and generate node representation

4. Deepwalk (Perozzi et al., KDD 2014)
● Deepwalk learns a latent representation of adjacency matrices using deep learning techniques
developed for language modeling
Zachary’s karate network example

5. Deepwalk (Perozzi et al., KDD 2014)
● Words frequency in a natural language corpus follows a
power law.
● Vertex frequency in random walks on scale free graphs also
follows a power law.

6. Deepwalk (Perozzi et al., KDD 2014)
● Nodes <--> Words
● Node sequences <--> Sentences
● short random walks = sentences

7. Deepwalk (Perozzi et al., KDD 2014)

8. Outline
● Network embedding methods
○ DeepWalk
○ Node2vec
○ LINE
○ Others
● Conclusion

9. Node2vec (Grover et al., KDD 2016)
● A generalized version of Deepwalk
● The key lies in how to find a neighbor on the graph.
– BFS: broader -> homophily
– DFS: deeper -> structural equivalence
-> Combine these two !

10. Node2vec (Grover et al., KDD 2016)
● Biased random walk
● Parameters , controls interpolation between
DFS and BFS
○ Parameter controls the likelihood of
immediately revisiting a node in the walk
○ Parameter allows the search to differentiate
between “inward” and “outward” nodes
● = 1, = 1 -> DeepWalk

11. Outline
● Network embedding methods
○ DeepWalk
○ Node2vec
○ LINE
○ Others
● Conclusion

12. LINE (Tang et al., WWW 2015)
● First-order Proximity
● The local pairwise proximity between the vertices
determined by the observed links
● Not sufficient for preserving the entire network structure

13. LINE (Tang et al., WWW 2015)
● Second-order Proximity
● The proximity between the neighborhood structures of the
vertices

14. Multi-label network classification

15. Outline
● Network embedding methods
○ DeepWalk
○ Node2vec
○ LINE
○ Others
● Conclusion

16. PTE (Tang et al., KDD 2015)
● Extend LINE on large-scale information network embedding
● Only consider the second-order proximity here
● Different levels of word co-occurrences: local context-level, document-level, label-level

17. Heterogeneous Network Embedding (Chang et al., KDD 2015)
● Transfers different objects in heterogeneous networks to unified vector representations
● Deep learning models capture the complex interactions between heterogeneous components

18. Conclusion
● Network embedding methods preserving network structure
○ Node context, pairwise proximity
● Network embedding for ..
○ Community detection
○ Network clustering
○ Node classification
○ Link prediction
● Can utilize graph-structured data in diverse domains
● Single node can be embedded using multimodal embedding