SlideShare une entreprise Scribd logo

Lec5 Pagerank

•Télécharger en tant que PPT, PDF•
•
M
mobius.cn

Graph representations, distributed Pagerank, distributed Dijkstra.

1  sur  33
Distributed Computing Seminar Lecture 5: Graph Algorithms & PageRank Christophe Bisciglia, Aaron Kimball, & Sierra Michels-Slettvet Summer 2007 Except as otherwise noted, the content of this presentation is © 2007 Google Inc. and licensed under the Creative Commons Attribution 2.5 License.
Outline ,[object Object],[object Object],[object Object],[object Object]
Motivating Concepts ,[object Object],[object Object],[object Object],[object Object]
Breadth-First Search ,[object Object],[object Object]
Breadth-First Search & MapReduce ,[object Object],[object Object]
Breadth-First Search & MapReduce ,[object Object],[object Object]
Graph Representations ,[object Object]
Direct References ,[object Object],[object Object],[object Object],[object Object],class GraphNode { Object data; Vector<GraphNode> out_edges; GraphNode iter_next; }
Adjacency Matrices ,[object Object],[object Object],0 1 0 1 4 0 0 1 0 3 1 1 0 1 2 1 0 1 0 1 4 3 2 1
Adjacency Matrices: Sparse Representation ,[object Object],[object Object],[object Object]
Sparse Matrix Representation ,[object Object],[object Object],[object Object],[object Object]
Sparse Matrix Representation ,[object Object],[object Object],[object Object],[object Object]
Finding the Shortest Path ,[object Object],[object Object],[object Object],This is called the single-source shortest path problem. (a.k.a. SSSP)
Finding the Shortest Path: Intuition ,[object Object],[object Object],[object Object],[object Object],[object Object]
From Intuition to Algorithm ,[object Object],[object Object],[object Object],[object Object],[object Object]
What This Gives Us ,[object Object],[object Object],[object Object],[object Object]
Blow-up and Termination ,[object Object],[object Object],[object Object],[object Object],[object Object]
Adding weights ,[object Object],[object Object],[object Object],[object Object]
Comparison to Dijkstra ,[object Object],[object Object],[object Object]
PageRank: Random Walks Over The Web ,[object Object],[object Object],[object Object]
PageRank: Visually
PageRank: Formula ,[object Object],[object Object],[object Object],[object Object],[object Object]
PageRank: Intuition ,[object Object],[object Object],[object Object],[object Object]
PageRank: Issues ,[object Object],[object Object],[object Object]
PageRank: First Implementation ,[object Object],[object Object],[object Object],[object Object]
Distribution of the Algorithm ,[object Object],[object Object],[object Object],[object Object]
Distribution of the Algorithm ,[object Object],[object Object],[object Object],[object Object]
 
Phase 1: Parse HTML ,[object Object],[object Object],[object Object],[object Object]
Phase 2: PageRank Distribution ,[object Object],[object Object],[object Object],[object Object]
Phase 2: PageRank Distribution ,[object Object],[object Object],[object Object],[object Object]
Finishing up... ,[object Object],[object Object],[object Object]
Conclusions ,[object Object],[object Object],[object Object],[object Object]

Recommandé

Pagerank (from Google)
Pagerank (from Google)Pagerank (from Google)
Pagerank (from Google)Sri Prasanna
 
Lec5 Pagerank
Lec5 PagerankLec5 Pagerank
Lec5 PagerankJeff Hammerbacher
 
Lecture set 5
Lecture set 5Lecture set 5
Lecture set 5Gopi Saiteja
 
Unit26 shortest pathalgorithm
Unit26 shortest pathalgorithmUnit26 shortest pathalgorithm
Unit26 shortest pathalgorithmmeisamstar
 
1535 graph algorithms
1535 graph algorithms1535 graph algorithms
1535 graph algorithmsDr Fereidoun Dejahang
 
18 Basic Graph Algorithms
18 Basic Graph Algorithms18 Basic Graph Algorithms
18 Basic Graph AlgorithmsAndres Mendez-Vazquez
 
Review and evaluations of shortest path algorithms
Review and evaluations of shortest path algorithmsReview and evaluations of shortest path algorithms
Review and evaluations of shortest path algorithmsPawan Kumar Tiwari
 
Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)
Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)
Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)Mohanlal Sukhadia University (MLSU)
 

Recommandé

Pagerank (from Google)
Pagerank (from Google)Pagerank (from Google)
Pagerank (from Google)Sri Prasanna
 
Lec5 Pagerank
Lec5 PagerankLec5 Pagerank
Lec5 PagerankJeff Hammerbacher
 
Lecture set 5
Lecture set 5Lecture set 5
Lecture set 5Gopi Saiteja
 
Unit26 shortest pathalgorithm
Unit26 shortest pathalgorithmUnit26 shortest pathalgorithm
Unit26 shortest pathalgorithmmeisamstar
 
1535 graph algorithms
1535 graph algorithms1535 graph algorithms
1535 graph algorithmsDr Fereidoun Dejahang
 
18 Basic Graph Algorithms
18 Basic Graph Algorithms18 Basic Graph Algorithms
18 Basic Graph AlgorithmsAndres Mendez-Vazquez
 
Review and evaluations of shortest path algorithms
Review and evaluations of shortest path algorithmsReview and evaluations of shortest path algorithms
Review and evaluations of shortest path algorithmsPawan Kumar Tiwari
 
Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)
Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)
Shortest path (Dijkistra's Algorithm) & Spanning Tree (Prim's Algorithm)Mohanlal Sukhadia University (MLSU)
 
Shortest Path Problem
Shortest Path ProblemShortest Path Problem
Shortest Path ProblemGuillaume Guérard
 
Prim's Algorithm on minimum spanning tree
Prim's Algorithm on minimum spanning treePrim's Algorithm on minimum spanning tree
Prim's Algorithm on minimum spanning treeoneous
 
Big Data Analysis with Signal Processing on Graphs
Big Data Analysis with Signal Processing on GraphsBig Data Analysis with Signal Processing on Graphs
Big Data Analysis with Signal Processing on GraphsMohamed Seif
 
All pairs shortest path algorithm
All pairs shortest path algorithmAll pairs shortest path algorithm
All pairs shortest path algorithmSrikrishnan Suresh
 
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...Ramoni Adeogun, PhD
 
Graph Analyses with Python and NetworkX
Graph Analyses with Python and NetworkXGraph Analyses with Python and NetworkX
Graph Analyses with Python and NetworkXBenjamin Bengfort
 
Djikstra's Algorithm
Djikstra's Algorithm Djikstra's Algorithm
Djikstra's Algorithm Samar Kenkre
 
A star algorithms
A star algorithmsA star algorithms
A star algorithmssandeep54552
 
Networkx tutorial
Networkx tutorialNetworkx tutorial
Networkx tutorialDeepakshankar S
 
Shortest path search for real road networks and dynamic costs with pgRouting
Shortest path search for real road networks and dynamic costs with pgRoutingShortest path search for real road networks and dynamic costs with pgRouting
Shortest path search for real road networks and dynamic costs with pgRoutingantonpa
 
Application of Dijkstra Algorithm in Robot path planning
Application of Dijkstra Algorithm in Robot path planningApplication of Dijkstra Algorithm in Robot path planning
Application of Dijkstra Algorithm in Robot path planningDarling Jemima
 
Graph Traversal Algorithms - Depth First Search Traversal
Graph Traversal Algorithms - Depth First Search TraversalGraph Traversal Algorithms - Depth First Search Traversal
Graph Traversal Algorithms - Depth First Search TraversalAmrinder Arora
 
1 chayes
1 chayes1 chayes
1 chayesYandex
 
Biconnected components (13024116056)
Biconnected components (13024116056)Biconnected components (13024116056)
Biconnected components (13024116056)Akshay soni
 
Scalar lenses-workshop
Scalar lenses-workshopScalar lenses-workshop
Scalar lenses-workshopMarcin Matuszak
 
(Icca 2014) shortest path analysis in social graphs
(Icca 2014) shortest path analysis in social graphs(Icca 2014) shortest path analysis in social graphs
(Icca 2014) shortest path analysis in social graphsWaqas Nawaz
 
cis98010
cis98010cis98010
cis98010perfj
 
Networks dijkstra's algorithm- pgsr
Networks dijkstra's algorithm- pgsrNetworks dijkstra's algorithm- pgsr
Networks dijkstra's algorithm- pgsrLinawati Adiman
 
From Data to Knowledge thru Grailog Visualization
From Data to Knowledge thru Grailog VisualizationFrom Data to Knowledge thru Grailog Visualization
From Data to Knowledge thru Grailog Visualizationgiurca
 
Combinatorial Optimization
Combinatorial OptimizationCombinatorial Optimization
Combinatorial OptimizationInstitute of Technology, Nirma University
 
MapReduceAlgorithms.ppt
MapReduceAlgorithms.pptMapReduceAlgorithms.ppt
MapReduceAlgorithms.pptCheeWeiTan10
 
Machine Learning and GraphX
Machine Learning and GraphXMachine Learning and GraphX
Machine Learning and GraphXAndy Petrella
 

Contenu connexe

Tendances

Prim's Algorithm on minimum spanning tree
Prim's Algorithm on minimum spanning treePrim's Algorithm on minimum spanning tree
Prim's Algorithm on minimum spanning treeoneous
 
Big Data Analysis with Signal Processing on Graphs
Big Data Analysis with Signal Processing on GraphsBig Data Analysis with Signal Processing on Graphs
Big Data Analysis with Signal Processing on GraphsMohamed Seif
 
All pairs shortest path algorithm
All pairs shortest path algorithmAll pairs shortest path algorithm
All pairs shortest path algorithmSrikrishnan Suresh
 
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...Ramoni Adeogun, PhD
 
Graph Analyses with Python and NetworkX
Graph Analyses with Python and NetworkXGraph Analyses with Python and NetworkX
Graph Analyses with Python and NetworkXBenjamin Bengfort
 
Djikstra's Algorithm
Djikstra's Algorithm Djikstra's Algorithm
Djikstra's Algorithm Samar Kenkre
 
Shortest path search for real road networks and dynamic costs with pgRouting
Shortest path search for real road networks and dynamic costs with pgRoutingShortest path search for real road networks and dynamic costs with pgRouting
Shortest path search for real road networks and dynamic costs with pgRoutingantonpa
 
Application of Dijkstra Algorithm in Robot path planning
Application of Dijkstra Algorithm in Robot path planningApplication of Dijkstra Algorithm in Robot path planning
Application of Dijkstra Algorithm in Robot path planningDarling Jemima
 
Graph Traversal Algorithms - Depth First Search Traversal
Graph Traversal Algorithms - Depth First Search TraversalGraph Traversal Algorithms - Depth First Search Traversal
Graph Traversal Algorithms - Depth First Search TraversalAmrinder Arora
 
1 chayes
1 chayes1 chayes
1 chayesYandex
 
Biconnected components (13024116056)
Biconnected components (13024116056)Biconnected components (13024116056)
Biconnected components (13024116056)Akshay soni
 
(Icca 2014) shortest path analysis in social graphs
(Icca 2014) shortest path analysis in social graphs(Icca 2014) shortest path analysis in social graphs
(Icca 2014) shortest path analysis in social graphsWaqas Nawaz
 
cis98010
cis98010cis98010
cis98010perfj
 
Networks dijkstra's algorithm- pgsr
Networks dijkstra's algorithm- pgsrNetworks dijkstra's algorithm- pgsr
Networks dijkstra's algorithm- pgsrLinawati Adiman
 
From Data to Knowledge thru Grailog Visualization
From Data to Knowledge thru Grailog VisualizationFrom Data to Knowledge thru Grailog Visualization
From Data to Knowledge thru Grailog Visualizationgiurca
 

Tendances (20)

Shortest Path Problem
Shortest Path ProblemShortest Path Problem
Shortest Path Problem
 
Prim's Algorithm on minimum spanning tree
Prim's Algorithm on minimum spanning treePrim's Algorithm on minimum spanning tree
Prim's Algorithm on minimum spanning tree
 
Big Data Analysis with Signal Processing on Graphs
Big Data Analysis with Signal Processing on GraphsBig Data Analysis with Signal Processing on Graphs
Big Data Analysis with Signal Processing on Graphs
 
All pairs shortest path algorithm
All pairs shortest path algorithmAll pairs shortest path algorithm
All pairs shortest path algorithm
 
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...
Reduced Complexity Transfer Function Computation for Complex Indoor Channels ...
 
Graph Analyses with Python and NetworkX
Graph Analyses with Python and NetworkXGraph Analyses with Python and NetworkX
Graph Analyses with Python and NetworkX
 
Djikstra's Algorithm
Djikstra's Algorithm Djikstra's Algorithm
Djikstra's Algorithm
 
A star algorithms
A star algorithmsA star algorithms
A star algorithms
 
Networkx tutorial
Networkx tutorialNetworkx tutorial
Networkx tutorial
 
Shortest path search for real road networks and dynamic costs with pgRouting
Shortest path search for real road networks and dynamic costs with pgRoutingShortest path search for real road networks and dynamic costs with pgRouting
Shortest path search for real road networks and dynamic costs with pgRouting
 
Application of Dijkstra Algorithm in Robot path planning
Application of Dijkstra Algorithm in Robot path planningApplication of Dijkstra Algorithm in Robot path planning
Application of Dijkstra Algorithm in Robot path planning
 
Graph Traversal Algorithms - Depth First Search Traversal
Graph Traversal Algorithms - Depth First Search TraversalGraph Traversal Algorithms - Depth First Search Traversal
Graph Traversal Algorithms - Depth First Search Traversal
 
1 chayes
1 chayes1 chayes
1 chayes
 
Biconnected components (13024116056)
Biconnected components (13024116056)Biconnected components (13024116056)
Biconnected components (13024116056)
 
Scalar lenses-workshop
Scalar lenses-workshopScalar lenses-workshop
Scalar lenses-workshop
 
(Icca 2014) shortest path analysis in social graphs
(Icca 2014) shortest path analysis in social graphs(Icca 2014) shortest path analysis in social graphs
(Icca 2014) shortest path analysis in social graphs
 
cis98010
cis98010cis98010
cis98010
 
Networks dijkstra's algorithm- pgsr
Networks dijkstra's algorithm- pgsrNetworks dijkstra's algorithm- pgsr
Networks dijkstra's algorithm- pgsr
 
From Data to Knowledge thru Grailog Visualization
From Data to Knowledge thru Grailog VisualizationFrom Data to Knowledge thru Grailog Visualization
From Data to Knowledge thru Grailog Visualization
 
Combinatorial Optimization
Combinatorial OptimizationCombinatorial Optimization
Combinatorial Optimization
 

Similaire à Lec5 Pagerank

MapReduceAlgorithms.ppt
MapReduceAlgorithms.pptMapReduceAlgorithms.ppt
MapReduceAlgorithms.pptCheeWeiTan10
 
Machine Learning and GraphX
Machine Learning and GraphXMachine Learning and GraphX
Machine Learning and GraphXAndy Petrella
 
Behm Shah Pagerank
Behm Shah PagerankBehm Shah Pagerank
Behm Shah Pagerankgothicane
 
PowerLyra@EuroSys2015
PowerLyra@EuroSys2015PowerLyra@EuroSys2015
PowerLyra@EuroSys2015realstolz
 
Graph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam LermaGraph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam LermaPyData
 
Multinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache SparkMultinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache SparkDB Tsai
 
Alpine Spark Implementation - Technical
Alpine Spark Implementation - TechnicalAlpine Spark Implementation - Technical
Alpine Spark Implementation - Technicalalpinedatalabs
 
Streaming Python on Hadoop
Streaming Python on HadoopStreaming Python on Hadoop
Streaming Python on HadoopVivian S. Zhang
 
Parallel algorithms
Parallel algorithmsParallel algorithms
Parallel algorithmsguest084d20
 
Graph convolutional networks in apache spark
Graph convolutional networks in apache sparkGraph convolutional networks in apache spark
Graph convolutional networks in apache sparkEmiliano Martinez Sanchez
 
Chapter8-Link_Analysis.pptx
Chapter8-Link_Analysis.pptxChapter8-Link_Analysis.pptx
Chapter8-Link_Analysis.pptxAmenahAbbood
 
Chapter8-Link_Analysis (1).pptx
Chapter8-Link_Analysis (1).pptxChapter8-Link_Analysis (1).pptx
Chapter8-Link_Analysis (1).pptxAmenahAbbood
 
Social network-analysis-in-python
Social network-analysis-in-pythonSocial network-analysis-in-python
Social network-analysis-in-pythonJoe OntheRocks
 
Tree distance algorithm
Tree distance algorithmTree distance algorithm
Tree distance algorithmTrector Rancor
 

Similaire à Lec5 Pagerank (20)

MapReduceAlgorithms.ppt
MapReduceAlgorithms.pptMapReduceAlgorithms.ppt
MapReduceAlgorithms.ppt
 
Machine Learning and GraphX
Machine Learning and GraphXMachine Learning and GraphX
Machine Learning and GraphX
 
Behm Shah Pagerank
Behm Shah PagerankBehm Shah Pagerank
Behm Shah Pagerank
 
PowerLyra@EuroSys2015
PowerLyra@EuroSys2015PowerLyra@EuroSys2015
PowerLyra@EuroSys2015
 
Dstar Lite
Dstar LiteDstar Lite
Dstar Lite
 
Dijkstra
DijkstraDijkstra
Dijkstra
 
d
dd
d
 
Graph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam LermaGraph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
Graph Analytics - From the Whiteboard to Your Toolbox - Sam Lerma
 
Multinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache SparkMultinomial Logistic Regression with Apache Spark
Multinomial Logistic Regression with Apache Spark
 
Alpine Spark Implementation - Technical
Alpine Spark Implementation - TechnicalAlpine Spark Implementation - Technical
Alpine Spark Implementation - Technical
 
Streaming Python on Hadoop
Streaming Python on HadoopStreaming Python on Hadoop
Streaming Python on Hadoop
 
Parallel algorithms
Parallel algorithmsParallel algorithms
Parallel algorithms
 
Graph convolutional networks in apache spark
Graph convolutional networks in apache sparkGraph convolutional networks in apache spark
Graph convolutional networks in apache spark
 
Lecture13
Lecture13Lecture13
Lecture13
 
Hadoop classes in mumbai
Hadoop classes in mumbaiHadoop classes in mumbai
Hadoop classes in mumbai
 
Chapter8-Link_Analysis.pptx
Chapter8-Link_Analysis.pptxChapter8-Link_Analysis.pptx
Chapter8-Link_Analysis.pptx
 
Chapter8-Link_Analysis (1).pptx
Chapter8-Link_Analysis (1).pptxChapter8-Link_Analysis (1).pptx
Chapter8-Link_Analysis (1).pptx
 
Social network-analysis-in-python
Social network-analysis-in-pythonSocial network-analysis-in-python
Social network-analysis-in-python
 
Tree distance algorithm
Tree distance algorithmTree distance algorithm
Tree distance algorithm
 
Unit3 MapReduce
Unit3 MapReduceUnit3 MapReduce
Unit3 MapReduce
 

Plus de mobius.cn

Lec4 Clustering
Lec4 ClusteringLec4 Clustering
Lec4 Clusteringmobius.cn
 
Parallel Programming Primer 1
Parallel Programming Primer 1Parallel Programming Primer 1
Parallel Programming Primer 1mobius.cn
 
Advanced Lighting Techniques Dan Baker (Meltdown 2005)
Advanced Lighting Techniques Dan Baker (Meltdown 2005)Advanced Lighting Techniques Dan Baker (Meltdown 2005)
Advanced Lighting Techniques Dan Baker (Meltdown 2005)mobius.cn
 
Influence map
Influence mapInfluence map
Influence mapmobius.cn
 

Plus de mobius.cn (7)

Lec4 Clustering
Lec4 ClusteringLec4 Clustering
Lec4 Clustering
 
Lec3 Dfs
Lec3 DfsLec3 Dfs
Lec3 Dfs
 
Lec2 Mapred
Lec2 MapredLec2 Mapred
Lec2 Mapred
 
Parallel Programming Primer 1
Parallel Programming Primer 1Parallel Programming Primer 1
Parallel Programming Primer 1
 
Lec1 Intro
Lec1 IntroLec1 Intro
Lec1 Intro
 
Advanced Lighting Techniques Dan Baker (Meltdown 2005)
Advanced Lighting Techniques Dan Baker (Meltdown 2005)Advanced Lighting Techniques Dan Baker (Meltdown 2005)
Advanced Lighting Techniques Dan Baker (Meltdown 2005)
 
Influence map
Influence mapInfluence map
Influence map
 

Lec5 Pagerank

  • 1. Distributed Computing Seminar Lecture 5: Graph Algorithms & PageRank Christophe Bisciglia, Aaron Kimball, & Sierra Michels-Slettvet Summer 2007 Except as otherwise noted, the content of this presentation is © 2007 Google Inc. and licensed under the Creative Commons Attribution 2.5 License.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.  
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.