Soumettre la recherche
Mettre en ligne
Kineograph: Taking the Pulse of a Fast-Changing and Connected World
•
Télécharger en tant que PPTX, PDF
•
0 j'aime
•
569 vues
Qian Lin
Suivre
Signaler
Partager
Signaler
Partager
1 sur 43
Télécharger maintenant
Recommandé
Direct QR factorizations for tall-and-skinny matrices in MapReduce architectu...
Direct QR factorizations for tall-and-skinny matrices in MapReduce architectu...
Austin Benson
Fast rcnn
Fast rcnn
limHoJun
Signal Flow Graph ( control system)
Signal Flow Graph ( control system)
Gourab Ghosh
Block Diagram Reduction
Block Diagram Reduction
Gauravsinh Parmar
Quiz
Quiz
Qazi Ejaz
Control system concepts by using matlab
Control system concepts by using matlab
CharltonInao1
Rules of block diagram
Rules of block diagram
ManishDubey118
Block diagrams and signal flow graphs
Block diagrams and signal flow graphs
Hussain K
Recommandé
Direct QR factorizations for tall-and-skinny matrices in MapReduce architectu...
Direct QR factorizations for tall-and-skinny matrices in MapReduce architectu...
Austin Benson
Fast rcnn
Fast rcnn
limHoJun
Signal Flow Graph ( control system)
Signal Flow Graph ( control system)
Gourab Ghosh
Block Diagram Reduction
Block Diagram Reduction
Gauravsinh Parmar
Quiz
Quiz
Qazi Ejaz
Control system concepts by using matlab
Control system concepts by using matlab
CharltonInao1
Rules of block diagram
Rules of block diagram
ManishDubey118
Block diagrams and signal flow graphs
Block diagrams and signal flow graphs
Hussain K
Block diagram reduction techniques
Block diagram reduction techniques
parimalagandhi ayyavu
Generalized Isomorphism between Synchronous Circuits and State Machines
Generalized Isomorphism between Synchronous Circuits and State Machines
Shunji Nishimura
Large-scale computation without sacrificing expressiveness
Large-scale computation without sacrificing expressiveness
Sangjin Han
Fast and Scalable NUMA-based Thread Parallel Breadth-first Search
Fast and Scalable NUMA-based Thread Parallel Breadth-first Search
Yuichiro Yasui
GTC 2009 OpenGL Barthold
GTC 2009 OpenGL Barthold
Mark Kilgard
3.1,2,3 pushdown automata definition, moves & id
3.1,2,3 pushdown automata definition, moves & id
Sampath Kumar S
Class 25: Reversing Reverse
Class 25: Reversing Reverse
David Evans
Reed Solomon Matlab Projects Research Ideas
Reed Solomon Matlab Projects Research Ideas
Matlab Simulation
Reduction of multiple subsystem [compatibility mode]
Reduction of multiple subsystem [compatibility mode]
azroyyazid
MySQL Spatial Extensions - Boston MySQL Meetup April 2005
MySQL Spatial Extensions - Boston MySQL Meetup April 2005
Andrew Collins
Parallel quicksort cz. 1
Parallel quicksort cz. 1
Mikołaj Olszewski
Broad-sense Synchronous Circuits on Partially Ordered Time
Broad-sense Synchronous Circuits on Partially Ordered Time
Shunji Nishimura
Andrew Goldberg. Highway Dimension and Provably Efficient Shortest Path Algor...
Andrew Goldberg. Highway Dimension and Provably Efficient Shortest Path Algor...
Computer Science Club
Parallel Algorithms: Sort & Merge, Image Processing, Fault Tolerance
Parallel Algorithms: Sort & Merge, Image Processing, Fault Tolerance
University of Technology - Iraq
Parallel sorting algorithm
Parallel sorting algorithm
Richa Kumari
Control chap7
Control chap7
Mohd Ashraf Shabarshah
Control chap3
Control chap3
Mohd Ashraf Shabarshah
Aa sort-v4
Aa sort-v4
Malithi Edirisinghe
Lec 08 - DESIGN PROCEDURE
Lec 08 - DESIGN PROCEDURE
Vajira Thambawita
Parallel searching
Parallel searching
Md. Mahedi Mahfuj
MSc Presentation
MSc Presentation
eriprandopacces
Time-Evolving Graph Processing On Commodity Clusters
Time-Evolving Graph Processing On Commodity Clusters
Jen Aman
Contenu connexe
Tendances
Block diagram reduction techniques
Block diagram reduction techniques
parimalagandhi ayyavu
Generalized Isomorphism between Synchronous Circuits and State Machines
Generalized Isomorphism between Synchronous Circuits and State Machines
Shunji Nishimura
Large-scale computation without sacrificing expressiveness
Large-scale computation without sacrificing expressiveness
Sangjin Han
Fast and Scalable NUMA-based Thread Parallel Breadth-first Search
Fast and Scalable NUMA-based Thread Parallel Breadth-first Search
Yuichiro Yasui
GTC 2009 OpenGL Barthold
GTC 2009 OpenGL Barthold
Mark Kilgard
3.1,2,3 pushdown automata definition, moves & id
3.1,2,3 pushdown automata definition, moves & id
Sampath Kumar S
Class 25: Reversing Reverse
Class 25: Reversing Reverse
David Evans
Reed Solomon Matlab Projects Research Ideas
Reed Solomon Matlab Projects Research Ideas
Matlab Simulation
Reduction of multiple subsystem [compatibility mode]
Reduction of multiple subsystem [compatibility mode]
azroyyazid
MySQL Spatial Extensions - Boston MySQL Meetup April 2005
MySQL Spatial Extensions - Boston MySQL Meetup April 2005
Andrew Collins
Parallel quicksort cz. 1
Parallel quicksort cz. 1
Mikołaj Olszewski
Broad-sense Synchronous Circuits on Partially Ordered Time
Broad-sense Synchronous Circuits on Partially Ordered Time
Shunji Nishimura
Andrew Goldberg. Highway Dimension and Provably Efficient Shortest Path Algor...
Andrew Goldberg. Highway Dimension and Provably Efficient Shortest Path Algor...
Computer Science Club
Parallel Algorithms: Sort & Merge, Image Processing, Fault Tolerance
Parallel Algorithms: Sort & Merge, Image Processing, Fault Tolerance
University of Technology - Iraq
Parallel sorting algorithm
Parallel sorting algorithm
Richa Kumari
Control chap7
Control chap7
Mohd Ashraf Shabarshah
Control chap3
Control chap3
Mohd Ashraf Shabarshah
Aa sort-v4
Aa sort-v4
Malithi Edirisinghe
Lec 08 - DESIGN PROCEDURE
Lec 08 - DESIGN PROCEDURE
Vajira Thambawita
Parallel searching
Parallel searching
Md. Mahedi Mahfuj
Tendances
(20)
Block diagram reduction techniques
Block diagram reduction techniques
Generalized Isomorphism between Synchronous Circuits and State Machines
Generalized Isomorphism between Synchronous Circuits and State Machines
Large-scale computation without sacrificing expressiveness
Large-scale computation without sacrificing expressiveness
Fast and Scalable NUMA-based Thread Parallel Breadth-first Search
Fast and Scalable NUMA-based Thread Parallel Breadth-first Search
GTC 2009 OpenGL Barthold
GTC 2009 OpenGL Barthold
3.1,2,3 pushdown automata definition, moves & id
3.1,2,3 pushdown automata definition, moves & id
Class 25: Reversing Reverse
Class 25: Reversing Reverse
Reed Solomon Matlab Projects Research Ideas
Reed Solomon Matlab Projects Research Ideas
Reduction of multiple subsystem [compatibility mode]
Reduction of multiple subsystem [compatibility mode]
MySQL Spatial Extensions - Boston MySQL Meetup April 2005
MySQL Spatial Extensions - Boston MySQL Meetup April 2005
Parallel quicksort cz. 1
Parallel quicksort cz. 1
Broad-sense Synchronous Circuits on Partially Ordered Time
Broad-sense Synchronous Circuits on Partially Ordered Time
Andrew Goldberg. Highway Dimension and Provably Efficient Shortest Path Algor...
Andrew Goldberg. Highway Dimension and Provably Efficient Shortest Path Algor...
Parallel Algorithms: Sort & Merge, Image Processing, Fault Tolerance
Parallel Algorithms: Sort & Merge, Image Processing, Fault Tolerance
Parallel sorting algorithm
Parallel sorting algorithm
Control chap7
Control chap7
Control chap3
Control chap3
Aa sort-v4
Aa sort-v4
Lec 08 - DESIGN PROCEDURE
Lec 08 - DESIGN PROCEDURE
Parallel searching
Parallel searching
Similaire à Kineograph: Taking the Pulse of a Fast-Changing and Connected World
MSc Presentation
MSc Presentation
eriprandopacces
Time-Evolving Graph Processing On Commodity Clusters
Time-Evolving Graph Processing On Commodity Clusters
Jen Aman
State Management in Apache Flink : Consistent Stateful Distributed Stream Pro...
State Management in Apache Flink : Consistent Stateful Distributed Stream Pro...
Paris Carbone
Incremental pattern matching in the VIATRA2 model transformation system
Incremental pattern matching in the VIATRA2 model transformation system
Istvan Rath
Graph processing
Graph processing
yeahjs
Sorting and Routing on Hypercubes and Hypercubic Architectures
Sorting and Routing on Hypercubes and Hypercubic Architectures
CTOGreenITHub
A Closed-Form Expression for Queuing Delay in Rayleigh Fading Channels Using ...
A Closed-Form Expression for Queuing Delay in Rayleigh Fading Channels Using ...
Giacomo Verticale
Lifting 1
Lifting 1
douglaslyon
Live model transformations driven by incremental pattern matching
Live model transformations driven by incremental pattern matching
Istvan Rath
[Harvard CS264] 16 - Managing Dynamic Parallelism on GPUs: A Case Study of Hi...
[Harvard CS264] 16 - Managing Dynamic Parallelism on GPUs: A Case Study of Hi...
npinto
Eventually, Scylla Chooses Consistency
Eventually, Scylla Chooses Consistency
ScyllaDB
A sync in_sync
A sync in_sync
pponnath
Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
Yuichiro Yasui
Code GPU with CUDA - SIMT
Code GPU with CUDA - SIMT
Marina Kolpakova
Tall-and-skinny QR factorizations in MapReduce architectures
Tall-and-skinny QR factorizations in MapReduce architectures
David Gleich
So you think you can stream.pptx
So you think you can stream.pptx
Prakash Chockalingam
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Databricks
Inside LoLA - Experiences from building a state space tool for place transiti...
Inside LoLA - Experiences from building a state space tool for place transiti...
Universität Rostock
Time-evolving Graph Processing on Commodity Clusters: Spark Summit East talk ...
Time-evolving Graph Processing on Commodity Clusters: Spark Summit East talk ...
Spark Summit
Pipeline
Pipeline
krish198825
Similaire à Kineograph: Taking the Pulse of a Fast-Changing and Connected World
(20)
MSc Presentation
MSc Presentation
Time-Evolving Graph Processing On Commodity Clusters
Time-Evolving Graph Processing On Commodity Clusters
State Management in Apache Flink : Consistent Stateful Distributed Stream Pro...
State Management in Apache Flink : Consistent Stateful Distributed Stream Pro...
Incremental pattern matching in the VIATRA2 model transformation system
Incremental pattern matching in the VIATRA2 model transformation system
Graph processing
Graph processing
Sorting and Routing on Hypercubes and Hypercubic Architectures
Sorting and Routing on Hypercubes and Hypercubic Architectures
A Closed-Form Expression for Queuing Delay in Rayleigh Fading Channels Using ...
A Closed-Form Expression for Queuing Delay in Rayleigh Fading Channels Using ...
Lifting 1
Lifting 1
Live model transformations driven by incremental pattern matching
Live model transformations driven by incremental pattern matching
[Harvard CS264] 16 - Managing Dynamic Parallelism on GPUs: A Case Study of Hi...
[Harvard CS264] 16 - Managing Dynamic Parallelism on GPUs: A Case Study of Hi...
Eventually, Scylla Chooses Consistency
Eventually, Scylla Chooses Consistency
A sync in_sync
A sync in_sync
Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
Code GPU with CUDA - SIMT
Code GPU with CUDA - SIMT
Tall-and-skinny QR factorizations in MapReduce architectures
Tall-and-skinny QR factorizations in MapReduce architectures
So you think you can stream.pptx
So you think you can stream.pptx
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Scaling and Unifying SciKit Learn and Apache Spark Pipelines
Inside LoLA - Experiences from building a state space tool for place transiti...
Inside LoLA - Experiences from building a state space tool for place transiti...
Time-evolving Graph Processing on Commodity Clusters: Spark Summit East talk ...
Time-evolving Graph Processing on Commodity Clusters: Spark Summit East talk ...
Pipeline
Pipeline
Plus de Qian Lin
Fine-Grained, Secure and Efficient Data Provenance on Blockchain Systems
Fine-Grained, Secure and Efficient Data Provenance on Blockchain Systems
Qian Lin
PaxosStore: High-availability Storage Made Practical in WeChat
PaxosStore: High-availability Storage Made Practical in WeChat
Qian Lin
Trinity: A Distributed Graph Engine on a Memory Cloud
Trinity: A Distributed Graph Engine on a Memory Cloud
Qian Lin
Presto: Distributed Machine Learning and Graph Processing with Sparse Matrices
Presto: Distributed Machine Learning and Graph Processing with Sparse Matrices
Qian Lin
Adaptive Execution Support for Malleable Computation
Adaptive Execution Support for Malleable Computation
Qian Lin
C-Cube: Elastic Continuous Clustering in the Cloud
C-Cube: Elastic Continuous Clustering in the Cloud
Qian Lin
Optimizing Virtual Machines Using Hybrid Virtualization
Optimizing Virtual Machines Using Hybrid Virtualization
Qian Lin
Virtual Machine Performance
Virtual Machine Performance
Qian Lin
Be an Explorer, Be a Coder, Be a Writer
Be an Explorer, Be a Coder, Be a Writer
Qian Lin
SciMATE: A Novel MapReduce-Like Framework for Multiple Scientific Data Formats
SciMATE: A Novel MapReduce-Like Framework for Multiple Scientific Data Formats
Qian Lin
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
Qian Lin
In-situ MapReduce for Log Processing
In-situ MapReduce for Log Processing
Qian Lin
C-MR: Continuously Executing MapReduce Workflows on Multi-Core Processors
C-MR: Continuously Executing MapReduce Workflows on Multi-Core Processors
Qian Lin
Plus de Qian Lin
(13)
Fine-Grained, Secure and Efficient Data Provenance on Blockchain Systems
Fine-Grained, Secure and Efficient Data Provenance on Blockchain Systems
PaxosStore: High-availability Storage Made Practical in WeChat
PaxosStore: High-availability Storage Made Practical in WeChat
Trinity: A Distributed Graph Engine on a Memory Cloud
Trinity: A Distributed Graph Engine on a Memory Cloud
Presto: Distributed Machine Learning and Graph Processing with Sparse Matrices
Presto: Distributed Machine Learning and Graph Processing with Sparse Matrices
Adaptive Execution Support for Malleable Computation
Adaptive Execution Support for Malleable Computation
C-Cube: Elastic Continuous Clustering in the Cloud
C-Cube: Elastic Continuous Clustering in the Cloud
Optimizing Virtual Machines Using Hybrid Virtualization
Optimizing Virtual Machines Using Hybrid Virtualization
Virtual Machine Performance
Virtual Machine Performance
Be an Explorer, Be a Coder, Be a Writer
Be an Explorer, Be a Coder, Be a Writer
SciMATE: A Novel MapReduce-Like Framework for Multiple Scientific Data Formats
SciMATE: A Novel MapReduce-Like Framework for Multiple Scientific Data Formats
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
A Survey of Advanced Non-relational Database Systems: Approaches and Applicat...
In-situ MapReduce for Log Processing
In-situ MapReduce for Log Processing
C-MR: Continuously Executing MapReduce Workflows on Multi-Core Processors
C-MR: Continuously Executing MapReduce Workflows on Multi-Core Processors
Kineograph: Taking the Pulse of a Fast-Changing and Connected World
1.
Kineograph: Taking the
Pulse of a Fast-Changing and Connected World Speaker: LIN Qian http://www.comp.nus.edu.sg/~linqian
2.
Information time-sensitive rich
connections
3.
Challenges
4.
1. Timeliness guarantees
5.
2. Graph
6.
3. Graph-mining
7.
Kineograph distr. in-memory graph
storage incremental graph mining
8.
Master
Progress Continuous table Data feeds Ingest nodes Snapshooter Graph nodes Global consistent snapshots Graph Storage Computation Incremental computation on a static graph snapshot
9.
Graph computation Graph
updates
10.
Graph nodes
storage layer computation layer
11.
Storage layer key/value
store logical partitions
12.
Graph partitioning
edge-cut no locality consideration
13.
Snapshot
ingest nodes graph nodes global progress table
14.
Ingest node graph-update operations
sequence number
15.
Epoch commit protocol
16.
Progress table
s1 1 3 2 0 … … Global tx vector Ingest nodes s1 … sn sn 7 3 4 Snapshooter Partition u Partition v 1 2 4 s1 2 3 5 s1 … Epoch specified by progress … Graph nodes … table and snapshooter 4 6 7 sn 5 6 8 sn
17.
Graph update /
compute Pipeline Incoming Tweets … … Time Snapshot Si-1 Si Si+1 Construction Graph Epoch Ci Computation ti-1 ti ti’ ti’’ Timeliness
18.
Consistency
no global serialization (diff. from 2PL or t.s. ordering)
19.
Atomicity v
u v u
20.
Deterministic vertex creation
21.
Computation layer incremental graph-mining
22.
vertex-based computation model
23.
Incremental Graph
Computation Updates from other vertices N Detect Vertex Compute New Change Init Status Vertex Values Significantly? Graph-Scale Propagate Y Aggregation Updates
24.
Push model sender-side aggregation
25.
Pull model read a
subset of neighbors
26.
Execution model BSP +
Dynamic scheduling
27.
3 apps TunkRank
SP K-exposure
28.
TunkRank
29.
SP
30.
K-exposure
31.
Fault tolerance among servers Paxos-based
solution
32.
Ingest node failure
incarnation number
33.
Fault tolerance @
storage layer quorum-based replication
34.
Fault tolerance @ computation
layer roll back & re-execute primary/backup replication
35.
Incremental expansion
36.
Decaying
37.
C# 17,000 LOC
38.
Twitter feeds
8M vertices, 29M edges 100M tweets with 100K/sec power-law
39.
Graph-update throughput
40.
Incremental vs. Non-incremental
41.
Scalability
42.
Incoming data rate
43.
Failure recovery
Télécharger maintenant