Intermachine Parallelism

Folding@home distributed computing on PS3 now contributes 2/3rds of total performance (1050/1452 TFLOPS) but only 1/6 (~0.5M/~3M) CPUs in project. GPUs have even more impressive performance. 200W avg for PS3. inst.eecs.berkeley.edu/~cs61c UCB CS61C : Machine Structures Lecture 42 – Inter-machine Parallelism 2008-05-09 Lecturer SOE Dan Garcia http://fah-web.stanford.edu/cgi-bin/main.py?qtype=osstats

Review ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],en.wikipedia.org/wiki/Image:AmdahlsLaw.svg

Today’s Outline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Big Problems ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Example Applications ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Performance Requirements ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],www.epm.ornl.gov/chammp/chammp.html Reference:http://www.hpcwire.com/hpcwire/hpcwireWWW/04/0827/108259.html

High Resolution Climate Modeling on NERSC-3 P. Duffy, et al., LLNL

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],What Can We Do? Use Many CPUs!

Distributed Computing Themes ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Distributed Computing Challenges ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Programming Models: What is MPI? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],http://www.mpi-forum.org/ http://forum.stanford.edu/events/2007/plenary/slides/Olukotun.ppt http://www.tbray.org/ongoing/When/200x/2006/05/24/On-Grids

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Challenges with MPI

A New Hope: Google’s MapReduce ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],MapReduce Programming Model code.google.com/edu/parallel/mapreduce-tutorial.html

MapReduce WordCount Example ,[object Object],[object Object],[object Object],[object Object],[object Object]

MapReduce WordCount Diagram ah ah er ah if or or uh or ah if ah:1,1,1,1 ah:1 if:1 or:1 or:1 uh:1 or:1 ah:1 if:1 er:1 if:1,1 or:1,1,1 uh:1 ah:1 ah:1 er:1 4 1 2 3 1 file 1 file 2 file 3 file 4 file 5 file 6 file 7 (ah) (er) (if) (or) (uh) map (String input_key, String input_value): // input_key : doc name // input_value: doc contents for each word w in input_value: EmitIntermediate(w, "1"); reduce (String output_key, Iterator intermediate_values): // output_key : a word // output_values: a list of counts int result = 0; for each v in intermediate_values: result += ParseInt(v); Emit(AsString(result));

MapReduce in CS61A (and CS3?!) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Our Scheme MapReduce interface

MapReduce Advantages/Disadvantages ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Peer Instruction ABC 0: FFF 1: FF T 2: F T F 3: F TT 4: T FF 5: T F T 6: TT F 7: TTT ,[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],Peer Instruction Answer ,[object Object],[object Object],[object Object],ABC 0: FFF 1: FF T 2: F T F 3: F TT 4: T FF 5: T F T 6: TT F 7: TTT

Summary ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Bonus slides ,[object Object],[object Object],Bonus

To Learn More… ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Basic MPI Functions (1)

[object Object],[object Object],[object Object],[object Object],[object Object],Basic MPI Functions (2)

Basic MPI Functions (3) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],MPI Program Template

Intermachine Parallelism

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Intermachine Parallelism

Similaire à Intermachine Parallelism (20)

Plus de Sri Prasanna

Plus de Sri Prasanna (20)

Dernier

Dernier (20)

Intermachine Parallelism