SlideShare une entreprise Scribd logo
1  sur  38
Télécharger pour lire hors ligne
Introduction to Parallel
      Computing
         Part Ib
Processor Intercommunication
In part 1b we will look at the interconnection
network between processors. Using these
connections, various communication patterns
can be used to transport information from
one or more source processors, to one or more
destination processors.
Processor Topologies (1)
There are several ways in which processors
can be interconnected. The most important
include
    • Bus               • Ring
    • Star              • Mesh
    • Tree              • Wraparound mesh
    • Fully connected   • Hypercube
Topology Issues
Before looking at some of the major processor
topologies we have to know what makes a
certain topology well or ill suited to connect
processors.
There are two aspects of topologies that
should be looked at. These are scalability and
cost (communication and hardware).
Terminology
Communication is one-to-one when one
processor sends a message to another one.
In one-to-all or broadcast, one processor
  sends
a message to all other processors. In all-to-one
communication, all processors send their
message to one processor. Other forms of
communication include gather and all-to-all.
Connection Properties
There are three elements that are used in
Building the interconnection network. These
are the wire, a relay node, and a processor
node. The latter two elements can at a given
time send or receive only one message, no
matter how many wires enter or leave the
element.
Bus Topology (1)

      P2        P4        P…




P1         P3        P5
Bus Topology (2)
Hardware cost   1
One-to-one      1
One-to-all      1
All-to-all      p

Problems        Bus becomes bottleneck
Star Topology (1)
           P3
      P2        P4


 P1        P0        P5


      P8        P6
           P7
Star Topology (2)
Hardware cost   p–1
One-to-one      1 or 2
One-to-all      p–1
All-to-all      2 · (p – 1)

Problems        Central processor becomes
                bottleneck.
Tree Topology (1)




P1   P2   P3   P4   P5   P6   P7   P8
Tree Topology (2)
Hardware cost   2p – 2, when p power of 2
One-to-one      2 · 2log p
One-to-all      (2log p) · (1 + 2log p)
All-to-all      2 · (2log p) · (1 + 2log p)

Problems        Top node becomes bottleneck
                This can be solved by adding
                more wires at the top (fat tree)
Tree Topology – One-to-all




 P1   P2   P3   P4   P5   P6   P7   P8
Fully Connected Topology (1)

            P2   P3


       P1             P4


            P6   P5
Fully Connected Topology (2)
Hardware cost p·(p – 1)/2
One-to-one    1
One-to-all     2log p
All-to-all    2·2log p

Problems       Hardware cost increases
               quadratically with respect to p
Ring Topology (1)

      P2   P3


 P1             P4


      P6   P5
Ring Topology (2)
Hardware cost   p
One-to-one      p / 2
One-to-all      p / 2
All-to-all      2·p / 2

Problems        Processors are loaded with
                transport jobs. But hardware
                and communication cost low
2D-Mesh Topology (1)
 P12   P13   P14   P15


 P8    P9    P10   P11


 P4    P5    P6    P7


 P0    P1    P2    P3
2D-Mesh Topology (2)
Hardware cost   2·(√p)·(√p – 1)
One-to-one      2(√p – 1)
One-to-all      2(√p – 1)
All-to-all      2·√p / 2

Remarks         Scalable both in network and
                communication cost. Can be
                found in many architectures.
Mesh – All-to-all
Step 0
     6
     5
     4
     3
     2
     1


         P1     P2   P3      P4      P5

   1 23 45 1 23 45 1 23 45 1 23 45 1 23 45
2D Wrap-around Mesh (1)
   P12   P13   P14   P15


   P8    P9    P10   P11


   P4    P5    P6    P7


   P0    P1    P2    P3
2D Wrap-around Mesh (2)
Hardware cost   2p
One-to-one      2·√p / 2
One-to-all      2· √p / 2
All-to-all      4· √p / 2

Remarks         Scalable both in network and
                communication cost. Can be
                found in many architectures.
2D Wrap-around – One-to-all
     P12   P13   P14   P15


     P8    P9    P10   P11


     P4    P5    P6    P7


     P0    P1    P2    P3
Hypercube Topology (1)

        1D
                2D




   3D

                         4D
Hypercube Construction
   4D
   3D
   2D
   1D
Hypercube
Hypercube Topology (2)
Hardware cost     (p / 2) · 2log p
One-to-one      2
                 log p
One-to-all      2
                 log p
All-to-all      2 · 2log p
Remarks         The most elegant design, also
                when it comes down to routing
                algorithms. But difficult to build
                in hardware.
4D Hypercube – One-to-all
Communication Issues
There are some things left to be said about
communication :
• In general the time required for data transmission
  is of the form startup-time + transfer speed *
  package size. So it is more efficient to sent one
  large package instead of many small packages
  (startup-time can be high, e.g. think about internet)
• Asynchronous vs. Synchronous transfer and
  deadlocks.
Asynchronous vs. Synchronous
The major difference between asynchronous
and synchronous communication is that the
first methods sends a message and continues,
while the second sends a message and waits
for the receiver program to receive the
message.
Example asynchronous comm.
    Processor A      Processor B
  Instruction A1
  InstructionA1    Instruction B1
                   InstructionB1
  Send to B
  Send to B        Instruction B2
                   Instruction B2
  Instruction A2
  Instruction A2   Instruction B3
                   Instruction B3
  Instruction A3
  Instruction A3   Instruction B4
                   Instruction B4
  Instruction A4
  Instruction A4   Receive from A
                            from A
  Instruction A5
  Instruction A5   Instruction B5
                   Instruction B5
Asynchronous Comm.
• Convenient because processors do not have to
  wait for each other.
• However, we often need to know whether or not
  the destination processors has received the data,
  this often requires some checking code later in the
  program.
• Need to know whether the OS supports reliable
  communication layers.
• Receive instruction may or may not be blocking.
Example synchronous comm.
   Processor A      Processor B
 Instruction A1
 InstructionA1    Instruction B1
                  InstructionB1
 Send to B
 Send to B        Instruction B2
                  Instruction B2
 Instruction A2
 Instruction A2   Instruction B3
                  Instruction B3
 Instruction A3   Instruction B4
                  Instruction B4
 Instruction A4   Receive from A
                           from A
 Instruction A5   Instruction B5
                  Instruction B5
Synchronous comm.
• Both send and receive are blocking
• Processors have to wait for each other. This
  reduces efficiency.
• Implicitly offers a synchronisation point.
• Easy to program because fewer unexpected
  situations can arise.
• Problem: Deadlocks may occur.
Deadlocks (1)
A deadlock is a situation where two or more
processors are waiting to for each other
infinitely.
Deadlocks (2)

         Processor A                Processor B
      Send to B                  Send to A
      Receive from B             Receive from A




Note: Only occurs with synchronous communication.
Deadlocks (3)

  Processor A      Processor B
Send to B        Receive from A
Receive from B   Send to A
Deadlocks (4)
               P2      P3


        P1                   P4


               P6      P5


Pattern: P1 → P2 → P5 → P4 → P6 → P3 → P1
End of Part I
Are there any questions regarding part I

Contenu connexe

Tendances

Applications of paralleL processing
Applications of paralleL processingApplications of paralleL processing
Applications of paralleL processingPage Maker
 
Pipelining and vector processing
Pipelining and vector processingPipelining and vector processing
Pipelining and vector processingKamal Acharya
 
Parallel computing chapter 3
Parallel computing chapter 3Parallel computing chapter 3
Parallel computing chapter 3Md. Mahedi Mahfuj
 
Computer architecture multi processor
Computer architecture multi processorComputer architecture multi processor
Computer architecture multi processorMazin Alwaaly
 
Multivector and multiprocessor
Multivector and multiprocessorMultivector and multiprocessor
Multivector and multiprocessorKishan Panara
 
Input output organization
Input output organizationInput output organization
Input output organizationabdulugc
 
CPU Scheduling in OS Presentation
CPU Scheduling in OS  PresentationCPU Scheduling in OS  Presentation
CPU Scheduling in OS Presentationusmankiyani1
 
Multicast routing protocols in adhoc networks
Multicast routing protocols in adhoc networksMulticast routing protocols in adhoc networks
Multicast routing protocols in adhoc networksPradeep Kumar TS
 
Introduction to Parallel Computing
Introduction to Parallel ComputingIntroduction to Parallel Computing
Introduction to Parallel ComputingAkhila Prabhakaran
 
Multi processor scheduling
Multi  processor schedulingMulti  processor scheduling
Multi processor schedulingShashank Kapoor
 
MPI message passing interface
MPI message passing interfaceMPI message passing interface
MPI message passing interfaceMohit Raghuvanshi
 
CS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMSCS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMSKathirvel Ayyaswamy
 
ALOHA Protocol (in detail)
ALOHA Protocol (in detail)ALOHA Protocol (in detail)
ALOHA Protocol (in detail)Hinal Lunagariya
 
Disk Scheduling Algorithm in Operating System
Disk Scheduling Algorithm in Operating SystemDisk Scheduling Algorithm in Operating System
Disk Scheduling Algorithm in Operating SystemMeghaj Mallick
 

Tendances (20)

Applications of paralleL processing
Applications of paralleL processingApplications of paralleL processing
Applications of paralleL processing
 
Pipelining and vector processing
Pipelining and vector processingPipelining and vector processing
Pipelining and vector processing
 
Parallel computing chapter 3
Parallel computing chapter 3Parallel computing chapter 3
Parallel computing chapter 3
 
CS6601 DISTRIBUTED SYSTEMS
CS6601 DISTRIBUTED SYSTEMSCS6601 DISTRIBUTED SYSTEMS
CS6601 DISTRIBUTED SYSTEMS
 
Computer architecture multi processor
Computer architecture multi processorComputer architecture multi processor
Computer architecture multi processor
 
Processes and threads
Processes and threadsProcesses and threads
Processes and threads
 
Array Processor
Array ProcessorArray Processor
Array Processor
 
Multivector and multiprocessor
Multivector and multiprocessorMultivector and multiprocessor
Multivector and multiprocessor
 
Input output organization
Input output organizationInput output organization
Input output organization
 
CPU Scheduling in OS Presentation
CPU Scheduling in OS  PresentationCPU Scheduling in OS  Presentation
CPU Scheduling in OS Presentation
 
Distributed Operating System_1
Distributed Operating System_1Distributed Operating System_1
Distributed Operating System_1
 
Multicast routing protocols in adhoc networks
Multicast routing protocols in adhoc networksMulticast routing protocols in adhoc networks
Multicast routing protocols in adhoc networks
 
Introduction to Parallel Computing
Introduction to Parallel ComputingIntroduction to Parallel Computing
Introduction to Parallel Computing
 
Multi processor scheduling
Multi  processor schedulingMulti  processor scheduling
Multi processor scheduling
 
MPI message passing interface
MPI message passing interfaceMPI message passing interface
MPI message passing interface
 
SCHEDULING ALGORITHMS
SCHEDULING ALGORITHMSSCHEDULING ALGORITHMS
SCHEDULING ALGORITHMS
 
Memory management
Memory managementMemory management
Memory management
 
CS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMSCS9222 ADVANCED OPERATING SYSTEMS
CS9222 ADVANCED OPERATING SYSTEMS
 
ALOHA Protocol (in detail)
ALOHA Protocol (in detail)ALOHA Protocol (in detail)
ALOHA Protocol (in detail)
 
Disk Scheduling Algorithm in Operating System
Disk Scheduling Algorithm in Operating SystemDisk Scheduling Algorithm in Operating System
Disk Scheduling Algorithm in Operating System
 

En vedette

Introduction To Parallel Computing
Introduction To Parallel ComputingIntroduction To Parallel Computing
Introduction To Parallel ComputingJörn Dinkla
 
Higher nab preparation
Higher nab preparationHigher nab preparation
Higher nab preparationscaddell
 
Introduction to Parallel Computing
Introduction to Parallel ComputingIntroduction to Parallel Computing
Introduction to Parallel ComputingRoshan Karunarathna
 
Highly Surmountable Challenges in Ruby+OMR JIT Compilation
Highly Surmountable Challenges in Ruby+OMR JIT CompilationHighly Surmountable Challenges in Ruby+OMR JIT Compilation
Highly Surmountable Challenges in Ruby+OMR JIT CompilationMatthew Gaudet
 
network ram parallel computing
network ram parallel computingnetwork ram parallel computing
network ram parallel computingNiranjana Ambadi
 
Introduction to parallel_computing
Introduction to parallel_computingIntroduction to parallel_computing
Introduction to parallel_computingMehul Patel
 
VLSI Design(Fabrication)
VLSI Design(Fabrication)VLSI Design(Fabrication)
VLSI Design(Fabrication)Trijit Mallick
 
Database management system chapter15
Database management system chapter15Database management system chapter15
Database management system chapter15Md. Mahedi Mahfuj
 
Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)Mr SMAK
 
Parallel computing
Parallel computingParallel computing
Parallel computingvirend111
 
Lecture 1
Lecture 1Lecture 1
Lecture 1Mr SMAK
 
Parallel computing chapter 2
Parallel computing chapter 2Parallel computing chapter 2
Parallel computing chapter 2Md. Mahedi Mahfuj
 
Advanced Comuter Architecture Ch6 Problem Solutions
Advanced Comuter Architecture Ch6 Problem SolutionsAdvanced Comuter Architecture Ch6 Problem Solutions
Advanced Comuter Architecture Ch6 Problem SolutionsJoe Christensen
 
Computer architecture kai hwang
Computer architecture   kai hwangComputer architecture   kai hwang
Computer architecture kai hwangSumedha
 
Advanced computer architecture
Advanced computer architectureAdvanced computer architecture
Advanced computer architectureMd. Mahedi Mahfuj
 

En vedette (20)

Parallel computing(2)
Parallel computing(2)Parallel computing(2)
Parallel computing(2)
 
Introduction To Parallel Computing
Introduction To Parallel ComputingIntroduction To Parallel Computing
Introduction To Parallel Computing
 
Higher nab preparation
Higher nab preparationHigher nab preparation
Higher nab preparation
 
Introduction to Parallel Computing
Introduction to Parallel ComputingIntroduction to Parallel Computing
Introduction to Parallel Computing
 
Highly Surmountable Challenges in Ruby+OMR JIT Compilation
Highly Surmountable Challenges in Ruby+OMR JIT CompilationHighly Surmountable Challenges in Ruby+OMR JIT Compilation
Highly Surmountable Challenges in Ruby+OMR JIT Compilation
 
network ram parallel computing
network ram parallel computingnetwork ram parallel computing
network ram parallel computing
 
Chapter6 pipelining
Chapter6  pipeliningChapter6  pipelining
Chapter6 pipelining
 
Introduction to parallel_computing
Introduction to parallel_computingIntroduction to parallel_computing
Introduction to parallel_computing
 
VLSI Design(Fabrication)
VLSI Design(Fabrication)VLSI Design(Fabrication)
VLSI Design(Fabrication)
 
Database management system chapter15
Database management system chapter15Database management system chapter15
Database management system chapter15
 
Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)Fyp list batch-2009 (project approval -rejected list)
Fyp list batch-2009 (project approval -rejected list)
 
Parallel computing
Parallel computingParallel computing
Parallel computing
 
Lecture 1
Lecture 1Lecture 1
Lecture 1
 
Parallel computing chapter 2
Parallel computing chapter 2Parallel computing chapter 2
Parallel computing chapter 2
 
Advanced Comuter Architecture Ch6 Problem Solutions
Advanced Comuter Architecture Ch6 Problem SolutionsAdvanced Comuter Architecture Ch6 Problem Solutions
Advanced Comuter Architecture Ch6 Problem Solutions
 
Parallel processing
Parallel processingParallel processing
Parallel processing
 
Computer architecture kai hwang
Computer architecture   kai hwangComputer architecture   kai hwang
Computer architecture kai hwang
 
Parallel Algorithms
Parallel AlgorithmsParallel Algorithms
Parallel Algorithms
 
Parallel Algorithms
Parallel AlgorithmsParallel Algorithms
Parallel Algorithms
 
Advanced computer architecture
Advanced computer architectureAdvanced computer architecture
Advanced computer architecture
 

Similaire à Parallel computing(1)

Efficient Topology Discovery in Software Defined Networks
Efficient Topology Discovery in Software Defined NetworksEfficient Topology Discovery in Software Defined Networks
Efficient Topology Discovery in Software Defined NetworksFarzaneh Pakzad
 
Computer network (13)
Computer network (13)Computer network (13)
Computer network (13)NYversity
 
Prescriptive Topology Daemon - ptmd
Prescriptive Topology Daemon - ptmdPrescriptive Topology Daemon - ptmd
Prescriptive Topology Daemon - ptmdNat Morris
 
MODULE II.pdf
MODULE II.pdfMODULE II.pdf
MODULE II.pdfADARSHN40
 
Network Topologies, L1-L2 Basics, Networking Devices
Network Topologies, L1-L2 Basics, Networking DevicesNetwork Topologies, L1-L2 Basics, Networking Devices
Network Topologies, L1-L2 Basics, Networking DevicesAalok Shah
 
IRJET - Improving TCP Performance in Wireless Networks
IRJET -  	  Improving TCP Performance in Wireless NetworksIRJET -  	  Improving TCP Performance in Wireless Networks
IRJET - Improving TCP Performance in Wireless NetworksIRJET Journal
 
Transport protocols
Transport protocolsTransport protocols
Transport protocolsAman Jaiswal
 
RIFT A New Approach to Building DC Fabrics
RIFT A New Approach to Building DC FabricsRIFT A New Approach to Building DC Fabrics
RIFT A New Approach to Building DC FabricsMyNOG
 
FEC & File Multicast
FEC & File MulticastFEC & File Multicast
FEC & File MulticastYoss Cohen
 
Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...
Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...
Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...Tokyo Institute of Technology
 
A Flexible Router Architecture for 3D Network-on-Chips
A Flexible Router Architecture for 3D Network-on-ChipsA Flexible Router Architecture for 3D Network-on-Chips
A Flexible Router Architecture for 3D Network-on-ChipsMostafa Khamis
 
IRJET- Performance Improvement of Wireless Network using Modern Simulation Tools
IRJET- Performance Improvement of Wireless Network using Modern Simulation ToolsIRJET- Performance Improvement of Wireless Network using Modern Simulation Tools
IRJET- Performance Improvement of Wireless Network using Modern Simulation ToolsIRJET Journal
 

Similaire à Parallel computing(1) (20)

Efficient Topology Discovery in Software Defined Networks
Efficient Topology Discovery in Software Defined NetworksEfficient Topology Discovery in Software Defined Networks
Efficient Topology Discovery in Software Defined Networks
 
Chapter 4 pc
Chapter 4 pcChapter 4 pc
Chapter 4 pc
 
Computer network (13)
Computer network (13)Computer network (13)
Computer network (13)
 
Prescriptive Topology Daemon - ptmd
Prescriptive Topology Daemon - ptmdPrescriptive Topology Daemon - ptmd
Prescriptive Topology Daemon - ptmd
 
5G mmwaves - problems and solutions (graduation project)
5G mmwaves - problems and solutions (graduation project)5G mmwaves - problems and solutions (graduation project)
5G mmwaves - problems and solutions (graduation project)
 
MODULE II.ppt
MODULE II.pptMODULE II.ppt
MODULE II.ppt
 
MODULE II.pdf
MODULE II.pdfMODULE II.pdf
MODULE II.pdf
 
Lec9
Lec9Lec9
Lec9
 
Network Topologies, L1-L2 Basics, Networking Devices
Network Topologies, L1-L2 Basics, Networking DevicesNetwork Topologies, L1-L2 Basics, Networking Devices
Network Topologies, L1-L2 Basics, Networking Devices
 
IRJET - Improving TCP Performance in Wireless Networks
IRJET -  	  Improving TCP Performance in Wireless NetworksIRJET -  	  Improving TCP Performance in Wireless Networks
IRJET - Improving TCP Performance in Wireless Networks
 
Transport Protocols
Transport ProtocolsTransport Protocols
Transport Protocols
 
Transport protocols
Transport protocolsTransport protocols
Transport protocols
 
gio's tesi
gio's tesigio's tesi
gio's tesi
 
RIFT A New Approach to Building DC Fabrics
RIFT A New Approach to Building DC FabricsRIFT A New Approach to Building DC Fabrics
RIFT A New Approach to Building DC Fabrics
 
FEC & File Multicast
FEC & File MulticastFEC & File Multicast
FEC & File Multicast
 
CN UNIT III.pptx
CN UNIT III.pptxCN UNIT III.pptx
CN UNIT III.pptx
 
Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...
Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...
Integrating Cache Oblivious Approach with Modern Processor Architecture: The ...
 
Networking concepts
Networking conceptsNetworking concepts
Networking concepts
 
A Flexible Router Architecture for 3D Network-on-Chips
A Flexible Router Architecture for 3D Network-on-ChipsA Flexible Router Architecture for 3D Network-on-Chips
A Flexible Router Architecture for 3D Network-on-Chips
 
IRJET- Performance Improvement of Wireless Network using Modern Simulation Tools
IRJET- Performance Improvement of Wireless Network using Modern Simulation ToolsIRJET- Performance Improvement of Wireless Network using Modern Simulation Tools
IRJET- Performance Improvement of Wireless Network using Modern Simulation Tools
 

Plus de Md. Mahedi Mahfuj (20)

Bengali optical character recognition system
Bengali optical character recognition systemBengali optical character recognition system
Bengali optical character recognition system
 
Message passing interface
Message passing interfaceMessage passing interface
Message passing interface
 
Parallel searching
Parallel searchingParallel searching
Parallel searching
 
Clustering manual
Clustering manualClustering manual
Clustering manual
 
Matrix multiplication graph
Matrix multiplication graphMatrix multiplication graph
Matrix multiplication graph
 
Strategy pattern
Strategy patternStrategy pattern
Strategy pattern
 
Observer pattern
Observer patternObserver pattern
Observer pattern
 
Mediator pattern
Mediator patternMediator pattern
Mediator pattern
 
Database management system chapter16
Database management system chapter16Database management system chapter16
Database management system chapter16
 
Database management system chapter12
Database management system chapter12Database management system chapter12
Database management system chapter12
 
Strategies in job search process
Strategies in job search processStrategies in job search process
Strategies in job search process
 
Report writing(short)
Report writing(short)Report writing(short)
Report writing(short)
 
Report writing(long)
Report writing(long)Report writing(long)
Report writing(long)
 
Job search_resume
Job search_resumeJob search_resume
Job search_resume
 
Job search_interview
Job search_interviewJob search_interview
Job search_interview
 
Apache hadoop & map reduce
Apache hadoop & map reduceApache hadoop & map reduce
Apache hadoop & map reduce
 
Basic and logical implementation of r language
Basic and logical implementation of r language Basic and logical implementation of r language
Basic and logical implementation of r language
 
Map reduce
Map reduceMap reduce
Map reduce
 
R with excel
R with excelR with excel
R with excel
 
R language
R languageR language
R language
 

Dernier

PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServiceRenan Moreira de Oliveira
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 

Dernier (20)

PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 

Parallel computing(1)

  • 1. Introduction to Parallel Computing Part Ib
  • 2. Processor Intercommunication In part 1b we will look at the interconnection network between processors. Using these connections, various communication patterns can be used to transport information from one or more source processors, to one or more destination processors.
  • 3. Processor Topologies (1) There are several ways in which processors can be interconnected. The most important include • Bus • Ring • Star • Mesh • Tree • Wraparound mesh • Fully connected • Hypercube
  • 4. Topology Issues Before looking at some of the major processor topologies we have to know what makes a certain topology well or ill suited to connect processors. There are two aspects of topologies that should be looked at. These are scalability and cost (communication and hardware).
  • 5. Terminology Communication is one-to-one when one processor sends a message to another one. In one-to-all or broadcast, one processor sends a message to all other processors. In all-to-one communication, all processors send their message to one processor. Other forms of communication include gather and all-to-all.
  • 6. Connection Properties There are three elements that are used in Building the interconnection network. These are the wire, a relay node, and a processor node. The latter two elements can at a given time send or receive only one message, no matter how many wires enter or leave the element.
  • 7. Bus Topology (1) P2 P4 P… P1 P3 P5
  • 8. Bus Topology (2) Hardware cost 1 One-to-one 1 One-to-all 1 All-to-all p Problems Bus becomes bottleneck
  • 9. Star Topology (1) P3 P2 P4 P1 P0 P5 P8 P6 P7
  • 10. Star Topology (2) Hardware cost p–1 One-to-one 1 or 2 One-to-all p–1 All-to-all 2 · (p – 1) Problems Central processor becomes bottleneck.
  • 11. Tree Topology (1) P1 P2 P3 P4 P5 P6 P7 P8
  • 12. Tree Topology (2) Hardware cost 2p – 2, when p power of 2 One-to-one 2 · 2log p One-to-all (2log p) · (1 + 2log p) All-to-all 2 · (2log p) · (1 + 2log p) Problems Top node becomes bottleneck This can be solved by adding more wires at the top (fat tree)
  • 13. Tree Topology – One-to-all P1 P2 P3 P4 P5 P6 P7 P8
  • 14. Fully Connected Topology (1) P2 P3 P1 P4 P6 P5
  • 15. Fully Connected Topology (2) Hardware cost p·(p – 1)/2 One-to-one 1 One-to-all 2log p All-to-all 2·2log p Problems Hardware cost increases quadratically with respect to p
  • 16. Ring Topology (1) P2 P3 P1 P4 P6 P5
  • 17. Ring Topology (2) Hardware cost p One-to-one p / 2 One-to-all p / 2 All-to-all 2·p / 2 Problems Processors are loaded with transport jobs. But hardware and communication cost low
  • 18. 2D-Mesh Topology (1) P12 P13 P14 P15 P8 P9 P10 P11 P4 P5 P6 P7 P0 P1 P2 P3
  • 19. 2D-Mesh Topology (2) Hardware cost 2·(√p)·(√p – 1) One-to-one 2(√p – 1) One-to-all 2(√p – 1) All-to-all 2·√p / 2 Remarks Scalable both in network and communication cost. Can be found in many architectures.
  • 20. Mesh – All-to-all Step 0 6 5 4 3 2 1 P1 P2 P3 P4 P5 1 23 45 1 23 45 1 23 45 1 23 45 1 23 45
  • 21. 2D Wrap-around Mesh (1) P12 P13 P14 P15 P8 P9 P10 P11 P4 P5 P6 P7 P0 P1 P2 P3
  • 22. 2D Wrap-around Mesh (2) Hardware cost 2p One-to-one 2·√p / 2 One-to-all 2· √p / 2 All-to-all 4· √p / 2 Remarks Scalable both in network and communication cost. Can be found in many architectures.
  • 23. 2D Wrap-around – One-to-all P12 P13 P14 P15 P8 P9 P10 P11 P4 P5 P6 P7 P0 P1 P2 P3
  • 24. Hypercube Topology (1) 1D 2D 3D 4D
  • 25. Hypercube Construction 4D 3D 2D 1D Hypercube
  • 26. Hypercube Topology (2) Hardware cost (p / 2) · 2log p One-to-one 2 log p One-to-all 2 log p All-to-all 2 · 2log p Remarks The most elegant design, also when it comes down to routing algorithms. But difficult to build in hardware.
  • 27. 4D Hypercube – One-to-all
  • 28. Communication Issues There are some things left to be said about communication : • In general the time required for data transmission is of the form startup-time + transfer speed * package size. So it is more efficient to sent one large package instead of many small packages (startup-time can be high, e.g. think about internet) • Asynchronous vs. Synchronous transfer and deadlocks.
  • 29. Asynchronous vs. Synchronous The major difference between asynchronous and synchronous communication is that the first methods sends a message and continues, while the second sends a message and waits for the receiver program to receive the message.
  • 30. Example asynchronous comm. Processor A Processor B Instruction A1 InstructionA1 Instruction B1 InstructionB1 Send to B Send to B Instruction B2 Instruction B2 Instruction A2 Instruction A2 Instruction B3 Instruction B3 Instruction A3 Instruction A3 Instruction B4 Instruction B4 Instruction A4 Instruction A4 Receive from A from A Instruction A5 Instruction A5 Instruction B5 Instruction B5
  • 31. Asynchronous Comm. • Convenient because processors do not have to wait for each other. • However, we often need to know whether or not the destination processors has received the data, this often requires some checking code later in the program. • Need to know whether the OS supports reliable communication layers. • Receive instruction may or may not be blocking.
  • 32. Example synchronous comm. Processor A Processor B Instruction A1 InstructionA1 Instruction B1 InstructionB1 Send to B Send to B Instruction B2 Instruction B2 Instruction A2 Instruction A2 Instruction B3 Instruction B3 Instruction A3 Instruction B4 Instruction B4 Instruction A4 Receive from A from A Instruction A5 Instruction B5 Instruction B5
  • 33. Synchronous comm. • Both send and receive are blocking • Processors have to wait for each other. This reduces efficiency. • Implicitly offers a synchronisation point. • Easy to program because fewer unexpected situations can arise. • Problem: Deadlocks may occur.
  • 34. Deadlocks (1) A deadlock is a situation where two or more processors are waiting to for each other infinitely.
  • 35. Deadlocks (2) Processor A Processor B Send to B Send to A Receive from B Receive from A Note: Only occurs with synchronous communication.
  • 36. Deadlocks (3) Processor A Processor B Send to B Receive from A Receive from B Send to A
  • 37. Deadlocks (4) P2 P3 P1 P4 P6 P5 Pattern: P1 → P2 → P5 → P4 → P6 → P3 → P1
  • 38. End of Part I Are there any questions regarding part I