Optimizing GPU to GPU Communication on Cray XK7
•
1 j'aime•1,163 vues
When developing an application for Cray XK7 systems, optimization of compute kernels is only a small part of maximizing scaling and performance. Programmers must consider the effect of the GPU’s distinct address space and the PCIe bus on application scalability. Without such considerations applications rapidly become limited by transfers to and from the GPU and fail to scale to large numbers of nodes. This paper will demonstrate methods for optimizing GPU to GPU communication and present XK7 results for these methods. This presentation was originally given at CUG 2013.
Recommandé
Contenu connexe
Tendances
Similaire à Optimizing GPU to GPU Communication on Cray XK7
Similaire à Optimizing GPU to GPU Communication on Cray XK7 (20)
Plus de Jeff Larkin
Plus de Jeff Larkin (17)
Dernier
Dernier (20)
Optimizing GPU to GPU Communication on Cray XK7
- 1. Optimizing GPU to GPU Communication on Cray XK7 Jeff Larkin
- 2. What Amdahl says about GPU communication If you make your GPU computation infinitely fast, performance will be bound by your communication. GPU-2-GPU communication has Higher latency (additional hop over PCIe) Lower bandwidth (limited by lowest bandwidth link) G2G communication cannot be an afterthought when running at scale.
- 3. CPU0 MPI_Send () GPU0 CPU1 MPI_Recv() GPU1 How do GPUs communicate? MPI But what really happens here?
- 5. Unified Virtual Addressing No UVA: Multiple Memory Spaces UVA : Single Address Space System Memory CPU GPU GPU Memory PCI-e 0x0000 0xFFFF 0x0000 0xFFFF System Memory CPU GPU GPU Memory PCI-e 0x0000 0xFFFF
- 6. Unified Virtual Addressing One address space for all CPU and GPU memory Determine physical memory location from a pointer value Enable libraries to simplify their interfaces (e.g. MPI and cudaMemcpy) Supported on Tesla starting with Fermi 64-bit applications on Linux and Windows TCC Pinned fabric Buffer Host Buffer memcpy GPU Buffer cudaMemcpy Pinned fabric BufferPinned fabric Buffer
- 7. MPI+CUDA //MPI rank 0 MPI_Send(s_buf_d,size,…); //MPI rank n-1 MPI_Recv(r_buf_d,size,…); With UVA and CUDA-aware MPI //MPI rank 0 cudaMemcpy(s_buf_h,s_buf_d,size,…); MPI_Send(s_buf_h,size,…); //MPI rank n-1 MPI_Recv(r_buf_h,size,…); cudaMemcpy(r_buf_d,r_buf_h,size,…); No UVA and regular MPI CUDA-aware MPI makes MPI+CUDA easier.
- 8. CUDA-Aware MPI Libraries May Use RDMA to completely remove CPU from the picture. Stage GPU buffers through CPU memory automatically Pipeline messages All the programmer needs to know is they’ve passed a GPU pointer to MPI, the library developer can optimize the rest GPU 0 GPU 1 GPU 0 GPU 1 CPU 0 CPU 1 GPU 0 GPU 1
- 9. GPU-Awareness in Cray MPI Cray began supporting GPU-awareness in 5.6.3 Functions on XK7, but not optimally performing Expected to work very well on XC30 Must be explicitly enabled via run-time environment variable MPICH_RDMA_ENABLED_CUDA Works with both CUDA and OpenACC Version 5.6.4 adds a pipelining feature that should help large messages Enabled with MPICH_G2G_PIPELINE
- 10. OMB Latency Host-to-Host will always have the lowest latency (fewest hops) Staging through host memory explicitly adds significant latency GPU-aware library is able to fall in the middle. Note: 2 nodes on separate blades.
- 11. OMB Bandwidth Once again, H2H wins out (probably by a difference of latency) Direct RDMA suffers badly with this benchmark. Setting MPICH_G2G_PIPELINE=64 pipelines messages and opens up more concurrency. Hand- Pipelining Came ~Here
- 12. OMB Bandwidth, Varying Pipelines OMB sends messages in a window of 64, so that is naturally optimal. Counter-intuitive that no intermediate values seemed to help. Additionally tried varying chunk sizes with no benefit.
- 13. Optimizing Performance of a Message MPI vendors know how to optimize performance for an interconnect Different approaches for different message sizes Multiple algorithms Unfortunately, on the XK7, this may not be the optimal approach.
- 14. MPI Lacks the Ability to Express Dependencies One way to negate the cost of G2G communication is to overlap with something computation. Restructuring order of computation may allow such overlapping Some communication patterns have a natural concurrency that isn’t easily exploited. N W E Exchange Cells With N,S,E,W Neighbors. S
- 15. Exploiting Communication Concurrency This cannot be expressed in GPU-aware MPI today. Pack North D2H North Transfer North H2D North Unpack North Pack East D2H East Transfer East H2D East Unpack East Pack South D2H South Transfer South H2D South Unpack South Pack West D2H West Transfer West H2D West Unpack West
- 16. Concurrent Messaging Pseudocode do i=N,W MPI_Irecv(i) enddo do i=N,W packBufferAsync(i) copyBufferD2HAsync(i) enddo while(more to send/recv) if Irecv completed copyBufferH2DAsync unpackBufferAsync() endif if D2H completed MPI_Isend() endif done
- 17. Talks Related to this Optimization HOMME - Matt Norman – CUG2012 S3D - John Levesque – CUG2013
- 18. Summary Optimizing kernels will only take you so far as you scale, communication cannot be an afterthought. GPU-aware MPI libraries are becoming available Easier to program Can optimize performance of individual message transfers Some communication patterns have a natural concurrency that can be exploited to make communication “free”, but this takes additional effort.