PCCC21：日本電気株式会社「一台何役？SX-Aurora TSUBASA最新情報」

© NEC Corporation 2021
SX-Aurora TSUBASA -PCIe card type vector computer-
November, 2021
NEC Corporation
SC21 Exhibitor Forum

1. History of SX Vector Supercomputer
2. Features of SX-Aurora TSUBASA
3. World famous HPC Centers utilizing SX-Aurora TSUBASA
4. Start card business
5. SX-Aurora TSUBASA Software
6. Value of Vector Engine
7. Roadmap and the future
8. Partnering
Agenda

3
Over 35years experience for
High Sustained Performance
History
of
SX
Vector
Supercomputer Technology:
CPU Frequency:
CPU Performance:
CPU Memory Bandwidth:
Technology:
CPU Frequency:
CPU Performance:
Technology:
CPU Frequency:
CPU Performance:
Technology:
CPU Frequency:
CPU Performance:
Technology:
CPU Frequency:
CPU Performance:
Technology:
CPU Frequency:
CPU Performance:
Technology:
CPU Frequency:
CPU Performance:
Technology:
CPU Frequency:
CPU Performance:
Technology:
CPU Frequency:
CPU Performance:
CPU Memory Bandwidth
Bipolar
166 MHz
1.3 GFlops
10.7 GB/sec
Bipolar
340 MHz
5.5 GFlops
12.8 GB/sec
350 nm
125 MHz
2.0 GFlops
16.0 GB/sec
250 nm
250 MHz
8.0 GFlops
64.0 GB/sec
150 nm
500 MHz
8.0 GFlops
32.0 GB/sec
150 nm
552 MHz
8.8 GFlops
35.3 GB/sec
90 nm
1.0 GHz
16.0 GFlops
64.0 GB/sec
65 nm
3.2 GHz
102.4 GFlops
256.0 GB/sec
28 nm
1.0 GHz
256.0 GFlops
256.0 GB/sec
SX-2
1983
SX-3
1989
SX-4
1994
SX-5
1998
SX-6
2001
SX-7
2002
SX-8
2004
SX-9
2007
SX-ACE®
2013

4
History of SX Vector Supercomputer
Vector Engine

6
Features of SX-Aurora TSUBASA
Downsizing of supercomputer realized by NEC’s Technology.
（Traditional Supercomputer）
High Memory Bandwidth
Ease of Use
Flexibility
1.53TB/s, 48GB
6x HBM2 implementation on Vector Engine
processor package
Fortran/C/C++ programing, OpenMP
Automatic vectorization/parallelization
3 kinds of execution modes
Product lineup from desk-side to large-scale
Data Centers
Tower
Type
Rack
Type
Downsize!
Vector Engine

7
Vector Engine processor
x86
Proces
sor
storage
Application
LINUX
VE Ctrl. SW
VH
Vector Host
VE
Vector Engine
Cache 16MB
core core core core
core core core core
1.53TB/s
core
core
HBM (48GB)
 8 - 10 vector cores
Peak performance:
2.45 - 3.07Tflops (DP)
4.91 - 6.14TFlops (SP)
 High memory bandwidth
1.53TB/s
 Technology
6x HBM2
implementation
The world’s first implementation in 2018
High Memory Bandwidth

8
Architecture of SX-Aurora TSUBASA
 SX-Aurora TSUBASA = VH + VE
 Linux + standard language (Fortran/C/C++)
 Enjoy high performance with easy programming
Software
 Linux OS
 Fortran/C/C++  Standard language
 Automatic vectorization compiler
Interconnect
 InfiniBand for MPI
 VE-VE direct communication support
Hardware
 VH(Standard x86 server) + Vector Engine
Application
Vector
Engine(VE)
x86 server
(VH)
Linux OS
PCIe
SX-Aurora TSUBASA
Architecture
Easy
programming
(standard language)
Automatic
vectorization
compiler
Enjoy high
Performance!
Ease of Use

9
VH VE
Linux
VEOS
VE
AP
Application
OS
Hardware
Native Mode
VH VE
Linux
VEOS
VE
Accelerator Mode
x86
AP
VE
VH VE
Linux
VEOS
Scalar Acceleration
Mode
x86
VE
AP
VE
VEO VH call
Execution mode
Ease of Use
OS
offload

10
Lineup of SX-Aurora TSUBASA
Vector Engine supports wide range products from desk-side to large-scale Data
Centers. Selling Vector Engine card has started from November, 2020.
Data Center Model
(Water-Cooling)
Rackmount Model
Edge Model
Data Center Model
Large-scale computation in Data Centers
Rackmount Model
Simulation in manufacturing industry, etc.
Use of AI / big data
Edge Model
Simulation in mid-sized manufacturing industry,
etc. Desk-side installation at laboratories
1VE
4VE 8VE
8VE
Vector Engine
Application Set Model,
Embedded System,
Cloud service
Vector Engine
Vector Engine
Application acceleration
Embedded use
Cloud service engine
Flexibility

1. History of SX Vector Supercomputer
2. Features of SX-Aurora TSUBASA
3. World famous HPC Centers utilizing SX-Aurora TSUBASA
4. Start card business
5. SX-Aurora TSUBASA Software
6. Value of Vector Engine
7. Roadmap and the future
8. Partnering

12
Trusted and Chosen by World Famous HPC Centers
NIFS : Fusion Science
National Institute for Fusion Science
Tohoku university : Academic
: Weather / Climate
JAMSTEC : Earth Science Osaka university：Academic
Cybermedia Center
Weather
Climate
ⒸJAMSTEC

13
Performance of large-scale computer system
JAMSTEC Earth Simulator is ranked in TOP10 in the latest HPCG ranking.
High Byte/Flops and high single core performance => High execution efficiency
Rank
Cores
HPCG
[TFlop/s]
Rpeak
[TFlop/s]
Execution
efficiency
HPCG HPL System Vendor
1 1 Fugaku Fujitsu 7,630,848 16,004.50 537,212.00 2.98%
2 2 Summit IBM 2,414,592 2,925.75 200,794.88 1.46%
3 5 Perlmutter HPE 706,304 1,905.44 89,794.48 2.12%
4 3 Sierra
IBM / NVIDIA
/ Mellanox
1,572,480 1,795.67 125,712.00 1.43%
5 6 Selene Nvidia 555,520 1,622.51 79,215.00 2.05%
6 8
JUWELS Booster
Module
Atos 449,280 1,275.36 70,980.00 1.80%
7 11 Dammam-7 HPE 672,520 881.40 55,423.56 1.59%
8 9 HPC5 Dell EMC 669,760 860.32 51,720.76 1.66%
9 13 Wisteria/BDEC-01 Fujitsu 368,640 817.58 25,952.26 3.15%
10 39
Earth Simulator -SX-
Aurora TSUBASA
NEC 43,776 747.80 13,447.99 5.56%
11 25 TOKI-SORA Fujitsu 276,480 614.22 19,464.20 3.16%
12 16 Trinity Cray/HPE 979,072 546.12 41,461.15 1.32%
13 54 Plasma Simulator NEC 34,560 529.16 10,510.66 5.03%
14 14 Marconi-100 IBM 347,776 498.43 29,354.00 1.70%
15 15 Piz Daint Cray/HPE 387,872 496.98 27,154.30 1.83%
NIFS
National Institute for Fusion Science
ⒸJAMSTEC
JAMSTEC
https://www.top500.org/lists/hpcg/2021/06/
Aurora Forum in ISC 2021

15
Start card business
Last year NEC announced starting PCIe card selling through system integrators as
NEC partners
https://www.nec.com/en/press/202011/global_20201119_01.html
Partner
ＮＥＣ
Large
scale
Small and Medium-sized
Enterprises
New/Volume zone
Supercomputer
center
Large-sized
Enterprises

16
Vector Engine Operation Verified Server List
Model name (as of Oct 2021)
• Axon 3
• Axon 4
• G242-Z11
• G492-Z50
• PowerEdge R7525
• PowerEdge R750r
• SYS-1029GQ-TRT
• SYS-4029GP-TRT2
• SYS-7049GP-TRT
URL: https://www.nec.com/en/global/solutions/hpc/sx/VH-Selection.html
 Vector Engine cards have been sold through
partners with the listed servers.
 The server list is posted on NEC global website.

18
 You will enjoy vector computing power without additional
SW license fee in Q4 CY21.
NEC LLVM-IR Vectorizer updated as a secondary compiler
Provide automatic vectorization feature for VE after clang/flang*1 and other compiler
front-ends and create execution file from source file free of charge.
LLVM-IR Vectorizer.
 Includes vectorizer and code generator for VE.
 Inputs LLVM-IR from memory or an IR-file and outputs an
assembler source code for VE.
 Applies automatic vectorization to LLVM-IR.
 Has APIs to support compiler directive.
 Provides runtime library including vector mathematical
functions (sin, cos etc.)
 Utilizes SW Development Environment (Community ver.)
composed of Binutils, Tuning Tool, Numeric Library Collection
and MPI development kit.
 Will support flang and MLIR.
Source file
･ Assembler source file including vectorized loops
*1 The release date is not fixed.
https://www.hpc.nec/forums/topic?id=pA1cPw
Front-end
(clang / flang / etc.)
LLVM
(Modules to create LLVM-IR)
LLVM-IR Vectorizer
VE Vectorizer
VE code generator
Your compiler
LLVM-IR Free of
charge
OSS
･ Execution file
SW Development
Environment
(Community ver.)
New!

19
 NEC will continue enhancing MPI functions
for Heterogeneous Computing!
History of NEC MPI functions
P
PCIe
switch
VE accelerated nodes
P
P
CPU(x86) nodes
GPU nodes
IB
P
PCIe
switch
P
P
PCIe
switch
P
P
CPU(x86) nodes
IB
P
P
PCIe
switch
P
2018
2019
2021
P
P
P : MPI process

20
Keep on enhancing NEC MPI functions
Improve the MPI communication performance between VEs on the Accelerator
Mode in Q1 CY22.
VH VE
Linux
VEOS
VE
AP
App.
OS
HW
Native Mode
VH VE
Linux
VEOS
VE
Accelerator
Mode
x86
AP
VE
VEO
OS
offload
P
P
PCIe
switch
P
P
VEO
VEO
MPI
P
P
PCIe
P
P
VEO
VEO
MPI
Current
1Q CY22
P
P
PCIe
switch
MPI
New!
Direct communication
between VEs like GPU Direct
3 copy
1 copy

21
NQSV(Network Queuing System V): Switch-over to the urgent job
NQSV supports Job scheduling for urgent jobs. DWD, a large-scale and mission
critical system, have been operating their service using this function for over a year.
 Implementation for the memory swapping
 The VE memory needed for the urgent job is assigned by swapping
out part of the VE memory used for the normal job to VH.
 Swap-out destination can be chosen from VH memory or a file..
• The performance of swapping-out is better in case VH memory is selected.
Note: Some parts of VE memory used by MPI communication etc. cannot be swapped-out.
 Job scheduling for urgent jobs
 An urgent job can be run immediately
suspending normal jobs.
 The normal jobs are resumed once the urgent
job is finished.

22
scikit-learn
API
NumPy
API
TensorFlow
(*1)
PyTorc
h
(*1)
VE ver.
DL4J
Vector Engine
NEC
Fraud
detection
for
Bank
PII
redaction
Campaign
optimization
Product
recommendation
Synthetic
fraud
identification
Insurance
fraud
&
claim
prediction
BERT
Partner
Python (scikit-learn, NumPy) TensorFlow PyTorch DL4J
NEC AI framework/middleware on SX-Aurora TSUBASA are available now and will be
used widely in partner’s applications.
SW stack of AI on SX-Aurora TSUBASA
ETL related ML related DL related
Spark
API
Spark
SQL
VE ver. Spark
Spark
Spark
MLlib
DataFrame
(*1) Maintained
with community
Framework/Middleware
Application

23
Combining HPC and AI
SX-Aurora TSUBASA supports Apache Spark and improve its performance.
This promotes the effective and efficient combination of HPC and AI.
NEC SX-Aurora TSUBASA/Vector Engine
HPC AI
 Data creation/preparation
 Machine Learning/Deep Learning
 Scientific computing
 Simulation
 Quantum annealing
Apache Spark
NEC HPC event website in SC21
https://www.nec.com/en/global/solutions/hpc/event/supercomputing/index.html

24
Features
NumPy-like library
• Provides a subset of NumPy's API
Highly optimized library
• Uses optimized library for VE (BLAS and ASL).
• Provides various vectorized operations.
Open-source library
• Licensed BSD License(3-clause).
• Published on GitHub and PyPI.
Future Outlooks
January 2022
• Supporting "Just-In-Time" compilation functionality.
April 2022 (Preview Release)
• Supporting VE-GPU Heterogeneous computation functionality.
NLCPy: NumPy-like API Accelerated with Aurora
Just by replacing the module name,
Python scripts using NumPy can utilize VE computing power.
import nlcpy as np
a = np.random.rand(1000, 1000)
b = np.fft.fft(a)
c = a + b
d = np.linalg.solve(a, c)
x86
node
Vector
Engine
Script Library
Data transfer between VH and VE
Computation
on VE

26
Vector technology
Weather
Climate
Manufacturing
Electromagnetic
Fluid
Optimal
Statistical
Geophysical
Genetic
Energy
Telecom
Financial
Weather
service
Progress of
analysis/science
Structural
Simulation
(HPC)
AI/BigData
Analytics
Logistic
Retail
Value of Vector Engine
NEC’s Vector technology can create new Social Values
- as the key to accelerate HPC + AI/Big Data Analytics

27
Stencil codes can be seen in a variety of application areas.
 Scientific Simulations
• Fluid Dynamics
• Thermal Analysis
• Electromagnetic Field Analysis
• Climate / Weather
 Signal Processing
• Audio, Sonar
• Radar, Radio Telescopes
 Image / Volume Data Processing
• Data Compression
• Recognition
• Medical Diagnosis (Biopsy, CT, MRI, …)
 Machine Learning
• Deep Learning (Convolutional Neural Networks)
Application areas where Stencil Code appears
Suitable application areas for Vector Engine
Climate / Weather Radar
Fluid Dynamics Electromagnetic Field Analysis

28
 SCA Interface
 Boosts performance of a wide variety of stencil calculations.
 Example
 User can easily define arbitrary stencil shapes by specifying relative locations.
Stencil Code Accelerator Interface of NLCPy
i i+1 i+2
i-2 i-1
j
j-1
j-2
j+1
j+2
0.25
0.25
0.25 0.25
0.0
𝑏𝑖,𝑗 =
1
4
𝑎𝑖,𝑗−1 + 𝑎𝑖−1,𝑗 + 𝑎𝑖+1,𝑗 + 𝑎𝑖,𝑗+1
𝜕
𝜕𝑥2
𝐴 +
𝜕
𝜕𝑦2
𝐴 = 0
Laplace Equation
discretization
finite-difference
import nlcpy as vp
a = vp.random.rand(100).reshape(10,10)
b = vp.zeros_like(a)
din, dout = vp.sca.create_descriptor((a, b))
desc_in = 0.25 * ( din[0,-1] + din[-1,0] + din[1,0] + din[0,1] )
desc_out = dout[0,0]
kern = vp.sca.create_kernel(desc_in, desc_o=desc_out)
for i in range(maxitr):
a[...] = kern.execute()

29
 Stencil Shapes and Time Steps
 2-D: XY-axial, size 6.
 3-D: XYZ-axial, size 6.
 For 100 time steps.
 Target Libraries
 Numba:
• A100 PCI-E 40GB
• Xeon Gold 6126 x2 (Skylake 2.60GHz, 48 cores)
 pystencils:
• A100 PCI-E 40GB
• Xeon Gold 6126 x2 (Skylake 2.60GHz, 48 cores)
 CuPy on A100 PCI-E 40GB
 NLCPy on VE Type 20B (8 cores)
 Data Size
 2-D: (64, 64) (128, 128) (256, 256) (512, 512) (1024, 1024)
 3-D: (64, 64, 64) (128, 128, 128) (256, 256, 256) (512, 512, 512) (1024, 1024, 1024)
For each case, single precision was used.
Benchmark Conditions
6x6y6za
6x6ya

30
Benchmark Result
 NLCPy on Vector Engine shows the highest performance.
6x6ya 6x6y6za
2-D XY-axial stencils 3-D XYZ-axial stencils

32
Vector Engine 3.0
■Targeting the largest memory bandwidth
■Inheriting and improving VE/VH architecture
■Higher Flops per processor
■Improved memory subsystem including cache
■Accelerating short vector, and scalar operations
■Adding instructions for AI/ML
■Maintaining high power efficiency
■Heterogeneous computing enhancement
■LLVM-IR Vectorizer(C/C++, Fortran) for VE30
■Virtual machine support
VE10
VE10E
VE20
VE30 2+TB/s memory bandwidth
8C/2.45TF
1.22TB/s
8C/2.45TF
1.35TB/s
10C/3.07TF
1.53TB/s

33
2019 2020 2021 2022
Memory
bandwidth
/
VE
VE10
VE30
8C/2.45TF
1.22TB/s memory bandwidth
8C/2.45TF
10C/3.07TF
2+TB/s memory bandwidth
2023 2024
VE40
VE50
2++TB/s memory bandwidth
VE10E
VE20
VE20
Roadmap

34
Background of multi-architecture system -towards Heterogeneous Computing-
Architecture is selected according to characteristics of each application. One of
trends in HPC system is heterogeneous, composed of a variety types of processors.
Scientific
calculation
Real-time
transaction
General Special
• Recommendation
• Demand prediction
• Fraud detection
• Weather forecast
• Aerodynamic analysis
• Collision analysis
• Self-driving
• Checking goods
• Cancer diagnosis
• Financial transaction
• Face recognition
• Industrial robot
• Financial portfolio
• Shift schedule
• Delivery planning
Combinatorial
optimization
Statistical
processing
Image
recognition
AISC/
FPGA
Vector
Engine
GPGPU
x86
Quantum
annealing

35
 4th generation Earth Simulator system
MPI communication on multi-architectural supercomputer
Higher performance by allocating appropriate resources with MPI communication
between CPU, GPU and Vector Engine nodes.
P
P
GPU
GPU
PCIe
switch
PCIe
switch
File System
Vector
Engine-
accelerated
nodes
GPU-
accelerated
nodes
5,472x VE
684node
InfiniBand HDR 200G, DragonFly+
64x A100 GPU
8node
61PB Luster
1.3 PB all-flash
Luster
CPU nodes
2x AMD EPYC 7742
1,440x CPU
720node
P
GPU accelerated nodes
Vector Engine accelerated nodes
P
https://www.conferenceharvester.com/uploads/harvester/VirtualBooths/13396/NKBNOCXO-PDF-1-412693%285%29.pdf
P: Process
2x AMD EPYC 7742
AMD EPYC 7742
A test benchmark execution was successful on JAMSTEC Earth Simulator!

36
Launch of a cloud service that uses a vector computer to enable use of
high-performance quantum-inspired simulated annealing machines
*1 Using SX-Aurora TSUBASA already commercialized as super computer
*2 Based on a survey conducted by NEC. Comparing to the case where the
100-city traveling salesman problem is solved by executing existing algorithm
(simulated annealing) on Xeon processor.
Annealing machine
Hardware
Middleware/OS
Annealing software
Co-Creating Service Image
Technology
verification
Demo plan Validation
Vector Computer
Press Release https://www.hpcwire.com/off-the-wire/nec-launches-simulated-annealing-service-utilizing-vector-supercomputers/
Evaluation
and operation
Annealing Machine Features
Ultra-high-speed processing using NEC’s proprietary
algorithm suitable for quantum annealing processing
and a vector computer. *1
More than 300 times faster*2
compared with existing simulated annealing systems
Equivalent to 100,000 qubits
of processing capacity for solving large-scale combinatorial
optimization problems
Facilitates the development of applications that integrate
with big data and artificial intelligence software
(from November 2021 in Japan)

37
Speeding up solving combinatorial optimization
with simulated annealing machine on SX-Aurora TSUBASA
 By utilizing, large/fast memory and high computation
performance
realize full connection 100Kbits
 Can support further large scale problem with scale-up
technology of supercomputer
How / Competitiveness, Uniqueness
NEC
Existing
good
solution
Extract issues
Formalize
Search
Check constraints
 Speeding up by limiting search space based on
the constraints of the problem
 Tuned perfectly for SX-Aurora TSUBASA
Formalize
Extract issues
Search considering
constraint
Constraints
Vector Engine
(Card)
SX-Aurora TSUBASA
Violation
→try again
Point 1
HW
Point 2
SW
Efficient search
considering the constraint
Wasteful search including
constraint violation
8 cores
2.45TF
1.53TB/s
6x HBM2
3D stacking
48GB

39
Partnership with Graphcore
NEC expects collaboration of Vector Engine and Graphcore IPU to achieve higher
performance and better TCO in AI data processing.
Data preparation
Obtain knowledge
and signals
Data cleanup
ETL (Extract/Transform/Load)
Training and inference
(AI algorithm execution)
Time for those phases can be shortened by choosing
appropriate platforms and calculation architectures.
Graphcore IPU for training
and inference
An example of NEC VE and Graphcore IPU collaborative usage in data processing flow
NEC VE for ETL

40
 NEC aims to expand business area into genome analysis and drug discovery market with Infinidat.
 Infinidat’s unique cache technology provides high-capacity and high-performance storage solution
to genome analysis and drug discovery, etc.
NEC and Infinidat partnership
Intelligent IO flow of Infinidat Storage
Read cache attributes
• Aggressive read-ahead
• Primary and secondary layers
• Uncompressed for superior latency
Write cache attributes
• 100% writes to DRAM
• De-staging every 5 min or less.
• Double protected across nodes.
Data persistence layer attributes
• Intelligent data placement
• Always sequential access
• High performance dual parity data protection
method
Host(s)
All
Writes
SSD
HDD Sequential
Reads
Random
Reads
Sequential
Writes
DRAM
Not compressed
9%
1%
Not compressed
Compressed
Cache
Extension
Sequential
Reads
3 Nodes of
Cache
DRAM + SSD
HOT
COLD
90% 2%
4%
94%
Machine Learning
Access Frequency Data Capacity
Other features:
• Next generation Snapshot
• High availability
• Triple redundant controller
• Available for mixed application
workloads

41
Find more information on our website
Aurora Web Forum
http://www.hpc.nec
SX-Aurora TSUBASA Website
http://www.nec.com/en/global/solutions/
hpc/sx/index.html
NEC HPC event website in SC21
 Latest updates
 Manual, documents
 Bulletin board
 Hardware and software overview
 Supported applications
info@hpc.jp.nec.com
 On-demand Webinar, Nov 15~
 Technical articles
https://www.nec.com/en/global/solutions
/hpc/event/supercomputing/index.html

PCCC21：日本電気株式会社「一台何役？SX-Aurora TSUBASA最新情報」

PCCC21：日本電気株式会社「一台何役？SX-Aurora TSUBASA最新情報」

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à PCCC21：日本電気株式会社「一台何役？SX-Aurora TSUBASA最新情報」

Similaire à PCCC21：日本電気株式会社「一台何役？SX-Aurora TSUBASA最新情報」 (20)

Plus de PC Cluster Consortium

Plus de PC Cluster Consortium (20)

Dernier

Dernier (20)

PCCC21：日本電気株式会社「一台何役？SX-Aurora TSUBASA最新情報」