Presentation of the 40th TOP500 List

1. Highlights of the 40th TOP500 List
SC12, Salt Lake City, Utah

2. Rmax Power
#
1
Site
Oak Ridge National
39th List: The TOP10
Manufacturer
Cray
Computer
Titan
Cray XK7, Opteron 16C 2.2GHz,
Country
USA
Cores [Pflops] [MW]
560,640 17.6 8.21
Laboratory
Gemini, NVIDIA K20x
Sequoia
Lawrence Livermore
2 IBM BlueGene/Q, USA 1,572,864 16.3 7.89
National Laboratory
Power BQC 16C 1.6GHz, Custom
K Computer
RIKEN Advanced Institute
3 Fujitsu SPARC64 VIIIfx 2.0GHz, Japan 795,024 10.5 12.66
for Computational Science
Tofu Interconnect
Mira
Argonne National
4 IBM BlueGene/Q, USA 786,432 8.16 3.95
Laboratory
Power BQC 16C 1.6GHz, Custom
JuQUEEN
Forschungszentrum
5 IBM BlueGene/Q, Germany 393,216 4.14 1.97
Juelich (FZJ)
Power BQC 16C 1.6GHz, Custom
SuperMUC
6 Leibniz Rechenzentrum IBM iDataPlex DX360M4, Germany 147,456 2.90 3.52
Xeon E5 8C 2.7GHz, Infiniband FDR
Stampede
Texas Advanced
7 Dell PowerEdge C8220, USA 204,900 2.66
Computing Center/UT
Xeon E5 8C 2.7GHz, Intel Xeon Phi
Tianhe-1A
National SuperComputer
8 NUDT NUDT TH MPP, China 186,368 2.57 4.04
Center in Tianjin
Xeon 6C, NVidia, FT-1000 8C
Fermi
9 CINECA IBM BlueGene/Q, Italy 163,840 1.73 0.82
Power BQC 16C 1.6GHz, Custom
DARPA Trial Subset
10 IBM IBM Power 775, USA 63,360 1.52 3.57
Power7 8C 3.84GHz, Custom

3. Rmax Power
#
1
Site
Oak Ridge National
39th List: The TOP10
Manufacturer
Cray
Computer
Titan
Cray XK7, Opteron 16C 2.2GHz,
Country
USA
Cores [Pflops] [MW]
560,640 17.6 8.21
Laboratory
Gemini, NVIDIA K20x
Sequoia
Lawrence Livermore
2 IBM BlueGene/Q, USA 1,572,864 16.3 7.89
National Laboratory
Power BQC 16C 1.6GHz, Custom
K Computer
RIKEN Advanced Institute
3 Fujitsu SPARC64 VIIIfx 2.0GHz, Japan 795,024 10.5 12.66
for Computational Science
Tofu Interconnect
Mira
Argonne National
4 IBM BlueGene/Q, USA 786,432 8.16 3.95
Laboratory
Power BQC 16C 1.6GHz, Custom
JuQUEEN
Forschungszentrum
5 IBM BlueGene/Q, Germany 393,216 4.14 1.97
Juelich (FZJ)
Power BQC 16C 1.6GHz, Custom
SuperMUC
6 Leibniz Rechenzentrum IBM iDataPlex DX360M4, Germany 147,456 2.90 3.52
Xeon E5 8C 2.7GHz, Infiniband FDR
Stampede
Texas Advanced
7 Dell PowerEdge C8220, USA 204,900 2.66
Computing Center/UT
Xeon E5 8C 2.7GHz, Intel Xeon Phi
Tianhe-1A
National SuperComputer
8 NUDT NUDT TH MPP, China 186,368 2.57 4.04
Center in Tianjin
Xeon 6C, NVidia, FT-1000 8C
Fermi
9 CINECA IBM BlueGene/Q, Italy 163,840 1.73 0.82
Power BQC 16C 1.6GHz, Custom
DARPA Trial Subset
10 IBM IBM Power 775, USA 63,360 1.52 3.57
Power7 8C 3.84GHz, Custom

4. 0
50
100
150
200
250
300
350
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
Replacement Rate
2008
2009
2010
2011
2012
164

5. Annual Performance Increase of the
TOP500
2.6
2.4
2.2
2
1.8
1.6
1.4
1.2
1
2006
2010
1994
1996
1998
2000
2002
2004
2008
2012

7. Performance Development
1 Eflop/s
1E+09 162 PFlop/s
100 Pflop/s
10000000
17.6 PFlop/s
10 Pflop/s
10000000
1 Pflop/s
1000000
SUM 76.5 TFlop/s
100 Tflop/s
100000
10 Tflop/s
10000
N=1
1 1000
Tflop/s 1.17 TFlop/s
100 Gflop/s
100
59.7 GFlop/s
10 Gflop/s N=500
10
1 Gflop/s
1
100 Mflop/s 400 MFlop/s
0.1
1994
1996
1998
2000
2002
2004
2006
2008
2010
2012

8. Projected Performance Development
1E+11
1E+10
11E+09
Eflop/s
10000000
100 Pflop/s
10 Pflop/s
10000000
SUM
1 Pflop/s
1000000
100000
100 Tflop/s
10000
10 Tflop/s
N=1
1000
1 Tflop/s
100 100
Gflop/s N=500
10
10 Gflop/s
1
1 Gflop/s
0.1
100 Mflop/s
2020
1996
2002
2008
2014

9. Student Cluster Competition
Team Vendor CPU # of Sockets Cores/ Inter- OS GPU # of
Partner Nodes / Node Socke connect GPU
t
U of Texas Dell Intel 2660 10 2 8 IB FDR Centos Nvidia 2
2090
Tsing Huan University Quanta Intel Xeon 7 2 8 IB FDR Centos Nvidia 7
E5260 2090
Purdue HP Intel 8 2 8 IB FDR RHEL
Sandybridge
MGHPCC Silicon AMD Abu 8 2 16 IB FDR Arch Nvidia 8
Mechanics Dhabi Linux K10 &
2090
National University of Inspur Intel E2650 6 2 8 IB FDR RHEL Nvidia 6
Defense Technology - 2090
China
Pacific Appro Intel E5-2070 8 2 8 IB QDR RHEL
University of Science Sugon Intel E5-2660 8 2 8 IB FDR RHEL Nvidia 6
& Technology - China 2090

10. Projected Performance Development
1E+11
1E+10
11E+09
Eflop/s
10000000
100 Pflop/s
10 Pflop/s
10000000
SUM
1 Pflop/s
1000000
100000
100 Tflop/s
10000
10 Tflop/s
N=1
1000
1 Tflop/s
100 100
Gflop/s N=500
10
10 Gflop/s
1
1 Gflop/s
0.1
100 Mflop/s
2020
1996
2002
2008
2014

11. Countries / System Share
others
0%
United States
others China
14% Japan
Canada
2% United States United Kingdom
Germany 50% France
4%
Germany
France
Canada
4%
others
United Kingdom
China others
5% Japan 15%
6%

12. Vendors / System Share
Fujitsu Others
IBM
7 46
Oracle 9% HP
1%
7 Cray Inc.
Dell 1% IBM
Bull 213 Appro
12
16 43%
3% SGI
3%
SGI Bull
16 Dell
3% Oracle
Appro
19 Cray Inc. HP Fujitsu
4% 137 Others
27
6% 27%

13. Vendors (TOP50) / System Share
IBM
Cray Inc
Fujitsu
Others
IBM Bull SA
15
20
30% Others
40%
Bull SA Cray Inc
3 8
6% Fujitsu
16%
4
8%

14. Cores per Socket
500
450
400 16
350 12
300 10
Systems
250 9
200 8
150 6
4
100
2
50
1
0
2003
2002
2004
2005
2006
2007
2008
2009
2010
2011
2012

15. Processors / Systems
1%1% Intel Nehalem
4%
5% Intel SandyBridge
5%
AMD x86_64
12% 45% Power
PowerPC
Intel Core
27% Sparc
Others

16. Accelerators/Co-Processors
70
60
50
Intel Xeon Phi
Systems
40 Clearspeed
IBM Cell
30
ATI Radeon
20 Nvidia Kepler
Nvidia Fermi
10
0
2006
2007
2008
2009
2010
2011
2012

17. Performance of Accelerators/CP
40
35 Clearspeed
Total Performance [Pflop/s]
30 ATI Radeon
25 IBM Cell
Intel Xeon Phi
20
Nvidia Fermi
15
Nvidia Kepler
10
5
0
2010
2011
2006
2007
2008
2009
2012

18. Linpack Efficiency
120%
100%
Linpack Efficiency
80%
60%
40%
20%
0%
0 100 200 300 400 500

19. Absolute Power Levels
13
12
11
10
Power [MWatts]
9
8
7
6
5
4
3
2
1
0
0 100 200 300 400 500
TOP500 Rank

20. Power Consumption
6
TOP10
5
4
Power [MW]
3
TOP50
2
1 TOP500
0
2008 2009 2010 2011 2012

21. Power Consumption
12
11 TOP10
10
9
Power [MW]
8
7
6
5 TOP50
4
3
2
1 TOP500
0

22. Power Efficiency
1600
Linpack/Power [Gflops/kW]
1400
1200
1000
TOP10
800
TOP50
600
400
200 TOP500
0
2008 2009 2010 2011 2012

23. Power Efficiency
Mic
3,000 BlueGene/Q
Max-Efficiency
Linpack/Power [Gflops/kW]
2,500
2,000
1,500 Cell
TOP10
1,000
TOP50
500
TOP500
0
2008 2009 2010 2011 2012

24. Most Power Efficient Architectures
Rmax/
Computer
Power
Appro GreenBlade, Xeon 8C 2.6GHz, Infiniband FDR, Intel Xeon Phi 2,450
Cray XK7, Opteron 16C 2.1GHz, Gemini, NVIDIA Kepler 2,243
BlueGene/Q, Power BQC 16C 1.60 GHz, Custom 2,102
iDataPlex DX360M4, Xeon 8C 2.6GHz, Infiniband QDR, Intel Xeon Phi 1,935
RSC Tornado, Xeon 8C 2.9GHz, Infiniband FDR, Intel Xeon Phi 1,687
SGI Rackable, Xeon 8C 2.6GHz, Infiniband FDR, Intel Xeon Phi 1,613
Chundoong Cluster, Xeon 8C 2GHz, Infiniband QDR, AMD Radeon HD 1,467
Bullx B505, Xeon 6C 2.53GHz, Infiniband QDR, NVIDIA 2090 1,266
Intel Cluster, Xeon 8C 2.6GHz, Infiniband FDR, Intel Xeon Phi 1,265
Xtreme-X , Xeon 8C 2.6GHz, Infiniband QDR, NVIDIA 2090 1,050
[Mflops/Watt]