2. Summit
Developers
Developers Summit 2013 Summer
Webサービスと1/10/40GbEの使い分け
D E V O P S M A K E S Y O U R L I F E H A P P Y !
SOURCE: SAKURA Internet Research Center. 12/2012 Project THORN.
Internet
1/10GbE
10/40GbE
Fat-Pipe
Backend
Network
STORAGE,
DATABASE
SSDSSD
SSDSSD
SSDSSD
SSDSSD
SERVER
SERVER
SERVER
SERVER
Web Front Backend
※すでにクラウド事業者では一般的なネットワーク構成
3. Summit
Developers
Developers Summit 2013 Summer
10 Gigabit Ethernet NICとは?
D E V O P S M A K E S Y O U R L I F E H A P P Y !
$299-から$1,110- までの標準価格帯
Mellanox, Myricom, Intel, Chelsio, Solarflare,
Broadcom など多様なベンダーからリリース済み
SOURCE: COLFAX International.
4. Summit
Developers
Developers Summit 2013 Summer
40 Gigabit Ethernet NICとは?
D E V O P S M A K E S Y O U R L I F E H A P P Y !
$725- の標準価格帯
Mellanoxからのみリリース済み
※現在唯一の実用的なカード
SOURCE: COLFAX International.
5. Summit
Developers
Developers Summit 2013 Summer
10 Gigabit Ethernet Cableは光とメタル
D E V O P S M A K E S Y O U R L I F E H A P P Y !
SFP+ Copper Cable
(短距離) SFP+ Optical Cable
(長距離)
SOURCE: COLFAX International.
6. Summit
Developers
Developers Summit 2013 Summer
40 Gigabit Ethernet Cableは光とメタル
D E V O P S M A K E S Y O U R L I F E H A P P Y !
QSFP+ Copper Cable
(短距離) QSFP+ Optical Cable
(長距離)
SOURCE: COLFAX International.
7. Summit
Developers
Developers Summit 2013 Summer
10GbE vs 40GbE Cable Connector比較
D E V O P S M A K E S Y O U R L I F E H A P P Y !
SFP+ Cable QSFP+ Cable
SOURCE: COLFAX International.
9. Summit
Developers
Developers Summit 2013 Summer
10 Gigabit Ethernet Switchの価格帯
D E V O P S M A K E S Y O U R L I F E H A P P Y !
Mellanox, Arista, Quanta, Fujitsu, Pica8
など多様なベンダーからリリース済み
$8,280-から$14,250-までの標準価格帯
SOURCE: COLFAX International.
10. Summit
Developers
Developers Summit 2013 Summer
40 Gigabit Ethernet Switchの価格帯
D E V O P S M A K E S Y O U R L I F E H A P P Y !
Mellanox,Gnodalなどからリリース済み
$14,670-からの標準価格帯
SOURCE: COLFAX International.
12. Summit
Developers
Developers Summit 2013 Summer
Intel x86サーバによる測定環境の作り方
D E V O P S M A K E S Y O U R L I F E H A P P Y !
CPU: Intel® Core™ i7-3930K CPU 3.20GHz
RAM: 32GB DDR3-DIMM
OS: Linux 2.6.32-220.23.1.el6.x86_64
NIC: Mellanox Connect X-3 (40GbE)
Benchmark Tool: pkt-gen
CPU: Intel® Core™ i7-3930K CPU 3.20GHz
RAM: 32GB DDR3-DIMM
OS: Linux 2.6.32-220.23.1.el6.x86_64
NIC: Mellanox Connect X-3 (40GbE)
10/40 Gigabit Ethernet
Back-to-Back Connection
SOURCE: SAKURA Internet Research Center. 12/2012 rev2 Project THORN.
13. Summit
Developers
Developers Summit 2013 Summer
Linux系ツールのバージョンアップが必須
D E V O P S M A K E S Y O U R L I F E H A P P Y !
SOURCE: SAKURA Internet Research Center. 12/2012 rev2 Project THORN.
# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10000baseT/Full
Supports auto-negotiation: No
Advertised link modes: 10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 40000Mb/s
Duplex: Full
:
Link detected: yes
帯域表示の不一致
15. Summit
Developers
Developers Summit 2013 Summer
10/40GbE測定結果(Packet/sec)
D E V O P S M A K E S Y O U R L I F E H A P P Y !
2.1Mpps
(10GbE)
4.5Mpps
(40GbE)
SOURCE: SAKURA Internet Research Center. 12/2012 rev2 Project THORN.
Million Packet/sec
Mpps
MTU Size (Byte)
16. Summit
Developers
Developers Summit 2013 Summer
ベンダーによる10GbE NIC性能差は?
D E V O P S M A K E S Y O U R L I F E H A P P Y !
Corei7-3930K CPU @ 3.20GHz / 32GB DDR3-DIMM / PCI Express 3.0 using linux standard pkt-gen with NETMAP.
SOURCE: SAKURA Internet Research Center. 05/2013 Project THORN.
2.62Mpps
2.81Mpps
2.78Mpps
1.39Gbit/s
1.41Gbit/s
1.39Gbit/s
MTU SIZE 64Byte Short Packet Processing Result(RX)
17. Summit
Developers
Developers Summit 2013 Summer
簡易的なベンチマーク手法の理解
D E V O P S M A K E S Y O U R L I F E H A P P Y !
SOURCE: SAKURA Internet Research Center. 12/2012 rev2 Project THORN.
1/10/40/56Gbit/s
Network Adapter
SERVER
RAMDISK (DRAM)
1/10/40/56Gbit/s
Network Adapter
SERVER
RAMDISK (DRAM)
Back-to-Back Interconnect
/ram /ram
[System; Intel® Core™ i7-3930K CPU @ 3.20GHz / 32GB DDR3-DIMM / Intel 82599-base 10GbE-NIC + Mellanox ConnectX3 40GbE-NIC]
18. Summit
Developers
Developers Summit 2013 Summer
アプリケーション性能が出ない時は…
D E V O P S M A K E S Y O U R L I F E H A P P Y !
[System; Intel® Core™ i7-3930K CPU @ 3.20GHz / 32GB DDR3-DIMM / Linux 3.7-rc7 / Mellanox ConnectX3 40GbE-NIC]
[Benchmark Tool: nc+dd+tmpfs*, rcopy+tmpfs**,]
5.28 Gbit/sec*
5.37 Gbit/sec*
5.17 Gbit/sec*
18.24 Gbit/sec**
ネットワークアプリケーションが
ボトルネックになっているケース
(BAD KnowHow)
SOURCE: SAKURA Internet Research Center. 12/2012 rev2 Project THORN.
19. Summit
Developers
Developers Summit 2013 Summer
可能性を秘めた10/40GbEの世界
4.5Mpps
(40GbE)
2.1Mpps
(10GbE)
14.2Mpps 10GbE Hardware Offload NIC (RX)
9.9Mpps 40Gbit/s Infiniband HCA(RDMA) RoCEならば…?
Million
Packet/sec
(Mpps)
MTU Size (Byte)