Presentation by Shishio Tsuchiya at APRICOT 2017 on Monday, 1 March 2017.

1. Copyright © Arista 2017. All rights reserved.Copyright © Arista 2017. All rights reserved.1
Beyond 100GE
-What will be happened Ethernet?-
Shishio Tsuchiya
shtsuchi@arista.com

2. Copyright © Arista 2017. All rights reserved.
Network Topology today
• East-West Traffic is increasing
• Server Performance more efficient
2
10GE/40GE Uplink
1GE/10GE
Server

3. Copyright © Arista 2017. All rights reserved.
Ethernet Server Shipment
-Source: Dell’Oro Group 9/2016
• Dell`Oro group focusing 25GE ports overs 40GE in 2017
• It will be reach x2.5 than 40GE by 2018
3
25Gbps
50Gbps
40Gbps

4. Copyright © Arista 2017. All rights reserved.
Why 25GE? Back to 100GE
• 100GE was first defined IEEE 802.3ba-2010
• And 802.3bg-2011, 802.3bj-2014 and 802.3bm-
2015 standards
• 802.3bm and 802.3bj were defined 4 lanes
specification for 100GE.
4
PHYASIC
4 Lane x 25Gbps

5. Copyright © Arista 2017. All rights reserved.
What is Lanes and Clock rate?
• High-speed components are connected by SerDes
• Data to be transmitted and Serialized , and the Deserialized on received.
• SerDes Technology’s latest speed is 25Ghz
• The Number of SerDes connection are called “lanes”
5
Interface Clock rate Lanes
1GE 1.25Ghz 1
10GE 10.31Ghz 1
25GE 25.78Ghz 1
40GE 10.31Ghz 4
100GE 25.78Ghz 4

6. Copyright © Arista 2017. All rights reserved.
PCIe 3.0 and Network interface
• PCIe 3.0 x 4 slots is widely shipping as server platforms
• PCIe 3.0 physical bandwidth 8.0Gbps/lanes
• 25Gbps could smoothly upgrade from 10Gbps server platform
and it’s bandwidth efficiency is better than 10GE.
6
Interface Lane requirement Bandwidth Utilization
10GE 2 62.5%
25GE 4 78.1%
40GE 8 62.5%
100GE 16 78.1%

7. Copyright © Arista 2017. All rights reserved.
Migration to 25Gbps
• 25GE uses SFP28 optics which can use same cabling infrastructure.
• OPEX of the migration is minimum
• Bandwidth will increase x 2.5
7
10G ToR to 10G Server
10/40G Switch
SFP+Switch
Silicon
1x10G
10G ServerSFP+
25G ToR to 25G Server
100G Switch
SFP28Switch
Silicon
1x25G
25G ServerSFP28
40G ToR to 10G Servers
10G ServerSFP+
10/40G Switch
QSFP+Switch
Silicon
4x10G
10G ServerSFP+
10G ServerSFP+
10G ServerSFP+
100G ToR to 25G Servers
25G ServerSFP28
100G Switch
QSFP28Switch
Silicon
4x25G
25G ServerSFP28
25G ServerSFP28
25G ServerSFP28

8. Copyright © Arista 2017. All rights reserved.
Merchant Silicon
8
How Many Lanes?
How much total
bandwidth?
Vender Broadcom Broadcom Broadcom Cavium
Name XGS
Trident +
XGS
Trident II
XGS
Tomahawk
XPA
XP80
Total Bandwidth 640Gbps 1.28Tbps 3.2Tbps 3.2Tbps
Number of SerDes 64 128 128 128
Support interfaces 40GE/10GE 40GE/10GE 10GE/25GE/
40GE/50GE/
100GE
10GE/25GE/
40GE/50GE/
100GE

9. Copyright © Arista 2017. All rights reserved.
25G Consortium and IEEE802.3by
• 25G Consortium
- http://25gethernet.org/
- draft was completed in September 2015 and use technology 802.3ba and 802.3bj
• IEEE802.3by
- http://www.ieee802.org/3/by/
- was formed in November 2015
- June 30, 2016, the IEEE 802.3by standard was approved.
• There is some difference
- Forwarding Error Detection(FEC)
≫ Cause 74(Fire Code) FEC
≫ Cause 91(Reed-Solomon) FEC
≫ Auto negotiation detects FC-FEC , RS-FEC and No-FEC
• Arista 7160 supports both technology IXIA Novus 100/25GE module will
support IxOS 8.21EA
9

10. Copyright © Arista 2017. All rights reserved.
Transceiver and cable
10
Standard Distance support
25GBASE-SR Up to 70m over Parallel OM3 Multi-mode fiber and 100m over parallelOM4 Multi-
mode fiber
25GBASE-LR Up to 10km over Single-mode fiber
25GBASE-AOC Up to 30m pre-terminated 25GbESFP Active optical cable
25GBASE-CR Up to 3m pre-terminated 25GbESFP twinax copper cable

11. Copyright © Arista 2017. All rights reserved.
Key takeaway
• 25GE is cost effective technology
- can use same NIC form factor( use one lane) with 10GE
- can use same cabling infrastructure
• it increases x 2.5 Speed than current 10GE
• ¼ resources compare with 40GE
• There are two implementation IEEE802.3by and
25G Consortium.
11

12. Copyright © Arista 2017. All rights reserved.
Network Topology today
• What is the best speed of Spine-Leaf uplink ?
12
10GE/40GE Uplink
25GE in 2017

13. Copyright © Arista 2017. All rights reserved.
Yes, 100GE!
13
Source:
http://www.fiber-optic-transceiver-
module.com/overview-of-100g-client-side-
transceivers.html
*Cisco Proprietary form factor
http://www.cisco.com/c/en/us/products/collateral/int
erfaces-modules/transceiver-
modules/data_sheet_c78-728110.pdf
Module CFP CFP2 CXP CFP4 CPAK* QSFP28
Line rate 40/100G 40/100G 12x12G or
10x12G
40/100G 40/100G 4x25-28G
Applications Ethernet
SONET/SDH
OTN
Ethernet
SONET/SDH
OTN
Infiniband
Ethernet
Ethernet
SONET/SDH
OTN
Ethernet
SONET/SDH
OTN
Infiniband
Ethernet
Dimensions 82x145x14 41x104x14 21x29 22x92x10 34.8x101.2x11.6 18x52x8.5
Max Power 8-32W 3-18W 6W 1.5-6W 4.5-7.5W* 3.5W
• QSFP28 most smallest/ low power standard
technology
• CFP2/CFP4 would be used in long reach
environment

14. Copyright © Arista 2017. All rights reserved.
Then what is next?
• Industry target is 400Gbps, it will use SerDes technology by 50Gbps x n
• It includes 1x400Gbps,2x200Gbps, 4x100Gbps and 8x50Gbps
• IEEE 802.3bs 200 Gb/s and 400 Gb/s Ethernet Task Force is targeting by End
of 2017 for IEEE standard
14
40
25
50
100
50
100
200
400
0
50
100
150
200
250
300
350
400
450
1 2 3 4 5 6 7 8
Speed[Gbps]
Number of Lanes
SerDes Speed growth and prediction
2013
2015
2018
x2.5 /2Years
x2 /3Years?

15. Copyright © Arista 2017. All rights reserved.
IEEE 802.3bs 200 Gb/s and 400 Gb/s Ethernet Task Force
15
Standard Speed Technology distance
200GBASE-DR4 200Gbps 4 lanes SMF 500m
200GBASE-FR4 200Gbps 4 WDM lanes 2km
200GBASE-LR4 200Gbps 4 lanes WDM 10km
400GBASE-SR16 400Gbps 16 lanes MMF 100m
400GBASE-DR4 400Gbps 4 lanes SMF 500m
400GBASE-FR8 400Gbps 8 WDM lanes 2km
400GBASE-LR8 400Gbps 8 WDM lanes 10km

16. Copyright © Arista 2017. All rights reserved.
Form Factor?
CFP8 QSFP-DD OSFP
Size(HxLxT) 40x102x9.5 18.35x89.4x8.5 22.58x107.8x13.0
Terminal Capacity 12-18W 7-10W 12-15W
Organization http://www.cfp-msa.org/ http://www.qsfp-dd.com/ http://osfpmsa.org/
16

17. Copyright © Arista 2017. All rights reserved.
What is important thing for next gen form factor?
• Don’t repeat 100GE mistakes
• Size , it should support at least 32 port per 1U
- CFP8 is too wide in this requirement
• Terminal capacity, it requires 12-15W to support
12.8Tbps
17

18. Copyright © Arista 2017. All rights reserved.
OSFP MSA FAQ
• Q: What is the OSFP (Octal Small Form Factor Pluggable)?
• A: The OSFP is a new pluggable form factor with eight high speed electrical
lanes that will initially support 400 Gbps (8x50G). It is slightly wider and
deeper than the QSFP but it still supports 32 OSFP ports per 1U front panel,
enabling 12.8 Tbps per 1U.
•
Q: Why a New Optics Module Form Factor?
• A: 400G Optics require more power than 100G Optics and many of them will
not fit into the existing QSFP form factor. Depending on the specific 400G
optics technology the initial power projections range from 7.5 Watt to 15
Watt.
• Q: How does the OSFP Address this Thermal Requirement?
• A: The OSFP has an integrated heat-sink that greatly improves thermal
performance and enables modules with up to 15W power in a switch
chassis with conventional airflow.
18
http://osfpmsa.org/

19. Copyright © Arista 2017. All rights reserved.
Key Takeaway
• IEEE802.3bs will be standardized 400GE in 2017
• It will use current SerDes Technology with
enhancement
• OSFP MSA is only one solution form factor to
support high density 400GE
19

20. Copyright © Arista 2017. All rights reserved.
Key Takeaway
• 2017
- 25GE would increase at server side
- IEEE will be finalized 400GE technology
• 2018
- 400GE product would be expected in end of this year
20

21. Copyright © Arista 2017. All rights reserved.Copyright © Arista 2016. All rights reserved.
www.arista.com
Thank You