OIF Workshop at NGON & DCI World 2022

Overcoming Implementation Challenges
to Enable Interoperability
NGON & DCI World 2022
Dave Brown – OIF Director of Communications, Nokia
Karl Gass – OIF Physical & Link Layer Working Group, Optical Vice Chair
Copyright © 2022 OIF 1

Agenda
• OIF overview
• Networking and Operations projects
• Physical and Link Layer projects
• Questions?

OIF - Where the optical networking industry’s
interoperability work gets done
What:
• Identify needs, gaps
• Develop interoperable
optical, electrical, and
control solutions
• Publish Implementation
Agreements
Who:
• 130+ member companies
• Network operators
• System vendors
• Component vendors
• Test & measurement
vendors
• Academia & research
Why:
• Accelerate adoption of
advanced technology to
connect a global, open
networked world
Challenge: Support innovation while preserving interoperability, optimizing performance and cost

Where OIF Fits
ITU-T
IEEE
Ethernet
Alliance
Infiniband
Trade Assn
Fibre
Channel T11
IETF
MEF
ONF
TIP
Implementation
Agreements
Interoperability
Testing
Networking Physical & Link Layer
COBO

OIF - Membership-driven Work
Market
Awareness
and
Education
Committee
Technical Committee
Networking &
Operations
WG
Interoperability
WG
Physical & Link
Layer WG
Physical Layer
User Group
Interoperability Demonstrations
Implementation Agreements Implementation Agreements
Network
Operator WG
Equipment
Developers
Equipment
Developers
Equipment
Developers
Electrical,
Optical, and
Protocol
Projects
Network and
Cloud
Operators
Network
Control and
Operation
Projects

Agenda
• OIF overview
• Questions?

Networking and Operations Working Group
Hot projects
Application of Artificial Intelligence to
Enhanced Network Operations
• Initiated in 2021, in process
• Identify a collection of use cases and
requirements for applying AI (Artificial
Intelligence) to packet and optical networks to
achieve enhanced network operations
• Work to be published as a whitepaper
APIs for Transport SDN
• 2014 - present
• Identify architecture and requirements
• Prototype and test SDN architecture and
interfaces
• Work jointly with SDOs on addressing gaps
(e.g. ONF TAPI)
• Interop testing and demonstration of standards
• Allows hands-on experience for participating
operators
• Helps drive vendor implementation
• Liaise findings to SDOs
Today’s focus

OIF - Accelerating Adoption of Transport SDN
Networking and Operations Working Group
• Identify architecture and requirements
• Prototype and test SDN architecture and interfaces
• Work jointly with SDOs on addressing gaps (e.g. ONF TAPI)
• Interop testing and demonstration of standards
• Allows hands-on experience for participating operators
• Helps drive vendor implementation
• Liaise findings to SDOs
• Repeat
OIF Transport API Interop Demos (2014, 2016, 2018)

OIF Transport SDN Interop Demo History
Helping the industry develop standard, robust, interoperable APIs for widescale adoption

OIF 2020 Transport SDN API Interop
The end game:
• Manageability and flexibility of the network are critical to allow network operators to successfully deliver a range of
cloud-based services, meet dynamic bandwidth demands, and to accelerate transport network transformation for the
5G era
• Established open Transport SDN APIs can help network operators:
• Improve network agility to adapt to dynamic service demands and traffic patterns
• Improve service provisioning and time-to-revenue
• Reduce maintenance and management with simplified control and automation
Scope:
• Focus on SDN-based programmability, control, and automation
• Build on OIF’s 2018 and 2016 interoperability demonstrations that helped establish ONF Transport-API (T-API) as the
de facto northbound interface (NBI) standard
• Test Layer 1 and Layer 0 OTN control using ONF T-API 2.1.3, with additional testing of OpenConfig device APIs for
transport equipment in network-operator-defined use cases

Global numbers:
(1) Global host operator: Telefónica
(3) Consulting operators: China Tel, Telia, TELUS
(5) System vendors: ADVA, Ciena, Cisco, Infinera, Nokia
(10) Weeks of testing: Sept – Nov, 2020
(21) Integrations between suppliers
(29) T-API use cases tested
(31) OpenConfig operations tested
High-level scope:
• Programmability, control, automation
• Layer 1 and Layer 0 OTN
• ONF T-API 2.1.3
• OpenConfig

Takeaways - ONF TAPI Testing
• Quick adoption of ONF TR-547 reference implementation agreement.
• RESTCONF compliant API supported on all vendors with few deviations found.
• TAPI v2.1.3 models are implemented by all vendors
• Notifications subscription mechanism based on tapi-notification model is widely supported.
• SSE notification format is widespread across implementations. Websockets as an alternative is supported as well.
• Maturity:
• Quite high percentage of use cases are supported across the vendor participants (64% use cases supported on
average).
• High level of mature products. Production ready products are interoperable to some extent and products
implementation support exhaustive testing.
• Major interoperability issues:
• IETF RESTCONF
• Standard authentication for RESTCONF APIs need to be enforced.
• API scalability for large data trees need to be addressed by RESTCONF filters or Streaming not through RESTCONF
deviations.
• RESTCONF provisioning behavior for POST operations into lists need to be clarified.
• UUID keys provisioning by RESTCONF client must be supported.
• Consistency of data after RESTCONF provisioning must be assured, modifications of client request data is not allowed.
• ONF TR-547
• Workflow simplifications may be needed for discovery use cases.
• Provisioning lifecycle states are still not uniform, room for improved definition in TR-547.

Takeaways - OpenConfig Testing
• Very good level of compliance OpenConfig models
• Uniform HW components hierarchy relation using parent and location attributes.
• Cross-connection logics discovery is widely implemented.
• Optical channel configuration (Frequency, power and operational mode) 100% supported.
• Performance indicators not fully supported yet. At least pre-fec-ber and/or q-value must be supported.
• NETCONF and gNMI
• NETCONF standard is widely supported across the industry.
• Subscription and notifications for performance streaming telemetry compliant with both, gNMI and NETCONF.
• Major interoperability issues:
• IETF NETCONF:
• Standard handshake for SSH is a must. Key exchange algorithms were different and its not fully standardized.
• Candidate datastore MUST be supported and it is not fully supported across the industry.
• OpenConfig:
• openconfig-transport-line-common version 0.6.0 is not widely supported a previous version is commonly
adopted (0.5.2)
• Further standardization needed for complex use cases:
• The commissioning and provisioning process for devices MUST be uniform. OTN Logical channel cross-
connections flexibility is hard to manage due to strong dependency on different hardware capabilities
• HW Dependencies does not allow some configurations. This must be clarified for understanding configuration
workflows.

More info and infographic:
https://www.oiforum.com/techni
cal-work/2020-oif-transport-sdn-
api-interoperability-demo/
Press release:
OIF Announces Participants in
2020 Joint Network Operator,
Multi-Vendor Transport SDN API
Interoperability Demonstration

Call for Interest
• OIF members are planning the next interop demo
• Evolve and refine T-API and OpenConfig for operator use cases and
operations
• 6-week testing timeframe proposed for 1Q23
• Define architecture/tech spec and test spec by 4Q22
• Open to OIF network operator and vendor member companies
Get on board! Commitment deadline is June 24th

Agenda
• OIF overview
• Questions?

Physical and Link Layer Working Group
Electrical
• Common Electrical I/O (CEI)
• 112
• 224
Optical
• Coherent components
• 400ZR
• 800ZR/800LR
Protocol
• Flex Ethernet
Management
• Common Management
Interface Specification
Co-Packaging
• Framework
• 3.2T Module
• External Laser Small Form
Factor Pluggable

PLL Electrical

What is CEI?
The Common Electrical I/O (CEI) Implementation Agreement (IA)
specifies the transmitter, receiver and interconnect channel associated
with 6G+ bps, 11G+ bps, 25G+ bps, 56G+ bps, 112G+ bps, and in the
future, 224G+ bps interfaces for application in high speed backplanes,
chip to chip interconnect and optical modules. Also included is the
Jitter definition and measurement methodologies associated with CEI
interfaces.

Common Electrical I/O (CEI) Implementation Agreements
The CEI Implementation Agreement is a clause-based format supporting publication
of new clauses over time:
• CEI-1.0: included CEI-6G-SR, CEI-6G-LR, and CEI-11G-SR.
• CEI-2.0: added CEI-11G-LR
• CEI-3.0: added CEI-25G-LR, CEI-28G-SR
• CEI-3.1: added CEI-28G-MR and CEI-28G-VSR
• CEI-4.0: added CEI-56G-USR-NRZ, CEI-56G-XSR-NRZ, CEI-56G-VSR-PAM4, CEI-56G-MR-PAM4, CEI-56G-LR-
PAM4, and CEI-56G-LR-ENRZ.
• CEI-5.0: added CEI-112G-MCM-CNRZ, CEI-112G-XSR-PAM4, CEI-112G-MR-PAM4, and CEI-112G-LR-PAM4.
Existing document available at www.oiforum.com > Documents>Implementation Agreements

CEI Has Been a Significant Industry Contributor
Name Rate per pair Year Activities that Adopted, Adapted or were influenced
by the OIF CEI
CEI-112G 112 Gbps 2022 Seven channel reach projects in progress, IEEE,
InfiniBand, T11 (Fibre Channel), Interlaken, ITU.
CEI-56G 56 Gbps 2017 IEEE, InfiniBand, T11 (Fibre Channel), Interlaken, ITU
CEI-28G 28 Gbps 2012 InfiniBand EDR, 32GFC, SATA 3.2, SAS-4,100GBASE-KR4,
CR4, CAUI4, Interlaken, ITU
CEI-11G 11 Gbps 2008 InfiniBand QDR, 10GBASE-KR, 10GFC, 16GFC, SAS-3,
RapidIO v3, Interlaken, ITU
CEI-6G 6 Gbps 2004 4GFC, 8GFC, InfiniBand DDR, SATA 3.0, SAS-2, RapidIO
v2, HyperTransport 3.1, Interlaken, ITU
SxI5 3.125 Gbps 2002-3 Interlaken, FC 2G, InfiniBand SDR, XAUI, 10GBASE-KX4,
10GBASE-CX4, SATA 2.0, SAS-1, RapidIO v1, ITU
SPI4, SFI4 1.6 Gbps 2001-2 SPI-4.2, HyperTransport 1.03
SPI3, SFI3 0.800 Gbps 2000 (from PL3)

CEI-112G Development Application Space
CEI-112G-MR Chip Chip
Chip-to-Chip & Midplane Applications
CEI-112G-MCM
3D Stack
CNRZ-5: up to 25mm package substrate
No equalization/FEC
Minimize power (pJ/bit)
2.5D Chip-to-Chiplet
CEI-112G-XSR Chip Optics
Chip to Co-Pkg Optics Engine
2.5D Chip-to-Chip
PAM4: up to 10 dB at 28 GHz
Lite FEC, Rx CTLE
50mm pkg substrate
CEI-112G-VSR Chip
Pluggable
Optics
Chip to Module
PAM4: 16 dB at 28 GHz (125mm + connector + 25mm)
FEC to relax BER to 1e-6
Multi-tap Tx FIR and Rx CTLE + multi-tap FFE or DFE
PAM4: 20dB at 28 GHz (500mm + 1 connector)
CEI-112G-LR Chip Chip
Backplane or Passive Copper Cable
PAM4: 28-30dB at 28 GHz (1m + 2 connectors)
CEI-112G-XSR+ Chip Optics
Chip to Near Pkg Optics Engine
PAM4: up to 13dB at 26.5 Ghz
Power target per SerDes: 1.8pJ/bit
Enables NPO implementations
2.5D Chip-to-Chip
CEI-112G-Linear Chip Pluggable
Optics
PAM4: up to 11 dB at 28 Ghz
Without DSP/SERDES in Optical Module
Lower power and cost targets
PAM4 modulation
scheme becomes
dominant in OIF
CEI-112 Gbps
interface IA
One SerDes core
might not be able to
cover multiple
applications from
XSR to LR
For short reach
applications,
simpler and lower
power equalizations
are desired

What is a Framework Document?
• An OIF Framework document is a publication of the results of a technical study and is the product
of the OIF membership
• The underlying project enables the OIF membership to explore next generation industry needs
and to forge a pre-competitive industry consensus, particularly with respect to interoperable
solutions by:
• Identifying the various customer needs (application spaces)
• Identifying points where interoperability is important
• Identifying what kinds of parameters should be interoperable
• Identifying additional projects which can focus on those points of interoperability, resulting in
Implementation Agreements
• The Framework document identifies potential gaps to the roadmap, aids the industry in
developing common directions, and specifies language for next generation solutions

CEI-224G Framework
• Summarizes the consensus findings and
guidance for new OIF CEI-224G projects
• Identifies key technical challenges for next
generation systems
• Power, density, performance, reach and cost
• Defines electrical interconnection applications
and discusses some of the interoperability test
challenges
Adopted and published in Feb 2022.
Available at www.oiforum.com>
Documents>White Papers > Technical White
Papers

CEI-224G: Modulation
• Common modulation with application-
specific performance tuning facilitates
• Multi-purpose designs
• Design re-use
• A simpler interface to direct-detect optics
• Backward compatibility with previous
generations (112G and 56G)
• Can this continue (and does it need to
continue) for 224G?
Interface Application 112G 224G
XSR/XSR+ Die to die/OE, CPO/NPO PAM-4 PAM-4
VSR Chip to optical module PAM-4 PAM-4
MR Chip to chip PAM-4 PAM-4
LR Backplane, mid-plane, copper cable PAM-4 ?
Optical Direct-detect optics PAM-4 PAM-4

CEI-224G: Performance
Connector model
courtesy of Amphenol
1.33dB/in IL at 56GHz
Bump-bump IL up to 26dB at 56GHz
PAM
BER target
1e-6
1e-5
1e-4
SNR margin relative to 1e-6
1dB additional margin for 1e-5
2.3dB additional margin for 1e-4
• Analysis suggests that 224G/lane chip-to-module (C2M) and C2C interfaces are feasible
• PAM4 outperforms other modulation schemes
• Opportunities to leverage existing error correction architecture(s) or consider stronger FEC for better margins
Source materials: https://www.ieee802.org/3/B400G/public/21_03/healey_b400g_01a_210329.pdf

CEI-224G: FEC
End-to-end FEC
Segmented FEC
Concatenated FEC
FEC schemes Example options Coding gain over
KP FEC
Overhead Latency Power/Area
End-to-end RS (576,514,31) ~1.5 dB more 6% more Incremental increase Incremental increase
Segmented KP and FECo FECo dominant FECo dominant Significant increase Significant increase
Concatenated KP+BCH/Hamming ~ 0.5-1.5 dB1 3% -6% more1 Incremental increase Incremental increase
To improve system margin stronger FEC could be considered for 224G
Alternative FEC schemes compared with baseline end-to-end RS (544,514,15)
Note 1: The coding gain of a concatenated code depends on inner code error correction capability and decoding algorithms. A coding gain larger than 1dB normally
assumes inner code of t>1 with low miscorrection ratio or soft-decision decoding with the cost of larger overhead, latency and complexity.

CEI-224G New Project Starts
• New Projects started at OIF Q1 2022 meeting
• One SerDes core might not be able to cover multiple applications from XSR to LR
• For short reach applications, simpler and lower power equalizations are desired
CEI-224G-MR Chip Chip
Chip-to-Chip & Midplane Applications
CEI-224G-XSR Chip Optics
Chip to Co-Pkg Optics Engine
2.5D Chip-to-Chip
Up to 50mm package substrate
1e-15 or lower (FEC is allowed)
CEI-224G-VSR Chip
Pluggable
Optics
Chip to Module
200mm of host, 20mm of module
1 connector
500mm of reach
1 connector
CEI-224G-LR Chip Chip
Backplane or Passive Copper Cable
1000mm of host and daughter cards
2 connectors

PLL Optical

Router Router
OTN
Switch
100GbE OTH, OTU4 100GbE
IEEE 802.3ba
ITU-T
OIF
Framer
Photonic TX
Photonic RX
100G ULH MSA
FEC
OTN
Switch
• 40G development highly fragmented
• Collaboration much improved on 100G
• Clear business case
• Stronger ecosystem
• Consistent standards and IAs
• OIF work on 100G DWDM transport united the industry around
• An overall framework including a modulation format
• Detailed IA's including photonics Tx/Rx modules
How OIF Accelerated 100G

Coherent Transport
• Optical components still dominate coherent system costs and this cost is very
volume sensitive.
• Complexity and system level flexibility at the coherent DSP engine level enables a
substantial cost savings for the industry since it enables the consolidation of the
industry EO component volumes into only three functions.
• Tunable laser (ITLA)
• Polarization Multiplexed Quadrature Modulator (PMQ-TX)
• Dual Polarization Intradyne Coherent Receivers (ICR)
• This is the strength of the coherent solution for transport.
• It has enabled coherent to move into markets not originally expected
32
Copyright © 2022 OIF

Coherent Transport Components +

Integrated Coherent Transmitter-Receiver Optical Sub-
Assembly (IC-TROSA) (2019)
• The IC-TROSA incorporates all of our coherent components into a single, highly-
integrated device for immediate insertion into next generation modules.
• Optical performance per HBPMQ-TX and DPC-MRX-2
34

What is 400ZR?
400ZR is an interoperable, cost-effective, 400Gb/s interface based on single-carrier
coherent DP-16QAM modulation, low power DSP supporting absolute (Non-
Differential) phase encoding/decoding, and a Concatenated FEC (C-FEC) with a
post-FEC error floor <1.0E-15.
400ZR operates as a 400GBASE-R PHY.

400ZR Point-to-Point Use Cases
400ZR
400ZR
Switch /
Router A
Switch /
Router B
DWDM Mux
Amplifier
DWDM Mux
Amplifier
400ZR 400ZR
Switch /
Router A
Switch /
Router B
Transponder Transponder
DWDM Mux
Amplifier
DWDM Mux
Amplifier
Switch /
Router A
Switch /
Router B
400ZR 400ZR
Switch /
Router A
Switch /
Router B
400ZR 400ZR
Amplified
Unamplified

400ZR Benefits
• 400ZR enables the DCI application to happen without requiring significant
additional equipment (size, space, power, expense).
• 400ZR interoperability supports an IPoDWDM infrastructure where
switches are optically linked to each other over an open line system(OLS)
using 400ZR modules.
• Significant cost, space, and power savings by skipping grey optics and transport
chassis.
• DCI network architectures built around OLS and 400ZR give data center
operators and carriers greater choice in the equipment/components they
use to build them.
• Low cost 400ZR allows data centers to be distributed across a region,
providing more agility and scalability than the mega data center.
37

Typical 400ZR Module (QSFP-DD)

Ethernet Alliance
Booth #5409
Cisco
Open Line
System
EXFO
Traffic Generator/Analyzer
NeoPhotonics
OSFP
Cisco
QSFP-DD
Viavi
Cisco
CFP2-DCO
Fujitsu
QSFP-DD
Juniper
Router
NeoPhotonics
QSFP-DD
Fujitsu
QSFP-DD
Juniper
QSFP-DD
Ciena
QSFP-DD
Ciena
Open Line
System
Nokia
Open Line
System
Nokia
Open Line
System
Ciena
Router
Marvell
QSFP-DD
Cisco
QSFP-DD
80 km
80 km
80 km
80 km
25 km unamplified
25 km unamplified
Juniper
Router
Juniper
QSFP-DD
Cisco
QSFP-DD
Cisco
Router
Juniper
QSFP-DD
NeoPhotonics
QSFP-DD
Ciena
QSFP-DD
Marvell
QSFP-DD
VLAN
Connection
THz
l unamplified 193.7
l1 193.7
l2 193.85
l3 193.925
l4 194.0125
l1
l2
l3
l4
l1
l2
l3
l4
Amp Unamp
CFP2-DCO Cisco 1
OSFP NeoPhotonics 1
QSFP-DD Ciena 1 1
Cisco 1 2
Fujitsu 1 1
Juniper 1 2
Marvell 1 1
NeoPhotonics 1 1
Module Allocation
OIF 400ZR @ OFC 2022

OFC 400ZR Point-to-Point Use Cases Demonstrated
400ZR
400ZR
Switch /
Router A
Switch /
Router B
DWDM Mux
Amplifier
DWDM Mux
Amplifier
400ZR 400ZR
Switch /
Router A
Switch /
Router B
DWDM Mux
Amplifier
DWDM Mux
Amplifier
Switch /
Router A
Switch /
Router B
400ZR 400ZR
Switch /
Router A
Switch /
Router B
400ZR 400ZR
Amplified
Unamplified
More or less

Cisco
Open Line
System
EXFO
NeoPhotonics
OSFP
Viavi
Cisco
CFP2-DCO
Ciena
Open Line
System
80 km
80 km
l1
l1
1
2 3
4
Path 1
OFC 400ZR: 400GE to Transport DWDM Link

Ethernet Alliance
Booth #5409
Cisco
Open Line
System
EXFO
Cisco
QSFP-DD
Viavi
Fujitsu
QSFP-DD
Juniper
Router
NeoPhotonics
QSFP-DD
Fujitsu
QSFP-DD
Juniper
QSFP-DD
Ciena
QSFP-DD
Ciena
Open Line
System
Nokia
Open Line
System
Nokia
Open Line
System
Ciena
Router
Marvell
QSFP-DD
Cisco
QSFP-DD
80 km
80 km
80 km
80 km
25 km unamplified
25 km unamplified
Juniper
Router
Juniper
QSFP-DD
Cisco
QSFP-DD
Cisco
Router
Juniper
QSFP-DD
NeoPhotonics
QSFP-DD
Ciena
QSFP-DD
Marvell
QSFP-DD
VLAN
Connection
THz
l unamplified 193.7
l1 193.7
l2 193.85
l3 193.925
l4 194.0125
l2
l3
l4
l2
l3
l4
1
2
3
4
5
6
8 9
10
11
12
13
14
15
16
7
Path 2
OFC 400ZR: Daisy Chain Using Unamplified and DWDM Links

800G Coherent
• 800ZR - single-span 80-120km amplified DWDM link for data center
interconnects
• 800LR - unamplified 2-10km links for campus applications
• Support clients (minimum 100GE) up to 800G aggregate bandwidth
• Includes muxing of low-rate services
• Fewer clients for 800LR
• Single-lambda 800G coherent line interface
• 800ZR and 800LR applications are separate objectives and will
probably be published on different timelines

800ZR Key Decisions & Status
800ZR to 800ZR-OFEC-16QAM Expansion Rates
800ZR PAD OFEC FAW, TS,RES PS Symbol Rate (Baud)
804 899 008 669
804 979 506 619
805 633 039 811
805 649 152 794
929 018 280 143
929 036 860 880
930 832 768 971
930 851 385 999
945 607 892 288
945 626 804 824
118 200 986 536
118 203 350 603
800GBASE-R
FEC RS (544,514)
Termination
850 085 000 000 803 205 312 500
850 000 000 000 803 125 000 000
849 915 000 000 803 044 687 500
Padding
992b 1193472b Scrambler
800ZR
1192480b
116 rows
(4640 x 257b)
(9316.25 x 128b)
Interleaver
Size = 172032b
4096
4096
3552
3552
ENC0
ENC1
84 × oFEC Coder blocks - 1376256b
oFEC Coder
800ZR-OFEC-16QAM
FAW,TS, RES
Symbols
(336 symbols)
172368
172368
Pilot insert with
spacing 63
(2736 symbols)
64/63
175104
175104
X pol
Y pol
172032
172032
H
V
344064 Polarization
Distribution
1376256b
Symbol
Map
16QAM
800ZR-OFEC
3552
3552
ENC2
ENC3
14208
4096
4096 Interleaver
Size = 172032b
MUX
FAW,TS, RES
Symbols
(336 symbols)
Pilot insert with
spacing 63
(2736 symbols)
• 16QAM and 150 GHz channel spacing
• OFEC - Prioritize performance over power dissipation
• GMP mapping 8 x 100G ZR clients to the 800ZR frame
• FLEXO-8e transport container
• First Straw Ballot!

800ZR TBRs
• ROSNR
• PMD tolerance
• Generally, optical specifications require more measurements
prior to concensus
• Initial Straw Ballot helps encourage contributions

800LR Status
• Reduce scope to only support 400GE and 800GE clients
• Significant discussion on FEC
• Ensure competitive power/latency compared to DD solutions by
reducing “overhead” of coherent

PLL Protocol

Motivations
• Long-haul optical modem evolution
• Speeds don’t have to correspond to standard Ethernet rates
• Availability of Ethernet and long-haul rates can be at different times
• Modems can trade off bit-rate and reach/loss
• Desire to not waste router-to-transport bandwidth
• Initial 400GbE work
• One proposal was to bond 4x100GbE together to get 400GbE
• Multi-Lane Gearbox work in OIF
• Standard needed to change every time Ethernet PCS changed
• 5G network slicing
• Not an original goal, but has become an important driver
• Build something that is spiritually Ethernet

What is FlexE?
FlexE defines a convention to flexibly use network bandwidth by
supporting bonding, sub-rating and channelization of links.
• Bonding of Multiple Links – allows an operator to create a larger “client” link
out of multiple slower ”transport” links.
• Sub-rating of Links – allows an operator to only use a portion of a ”transport”
link.
• Channelization of Links – allows one ”transport” link to carry several lower-
speed or sub-rated “client” links from different sources.

FlexE Motivating Use Cases
• Decouple Ethernet (client) and Transport optics rates
• Router-to-Transport handoff with slower grey optics
• MAC rate to match Modem rate
• Reuse of transport gear to get fatter packet pipes
• Eliminate the 20+% Link Aggragation (LAG) inefficiency
• Channelization
• Inside the box (MLG)
• Outside the box
• Router-to-transport handoff
• Network slicing

FlexE Demonstrations
• Starting in 2018, FlexE demos @ OFC
• Five vendors with FlexE hardware
• 400Gb/s (4x100GbE) link between OIF
and EA
• Bonding, subrating, and channelization
features shown in booth
• Ring connecting five vendors carrying
channelized 10GbE traffic.
• OFC 2019 and OFC 2022 upgraded demos

PLL Management

Management Interface Specifications in the OIF
• Updates/additions to the CFP MSA MIS (MDIO) to support coherent
operations:
• 5x7” Coherent Long Haul Module (2011)
• 4x5” Coherent Long Haul Module (2015)
• CFP2-ACO (2018)
• CFP2-DCO (2018)
• New OIF project to consolidate Coherent Module Management Interface
Specifications (2016)
• C-CMIS (2020)
• New PLL Management track formed (7/2021)
• QSFP MSA transfers CMIS to OIF (1/2022)
• CMIS 5.2 published (4/2022)

Why was CMIS started?
• CMIS was originally conceived to address industry pain points in
module management :
• Management of multiple form factors
• Module initialization
• Breakout – managing multiple different services
• The industry has embraced CMIS leading to continued efforts to
evolve CMIS with the addition of support for:
• Coherent modules
• Multiplexing modules
• Fibre channel

Eliminating Complexity for Pluggable Modules
• Module speeds ranging from 100G to
800G and CMIS versions 3.0, 4.0, 5.0
and 5.2. Unites a wide range of
transceiver classes under one
management protocol
• Fully form factor agnostic: CMIS
implementation is consistent
and interchangeable between
OSFP, QSFP-DD and QSFP112
modules and more.
• CMIS gives access to the low-speed
I2C interface to control and program
the module.
• Supports module types ranging from:
- Active Cable Assemblies
- Optical Transceivers
- Coherent DWDM modules
• Provides communication
between all compliant optical
modules, switches, and server
Network Interface Cards
• Enables interoperability
between module and host and
is used to test and debug the
module

CMIS Evolution Timeline

CMIS Values
• Common: Standardized rule book for all MSA vendors -> all modules
seamlessly plug and play into your host
• Flexible: CMIS is defined to support variety of modules with different
speeds, form factors, link ratings, use cases, etc.
• Extendable: CMIS is futureproofed for tomorrow’s pluggable
innovations.

MIS Host-to-Module Electrical Interface
• CEI-28G-VSR-NRZ – Programmed input equalization
• Requires the optical transmitter to support electrical-input equalization known as Continuous
Time Linear Equalization (CTLE)
• Programming of values from 1 to 9 dB in 1 dB steps
• CEI-56G-VSR-PAM4 – Adaptive input equalization
• Requires the optical transmitter to support autonomous adaptive electrical-input equalization
• Requires a defined startup sequence to set the expectation for the time order of events between
the host and the module
• When is the module receiving a valid electrical input signal to adapt to?
• CEI-112G-VSR-PAM4 –Programmed output characteristics and adaptive input
equalization
• Requires the optical transmitter to support adaptive electrical-input equalization
• Requires the optical receiver to support two programmed output configurations known simply as
“short” and “long” corresponding to whether the host has short (lower loss) or long (higher loss)
channels/traces
• IEEE P802.3ck defined AUI-S C2M and AUI-L C2M for Ethernet

MIS Look Ahead to CEI-224G-VSR
• Concerned there will be need for continuous adjustment of output of
• Host to module (Optical Tx Path)
• Module to host (Optical Rx Path)
• Need scheme for refining in real time these outputs
• Today those examining means of continuous adjustment are doing so to improve
the bit error ratio (BER) after correction by forward error correction (FEC) beyond
the requirements of the standard
• Concern some implementations in the future may only achieve the target post-
FEC BER, which is defined as an absolute worst case not necessarily the BER
required for the application (e.g., 10^-13 versus 10^-15 or better) – Need means
to do better
• System companies need to pay attention with their customers as to the required
post-FEC BER for critical applications as margin for additional performance
improvement shrinks
63

CMIS Prevailing Revision
• Available at www.oiforum.com>Documents>Implementation Agreements
64
Stefan Langenbach David R. Stauffer, Ph.D.
Cisco Systems, Inc. Kandou Bus, S.A.
Phone: +49-911-5805-6332 Phone: +1-802-316-0808
Email: stefan.langenbach@cisco.com Email: david@kandou.com
PLL Working Group, Management Track, Vice Chairs
Ian Alderdice Gary Nicholl
Ciena Corp. Cisco Systems, Inc.
Phone: +1-613-670-2523 Phone: +1 613-254-3535
Email: ialderdi@ciena.com Email: gary.nicholl@cisco.com
Abstract
This Implementation Agreement (IA) defines the Common Management Interface
Specification (CMIS), which may be used by pluggable or on-board modules, such as
QSFP Double Density (QSFP-DD), OSFP, COBO, QSFP, as well as by existing or future
module developments with host to module management communication based on a
two-wire interface. This IA is targeted for systems manufacturers, system integrators,
and suppliers of CMIS compliant modules.
Notice
Implementation Agreement (IA)
Common Management Interface Specification (CMIS)
Revision 5.2
IA Identification #
OIF-CMIS-05.2
www.oiforum.com Page 1
OIF-CMIS-05.2
April 27, 2022
Implementation Agreement created and approved
OIF
www.oiforum.com

CMIS 5.3 Underway
• CMIS support for CPO and ELSFP
• CMIS Support for Host-Module Link Training
• Existing maintenance items
65

PLL Co-Packaging

XSR+
How is Co-Packaging Being Tackled by the OIF?
Framework
Framework umbrella documents Co-Packaging study
• Defines the application spaces and explores technological considerations
Initiated a ”3.2T OE” project for 51.2T Ethernet Switching Application
• Defines an optical engine module, 16 of which can be used to provide optical
interfaces for a 51.2Tb/s switch using XSR electrical interfaces and FR & DR optical
interfaces
Initiated an ”XSR+” electrical interface project to to CEI to support NPO
• Adds new clauses to support D2D and D2IOE at 13dB loss at ~26.5GHz
Initiated a ”ELSFP” (External Laser Small Form Factor) project
• Defines a front panel pluggable laser source for Co-Packaging
Elevated Co-Packaging device management to the Management Track work
items
3.2T
Module
ELSFP
Co-Packaging
CMIS

Co-Packaging Framework
• Identifies the application spaces and subsequent
challenges
• Aids the industry in developing common
directions and language for next generation
solutions
Framework document adopted and published in
Feb 2022.
Available at www.oiforum.com>
Documents>Implementation Agreements
OIF-Co-Packaging-FD-01.0
www.oiforum.com 4
1 Table of Contents
1
1 TABLE OF CONTENTS.............................................................................................. 4
2 LIST OF FIGURES ...................................................................................................... 6
3 LIST OF TABLES........................................................................................................ 7
4 DOCUMENT REVISION HISTORY ............................................................................ 8
5 INTRODUCTION ......................................................................................................... 9
6 APPLICATIONS OVERVIEW ................................................................................... 11
7 POTENTIAL INTERFACES FOR INTEROPERABILITY STANDARDS ................ 14
7.1 Introduction .......................................................................................................................................14
7.2 Electrical Interfaces............................................................................................................................14
7.2.1 Electrical Footprint .....................................................................................................................17
7.2.2 Socket Retention Mechanism.....................................................................................................18
7.3 Optical Interfaces...............................................................................................................................18
7.3.1 Light (Laser) Sources...................................................................................................................18
7.3.2 Pigtailed and/or Connectorized..................................................................................................21
7.3.3 Connector or Fiber Exit Location and Size..................................................................................21
7.3.4 Optical Budget ............................................................................................................................21
7.4 Thermal..............................................................................................................................................22
7.4.1 Cooling Systems for Co-Packaging..............................................................................................22
7.4.2 Reported Thermal Data: .............................................................................................................25
7.5 Power.................................................................................................................................................25
7.5.1 Supply Voltages, Currents...........................................................................................................25
7.6 Management Interface......................................................................................................................25
7.6.1 CMIS Over 2-Wire, SPI ................................................................................................................25
7.7 Environmental....................................................................................................................................26
7.8 Reliability, Redundancy and Repairability .........................................................................................26
7.8.1 Infant Mortality Targets and Over-Life Targets..........................................................................27
8 SUMMARY................................................................................................................. 28
9 REFERENCES .......................................................................................................... 28

Co-Packaging Architectures (1)
Co-Packaged
using socket for
engine
Co-Packaged
with soldered
engine

Co-Packaging Architectures (2)
Co-Packaged
using copper
cable assembly
Near-Packaged
using socketed
engine

Interfaces Studied for Interoperability
Optical (Electrical)
& RLS
Thermal, Mechanical &
Environmental
Electrical
Socket
Power
MIS
Application Example
• Switch Generation: 51.2Tb/s
• Lane Speed: 106 Gb/s
• Interface Architecture: XSR based AUI, 400G-FR4 PMD
• Motivation: System power reduction, ecosystem & operational readiness
Reliability and Repairability

3.2T Module Example System Attachment
• 16 x 3.2T Modules = 51.2T Switch Capacity
Co-Packaged Assembly Substrate (Interposer)
Channel components cross-section
3.2T Modules
LGA Socket
Optical
Copper Cable

3.2T Module Dimensions
• 32 x 112G XSR to Standard Optics:
• 8 x 400G DR4
• 8 x 400G FR4 (incl. 200G mode)
• Copper Cable Assembly compatible
• Power capability:
• 56W (Internal Laser option)
• 48W (External Laser option)
6.0
35.1
52.1
22.5

OFC 2022 Co-Packaging System Demo
Amphenol
AOI
Broadcom
Intel
Ragile
SENCO
Sumitomo
Electric
TE

CEI – An Essential Building Block for Co-packaging
• XSR/XSR+ allows significant power saving opportunity over VSR to be captured.
• A broad interoperable ecosystem is the key to success and can only be achieved through standardization.
75
CPO/NPO Channel Example Illustration
Pluggable Module Channel Example Illustration (VSR)
• Channel loss: 16dB ball to ball (22-24dB bump to bump)
• Typical pluggable connectors: IL of ~1dB with RL of -
10dB @26.5GHz
• Channel loss: CPO – 10dB bump to bump; NPO –
13dB bump to bump
• Optional separable interconnect performance
example: LGA socket: IL of ~0.05dB with RL of -40dB
@26.5GHz (oif2020.341.01, Nathan Tracy)
• Avoids/reduces major discontinuities.
• Optical modules are not end user pluggable.

CEI-112G-XSR-PAM4 for Co-packaging
• Baud rates supported: 36 Gsyms/s to 58
Gsyms/s
• Based on loss and jitter budgets between TX
and RX using copper signal traces in a
SIP(System in a Package) to enable low power
consumption
• Three channel categories are defined, allowing
optimization for various applications.
• Published in CEI-5.0, May 2022.
Optional separable interconnect
Category IL at Nyquist (Max, dB) BER (Max)
CAT1 10 1e-6
CAT2 10 1e-8
CAT3 8 1e-9

CEI-112G-XSR+-PAM4 for Near Packaging
• The emergence of Near Package Optics (NPO)
Architecture
• Co-packaging requires significant package substrate size
increase and technology advancement, which adds risk to
goals of availability, cost and multi-vendor support.
• Instead of a monolithic package approach, Near Packaging
relies on advanced PCB technology for dense high-speed
routing without significant power penalty.
• Near Packaging architecture takes advantage of existing
technologies and more robustly enables an open ecosystem
implementation.
• Additional margin also strengthens a broader
supply base for co-packaging implementation and
adoption.
• Baud rates supported: 36 Gsyms/s to 58 Gsyms/s
• Optimize for Ethernet rate @ 106.25Gbps – the key
application for CPO/NPO
• Insertion loss < 13dB @ 26.5625GHz Nyquist bump to
bump with up to 1 separable interconnect.
• Enable the lowest practical energy consumption
(pJ/b) implementation.
• Leverage specification methodology and other
work from existing CEI 112 projects.
Co-Packaged Assembly Substrate
(PCB Interposer)
Package Substrate (Optional) Package Substrate (Optional)
ASIC Optical Module (OM)
Optional Separable Interconnect
Fiber

External Laser Small Form Factor Pluggable (ELSFP)
ELSFP
ELSFP
…
• ELSFPs provide CW laser power for optical engines (OEs).
• Decreases thermal power density in the system
• Each large system will likely need multiple (i.e. 8 or 16)
ELSFPs
• The light from a given ELSFP can feed more than a single
OE.
• A pluggable form factor helps to ensure total system
reliability and a “hot swap” replacement if a single laser or
ELSFP module fails.
• Eye safety is achieved by a blind mate optical connector
internal to the system.
ELSFP
ELSFP
Examples: Switches and
Special Purpose ASICs
SIDE VIEW
ELSFP PCB
Optical
Conn
Packaged
Laser
DC-DC mC
Optical
Engines
or Switch or
AI ASIC
Servers,
Linecards, etc.

ELSFP Blind-mate Connector, Demo @ OFC 2022
• First public demonstration of ELSFP:
• Active Modules (2 Suppliers)
• 2x (8) PMF MT Blind-mate Connector
• CWDM DFB lasers >100mW ex fiber
79
PD 1
PD 2
PD 3
3dB
DFB 1
DFB 2
Connector 1
Connector 2
Connector 4
Connector 3
ELSFP-Loopback 2
ELSFP-Loopback 1

Eye Safety
ELSFP’s blind mate optical
connector paired with a
system interlock enables
a safer co-packaged system
implementation for users.
Similar to EDFAs with powerful
CW lasers, Class 3B and 4 lasers can
be used inside ELSFP and systems can
be deployed in unrestricted locations.
80

CMIS to Control Remote Laser
81
Host for MIS
ELS
Optical
Engine
Host for MIS
ELS
Optical
Engine
Host for MIS
Integrated
Lasers
Optical
Engine
Host for MIS
Integrated
Lasers
Optical
Engine
CMIS I/F CMIS I/F CMIS I/F
MIS I/F
MIS I/F
? I/F
? I/F ? I/F
? I/F
The 3.2T OE for Ethernet switching will support both internal and external laser sources
Laser Internal
To
Optical Engine
Laser External
To
Optical Engine
Controller directly
controls Laser
Looks like
pluggable optics

Agenda
• OIF overview
• Questions?

Thank you!
OIForum.com

OIF Workshop at NGON & DCI World 2022

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à OIF Workshop at NGON & DCI World 2022

Similaire à OIF Workshop at NGON & DCI World 2022 (20)

Plus de Leah Wilkinson

Plus de Leah Wilkinson (9)

Dernier

Dernier (20)

OIF Workshop at NGON & DCI World 2022

Notes de l'éditeur