The goal of the carrier today is to offer manageable end-user data services with a measurable QoS (Quality-of-Service) at the minimum cost per bit, using the smallest footprint systems, with the simplest implementation that allows for service-level agreements, operational efficiency, and traffic scalability. 

This has lead to the emergence of two design principles: the lower the layer at which...

1. Illuminating Optical Ethernet Networks!
Vishal Sharma, Ph.D., Principal Technologist & Consultant, Metanoia, Inc., Mountain View, CA
94041, USA. vsharma@metanoia-inc.com
Shahram Davari, MASc, Associate Technical Director, Network Switching, Broadcom
Corporation, 3151 Zanker Road, San Jose, CA 95134, USA. davari@broadcom.com
Table of Contents
1. Introduction: Evolution of Transport Networks and a Mélange of Terms............................... 2
2. Versatile Packet Networking with Ethernet: Service, Transport Technology, and PHY ......... 2
3. “Optical Ethernet Network” Defined ...................................................................................... 3
4. “Carrier Ethernet” and Its Relation to Optical Ethernet ........................................................ 4
5. Packet-Optical Transport (The New P-OTS!): What It Is and Where It Fits........................... 6
6. Optical Ethernet Applications/Services in Use Today ............................................................. 7
7. Service Provider Offerings Based on Optical Ethernet ........................................................... 8
8. How It All Fits: A Recap and a Look Towards the Future....................................................... 8
A Metanoia, Inc. Technology Paper Page 1 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

2. 1. Introduction: Evolution of Transport Networks and a Mélange of Terms
The TDM-to-packet network transformation has been underway in transport/telecommunications
networks for some years now, fueled primarily by two trends: (a) the advent of triple-play (voice,
video, data) for enterprise and residential customers, and, lately, the explosion in video and
mobile data services, and (b) the evolution in both packet- and transport-network equipment.
The goal of the carrier, of course, is to offer manageable end-user data services with a measurable
QoS (Quality-of-Service) at the minimum cost per bit, using the smallest footprint systems, with
the simplest implementation that allows for service-level agreements, operational efficiency, and
traffic scalability. This has lead to the emergence of two design principles: the lower the layer at
which packet data is carried, the lower the cost, and fewer layers (simpler systems) mean less
expensive systems, and, hence, lower cost. Thus, recent developments in both the Ethernet and
optical spheres have been geared towards this goal.
In this regard, there have been rapid advancements to make packet technologies, such as IP and
Ethernet, more “circuit-like”, and to make transport technologies and equipment more dynamic
and, thus, “packet friendly.” These developments have lead, over the last few years, to the
emergence of a mélange of terms: “optical Ethernet”, “metro optical Ethernet”, “packet-optical
transport”, “Carrier Ethernet”, “metro Ethernet”, which are often used interchangeably, blurring
the distinction between them, and leading to confusion in industry circles. Our objective in this
article is to define the terms: optical Ethernet, Carrier Ethernet, and packet-optical transport,
explain their relationships, and show how they all fit together in emerging optical Ethernet
networks.
2. Versatile Packet Networking with Ethernet: Service, Transport Technology, and
PHY
Before defining the term “optical Ethernet,” it is useful to point out that the term “Ethernet” itself
can apply to any one of the three roles of Ethernet technology: as a service, as a transport
technology, and as a PHY layer (cf. Figure 1).
A Metanoia, Inc. Technology Paper Page 2 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

3. Figure 1 Three key components of Optical Ethernet: service, transport, PHY,
together with the technologies and standards organizations involved in
specifying/developing each component
An Ethernet service is offered to the end-customer (the enterprise or residential customer), runs
end-to-end (customer premise-to-customer premise) and is one in which the traffic flow into/out
of the system at the customer consists of Ethernet frames. An Ethernet service is thus the Ethernet
connectivity between customer equipment.
A carrier-grade Ethernet service is one that is scalable (to many MAC addresses and end points),
offers QoS (traffic management), reliability (protection), and manageability (OAM and
monitoring), and can span long distances (of MAN/WAN scope; typically 10s to 1000s of
kilometers).
Ethernet transport refers to the ability to switch/route Ethernet frames (belonging to an Ethernet
service) between network nodes, by setting up/using connection-oriented, traffic engineered paths
in the network with deterministic performance (QoS, delay, jitter, loss, reliability). In other
words, Ethernet transport refers to the setting up of the “pipe” through which the Ethernet frames
travel, and to determining its routing within the cloud.
Ethernet transport makes it possible to realize connection-oriented Ethernet (COE). COE, in
essence, refers to the collection of control-plane protocols and data-plane settings that create a
connection-oriented capability for transferring the frames of an Ethernet service. We mention that
Ethernet transport could be provided either by enhancing Ethernet technology (e.g. as is done in
Provider Backbone Bridging with Traffic Engineering, PBB-TE, in the IEEE 802.1Qay standard)
or by a different technology (e.g. using MPLS-TP technology being developed jointly by the
IETF & the ITU-T). Both of these forms of transport involve switching/routing data frames, and
are, therefore, referred to as Layer 2 (or L2) transport.
It is also possible to embed Ethernet frames in a different transport networking layer, such as the
one provided by the ITU-T’s G.709 OTN (Optical Transport Network) standard. This form of
transport involves switching/routing traffic at the optical channel data unit (ODU) level and is
therefore referred to as Layer 1 (or L1) transport.
Ethernet PHY refers to the framing and timing of the actual bits of the Ethernet frame, and their
transmission over a physical medium – copper wire, coaxial cable, or optical fiber – to connect
switches at the physical layer. Some common Ethernet PHYs are the 1 GE (IEEE 802.3z), 10 GE
(IEEE 802.1ae), and 100 GE (IEEE 802.3ba) Ethernet PHYs. Note that Ethernet frames can also
be embedded in other PHY framing standards, such as those in the ITU-T’s G.709 OTN (Optical
Transport Network) standard.
3. “Optical Ethernet Network” Defined
With this background, we may now define an Optical Ethernet Network as a network spanning a
MAN/WAN that offers a carrier-grade Ethernet service, running over a connection-oriented
A Metanoia, Inc. Technology Paper Page 3 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

4. Ethernet (COE) transport infrastructure over an optical PHY (cf. Figure 2). The optical PHY
could be provided either by the OTN’s optical channel (OCh), or by an Ethernet PHY running
over optics, and may be multiplexed onto a given fiber using CWDM/DWDM technology.
Figure 2 Relationships of the different layers: service layer, transport layer, and PHY
layer, and their corresponding entities
A key characteristic of optical Ethernet is that its scope is beyond the enterprise LAN, and spans a
metropolitan-area or wide-area network.
4. “Carrier Ethernet” and Its Relation to Optical Ethernet
The term “Carrier Ethernet” was formalized by the work of the MEF (Metro Ethernet Forum) in
the 2004-2005 time frame, which defines Carrier Ethernet as “a ubiquitous carrier-grade Ethernet
service, which has the following five attributes: standardized services, scalability,
reliability/protection, hard QoS, and service management.” The technical work of the MEF (as
described in its specifications) together with the technical work of associated standards bodies
(ITU-T, IEEE, IETF) together enable the functionality and attributes of Carrier Ethernet.
• Standardized services refers to having a uniformly accepted definition of core services
that serve as the building block for applications running atop them (more on these below).
• Scalability refers to a service that scales to millions of UNIs (end-points) and MAC
addresses, spanning access, local, national, and global networks, with the ability to support
a wide bandwidth granularity and versatile QoS options.
• Reliability refers to the ability to detect and recover from errors/faults without impacting
customers, typically with rapid recovery times, as low as 50ms.
A Metanoia, Inc. Technology Paper Page 4 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

5. • Hard QoS implies providing end-to-end performance based on rates, frame loss, delay,
and delay variation, and the ability to deliver SLAs that guarantee performance that matches
the requirements of voice, video, and data traffic over heterogeneous converged networks.
• Service management implies having carrier-class OAM, and standards-based, vendor-
independent implementations to monitor, diagnose, and manage networks offering Carrier
Ethernet service.
The services defined by the MEF are in terms of an Ethernet Virtual Connection (EVC), which is
defined as an association of two or more User Network Interfaces (UNIs) at the edge of a metro
Ethernet network (MEN 1 ) cloud (i.e. subscriber sites), where the exchange of Ethernet service
frames is limited to the UNI’s in the EVC. The MEF defines 3 standardized services: E-Line (a
point-to-point EVC), E-LAN (a multipoint-to-multipoint EVC), and E-Tree (a point-to-multipoint
“rooted” EVC, where the root(s) can communicate with any of the leaves, but the leaves must
communicate with each other only via the root). Thus, an Ethernet Private Line service is built
using a point-to-point EVCs, while an Ethernet Private LAN service is built using mp2mp EVCs.
1
Even though the MEF specifications refer to MENs (metro Ethernet networks) this is now a generic term
that refers to the Carrier-Ethernet service enabled network, which can span a variety of access, metro, and
long-haul networks.
A Metanoia, Inc. Technology Paper Page 5 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

6. Figure 3 Optical Ethernet Network with the service, transport and PHY components
in operation
Thus, we see that Carrier Ethernet comprises the service component of optical Ethernet networks
(cf. Figure 1, Figure 2, and Figure 5).
5. Packet-Optical Transport (The New P-OTS!): What It Is and Where It Fits
Packet-optical transport systems (P-OTS or P-OTP) are a new class of networking platforms that
combine the functions and features of SONET/SDH/OTN ADMs or cross-connects, Ethernet
switching and aggregation systems, and WDM/ROADM transport systems into a single network
element, thus providing “data-aware optical networking.”
A P-OTS network element typically will have ITU-T G.709 OTN support, a COE component,
and support for WDM. These elements also offer transport of a wide range of client signals –
Ethernet (dominant), legacy SONET/SDH, SAN traffic, IP/ATM, video traffic, and can switch at
the wavelength level (WDM), sub-wavelength (or ODU) level, TDM level (SONET/SDH), and
A Metanoia, Inc. Technology Paper Page 6 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

7. packet level (Ethernet, MPLS). A P-OTS network element enables a carrier, especially in the
MAN/WAN, to quickly and cost-effectively change connectivity and bandwidth in the network,
without knowing about the actual services.
Key architectural features of P-OTS elements are:
• Universal switching architecture/fabric for switching traffic at different layers (OTN,
TDM and packet)
• Ability to switch, groom, and manage traffic in its native format (i.e. SONET/SDH traffic
as TDM traffic, and IP or Ethernet traffic as packet traffic), thus, allowing for the
percentage of each traffic type to vary dynamically (all Ethernet to all SONET/SDH and
anything in between, for instance)
• Software-selectable ports that can switch between switching SONET/SDH to switching
Ethernet, depending on the traffic
Even as this definition is gaining industry consensus, according to research firm Heavy Reading,
there are three architectures that are currently deemed to fall under the packet optical transport
umbrella, shown in Figure 4.
Figure 4 Packet-Optical Transport Systems (P-OTS): Architectures in use today
As a result, a number of vendor products fall in this umbrella e.g. Alcatel-Lucent 1850 TSS,
Ciena CN 4200 RS, Fujitsu Flashwave 9500, Meriton 7200 OSP, Tellabs 8800, 6300 & 7100
Nano, Cisco 15454 and 7600 (with appropriate blades), Nortel OME 6500, Juniper 9600, and so
on.
Thus, P-OTS platforms provide the transport and PHY components of optical Ethernet networks
(cf. Figure 5).
6. Optical Ethernet Applications/Services in Use Today
So which applications/services is optical Ethernet being used for (or envisaged for) today?
As expected, it is the business or residential services with triple-play applications
(voice, video, and data to the desktop), mobile backhaul applications (where the Ethernet PHY is
used between the base-station and the first switching node, and regular optical Ethernet networks
A Metanoia, Inc. Technology Paper Page 7 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

8. are used in the backhaul and backbone networks), and utility infrastructure networks (where oil,
gas, water, and electric utilities are transforming their aged communication systems into “data-
aware” systems that allow for automation of functions such as billing, monitoring, meter reading).
Applications such as software-as-a-service, VoIP, VoD, and hosted unified communications are
driving demand, as are ICT trends such as virtualization, data center outsourcing, data replication,
disaster recovery, remote backup, and IT outsourcing.
From a Chip/SoC perspective, several vendors offer solutions for Optical Ethernet transport and
PHY, such as Broadcom’s XGS (BCM56000 series), EZChip’s NP series, Xelerated’s X series
and T-PACK’s TPX series. These products can offer Ethernet and Connection Oriented Ethernet
(COE) switching, plus (in many cases) an integrated Ethernet PHY.
In addition, there are vendors that offer OTN transport and switching products: e.g. Broadcom
(BM8512), AMCC (Pemaquid), Cortina Systems (IXF30000 series) and T-PACK (P-OKET
series). These products typically map Ethernet to OTN, and some even provide OTN (ODU-
level) switching.
7. Service Provider Offerings Based on Optical Ethernet
Service providers worldwide (North America, Europe, EMEA) are rapidly moving to Ethernet
services. As of June 2009, Current Analysis research showed that metro (MAN) Ethernet circuits
constitute the bulk of (over 75%) optical Ethernet offerings, with p2p more common than
mp2mp.
The shift in deployment clearly favors fiber (FTTx: Fiber-to-the building, curb, or home), as
would be expected for optical Ethernet. A vast majority of US Ethernet is fiber-based, although in
Europe, providers are deploying EoCopper for the “last mile.” Finally, cable MSOs offer Ethernet
over HFC (hybrid fiber coax).
In the US, Verizon and AT&T are leaders, offering E-LAN, E-Line, and VPLS services, with
QoS, availability, regional/national/international coverage. In Europe, COLT is the leader
offering COLT LanLink and COLT Ethernet VPN service (E-LAN), while KPN offers a VPLS
service in 22 nations. In APAC, Sing-Tel and TataComms both offer E-Line and E-LAN services,
with SLAs, reachability, and broad coverage. In Middle East and Asia, Reliance Globalcom and
Reliance Communications are leaders, offering a VPLS-based optical Ethernet service.
8. How It All Fits: A Recap and a Look Towards the Future
Thus, we see that in the trio: service, transport and PHY, that are the components of optical
Ethernet: Carrier Ethernet provides the service component, packet-optical transport gear provides
the transport and PHY component, and the various IETF, IEEE, and ITU-T standards provide the
specifications for the PHY layers, as well as connection-oriented Ethernet (cf. Figure 5).
A Metanoia, Inc. Technology Paper Page 8 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

9. Figure 5 Optical Ethernet: How it all fits
As optical Ethernet evolves over the next few years, there will be further reduction in the layers
leading to a fused Ethernet-WDM packet transport layer with circuit-like capabilities, and to
packet-optical systems optimized for it. This allows the providers to handle increasing volume of
data traffic, while reducing the number of network elements by using Ethernet as the common
packet technology in access, aggregation and core networks.
***********************************
About Metanoia, Inc.
Metanoia, Inc. is a niche Bay-area consultancy that, since 2001, has been helping players across
the full telecom ecosystem (chip and semiconductor vendors, system vendors, operators and
carriers, technology houses, and software/planning tool vendors) solve complex problems in the
telecom space. Our contributions have spanned the strategy for, and the analysis, design, and
architecture of, systems, networks, and services, to the optimization of the equipment and
networks deploying them.
Our contributions have allowed a marquee list of client companies (ranging from fast-paced
innovative startups and international leaders, to giants and technology leaders in the US Fortune
1000) across 4 continents accelerate technology design and development or network design and
deployment, speed-up time-to-market, slash learning cycles, master complex technologies, and
enhance customer-interaction and revenues, yielding benefits many times their investments in
our services.
In short, we have been Powering Leadership Through InnovationTM! To learn more about how
we can help you, please contact us at experts@metanoia-inc.com or at +1-888-641-0082, and we
will be delighted to collaborate on efficiently solving your problem, and enhancing savings and
revenue for you.
About Broadcom, Inc.
A Metanoia, Inc. Technology Paper Page 9 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.

10. Broadcom Corporation is a major technology innovator and global leader in semiconductors for
wired and wireless communications. Broadcom(R) products enable the delivery of voice, video,
data and multimedia to and throughout the home, the office and the mobile environment. We
provide the industry's broadest portfolio of state-of-the-art system-on-a-chip and software
solutions to manufacturers of computing and networking equipment, digital entertainment and
broadband access products, and mobile devices. These solutions support our core mission:
Connecting everything(R).
Broadcom, one of the world's largest fabless communications semiconductor companies, with
2009 revenue of $4.49 billion, holds more than 3,800 U.S. and 1,550 foreign patents, and has
more than 7,800 additional pending patent applications, and one of the broadest intellectual
property portfolios addressing both wired and wireless transmission of voice, video, data and
multimedia.
A FORTUNE 500(R) company, Broadcom is headquartered in Irvine, Calif., and has offices and
research facilities in North America, Asia and Europe. Broadcom may be contacted at
+1.949.926.5000 or at www.broadcom.com.
A Metanoia, Inc. Technology Paper Page 10 of 10
Metanoia, Inc., 888 Villa Street, Suite 500, Mountain View, CA 94041, USA. http://www.metanoia-inc.com
© 2010 Metanoia, Inc.