PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center

ETHERNET VPN
CZY TYLKO DLA DATA CENTER?
Emil Gągała
PLNOG, 30.09.2014

WHAT IS ETHERNET VPN
A new standards based protocol to
inter-connects L2 domains
Improves network efficiency
Multi-vendor industry wide
WAN
BGP based
state
exchange
LAG
initiative -- JNPR, CSCO, ALU, ATT,
Verizon, Bloomberg ….
Ideally suited for Datacenter
Interconnectivity but ...
... NOT only
EVPN
router
EVPN
router
LAN
2 Copyright © 2013 Juniper Networks, Inc.

ETHERNET VPN SUPPORT
Authors listed on draft-ietf-l2vpn-evpn-08 include:
A. Sajassi – Cisco Systems
J. Drake – Juniper Networks
W. Henderickx – Alcatel-Lucent
R. Aggarwal– Arktan
N. Bitar – Verizon
A. Isaac – Bloomberg
J. Uttaro – ATT

VPLS TO EVPN COMPARISON

VPLS EMULATES AN ETHERNET SWITCH
Common Characteristics:
Forwarding of Ethernet Frames
Forwarding of Unicast frames with an unknown MAC
address
Replication of broadcast and multicast frames
Loop prevention
Dynamic Learning of MAC address
Site 1 Site 2

VPLS CHARACTERISTICS
Virtual Private LAN Service (VPLS) provides VLAN
Extension over a shared IP/MPLS network.
Full Mesh
VLAN Separation
Any-to-Any connectivity regardless of physical path
Separate VPLS instances per VLAN. Allows network-wide segmentation
with very large scale
Provisioning
Multicast, Broadcast
and Flooding
Availability
New site Auto Discovery
Scale forwarding with Multicast Point-to-Multipoint capabilities
Underlying MPLS offers ECMP, Fast Reroute

EVPN REQUIREMENTS (ON TOP OF VPLS)
EVPN provides VLAN Extension over a shared IP/MPLS
network.
All-Active Multi-Homing
Better Control Over MAC
All available paths should be used (CE-PE, PE-PE)
MAC learning happens in control plane
Learning
ARP/ND Flooding
Minimization
L3 Egress Traffic Forwarding
Optimization
Reducing Unknown
Unicast Flooding
Additional attributes added during MAC advertisement
Usage of Default Gateway Extended Community
By using MAC learning in control plane

EVPN: VALUE PROPOSITION AND
USE CASES

WHY ETHERNET VPN
EVPN Use Cases:
Next generation L2VPN technology that replaces VPLS, VPWS
As DC Interconnect – allowing L2 stretch between two data
centers over WAN
EVPN as control plane with VxLAN IP overlay DC networks
Which customers are interested in EVPN and why ?:
Service providers that offer E-LAN / E-LINE services
Today, use a PE router for L2 services with VPLS, VPWS
EVPN technology improves their service offering
Data Center Builders – SPs, Enterprises, Content providers
Today, use a DC WAN Edge Router
EVPN allows multi-tenant L2 stretch between DCs and within DC

USE CASE #1: EVPN FOR NEXT GENERATION
ETHERNET SERVICES
BGP signaling on WAN exchange
MAC/IP routes
EVPN PE2
EVPN
PE1
EVPN
PE3
EVPN
CE PE4
CE
MPLS
9
MP-BGP
Benefits:
• Allows more efficient, feature rich E-LAN and E-LINE services
• Solves shortcomings of VPLS; offers IP VPN like policy control
• Supports explosive traffic growth
• Active/Active multi-homing with load balancing
• Improves network efficiency
• Minimizes flooding of BUM traffic/improves MAC learning

USE CASE #2: EVPN FOR DATA CENTER
INTERCONNECT
VLAN 1
MAC1
VLAN 1
MAC11
Data Plane Learning BGP Control Plane based learning on WAN Data Plane Learning
E-VPN
Cloud
Legacy L2
Cloud
VXLAN
Cloud
VLAN 2
MAC 2
VLAN 2
MAC22
Data Center Site1 Data Center Interconnect Data Center Site 2
Benefits:
• Seamless interconnect for DCI – L3 aware L2 stretch between DCs
• Seamless workload migration - VM mobility across DCs
• Wide Applicability – Interconnects Native L2 and overlay DC
technologies like VxLAN, MPLS in DC

USE CASE #3: EVPN-VXLAN FOR DC OVERLAY
EVPN acts as control plane protocol
VNID used in place of Eth tag ID for EVPN
signaling
VxLAN is data plane encapsulation
Benefits of EVPN-VxLAN DC Overlay:
• Allows Simple All IP fabric in DC
Network
Orchestrator
Overlay
environment
IP Fabric
Management
Plane API
needed only
• No need for multi-layers L2 setup in DCs
• Allows L2 connectivity for VMs / applications
• Deliver a L2VPN straight to a hypervisor
• L2 Multi-tenancy in all IP DC
• Each tenant can have 4094 VNIDs
• VXLAN Tunnel Endpoints (VTEP) exist on
networks equipment and hypervisors
• All benefits of EVPN applicable in a DC
TOR
VDS
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM

EVPN
BUILDING BLOCKS OPERATIONS

EVPN INSTANCE AND EVPN SERVICE INTERFACES
EVPN Instance (EVI) represents a VPN in the MPLS/IP network
One or multiple broadcast domains can be part of the same EVI
Each broadcast domain is uniquely identified inside EVI by Ethernet
Tag
PE
VLANs
(all or N)
PE
VLANs
PE
VLANs
BD EVI
VLAN Bundle SI/Port Based SI
• All CEs MUST use same
CE-VIDs
• Encap. MPLS frames MUST
remain tagged
• No Tag translation allowed
BD EVI
BD EVI
VLAN Based SI
• One-to-One mapping
• Different CE-VIDs can be
used on CEs
• Tag translation allowed
• Ethernet Tag is set to 0
BD
EVI
BD
VLAN Aware Bundle SI
• Many-to-One mapping
• Different CE-VIDs can be
used on CEs BUT
• Normalized tag MUST be used
• Ethernet tag == NormalizedTag

EVPN BASIC CONFIGURATION –
VLAN-BASED SERVICE INTERFACE
interfaces {
ge-1/0/1 {
flexible-vlan-tagging;
encapsulation extended-vlan-bridge;
unit 10 {
vlan-id 10;
family bridge;
}
}
ge-1/0/2 {
protocols {
bgp {
group iBGP-EVPN {
type internal;
local-address 11.99.0.13;
family evpn {
signaling;
}
neighbor 11.99.0.86;
}
}
}
unit 30 {
vlan-id 30;
family bridge;
}
}
}
routing-instances {
EVPN-1 {
instance-type evpn;
vlan-id 200;
interface ge-1/0/1.10;
interface ge-1/0/2.30;
route-distinguisher 11.99.0.13:200;
vrf-target target:65320:200;
protocols {
evpn;
}
}
}

EVPN INFORMATION EXCHANGE OVERVIEW
MPLS or IP
detours
LAG
LAG
Route
Reflector
VLAN 1
MAC1,
IP1.1
VLAN 2
MAC2, IP2.1
VLAN 1
MAC11, IP1.11
VLAN 2
MAC22, IP2.22
EVPN reachability
advertisement
Route Distinguisher
ESI
Ethernet Tag
MAC Address
IPv4 or IPv6 Address
Service Tag
• EVPN advertises MAC (L2) and IP (ARP) bindings for each
segment along with service tags
• Allowing Control Plane based L2 and ARP learning
• Minimizes flooding across WAN
• Allows proxy-ARP to respond queries locally
• IRB MAC address exchange allows same gateway MAC
address across sites
• VM mobility: egress traffic optimization

ETHERNET TAG IDENTIFIER
An Ethernet Tag ID is a 32-bit field
Contains a 12-bit or a 24-bit identifier to identify a broadcast
domain in an EVPN instance.
12-bit identifier is used for normalized VLAN ID for EVPN (MPLS)
24-bit identifier is used for VNID for EVPN-VxLAN
24- bit identified is used for I-SID for PBB-EVPN.
An EVI can have one or more broadcast domains – VLANs -
assigned to a given EVPN instance

ETHERNET SEGMENT IDENTIFIER (ESI)
If CE is multi-homed to two or more PEs, the set of Ethernet
links constitutes an “Ethernet Segment”.
A/P or A/A multi-homing is supported
An Ethernet Segment MUST have a non-reserved ESI that is
unique network wide. ESI can be auto-provisioned
CE
PE1
PE2
MPLS
ESI Auto-Provisioning with MC-LAG
System Prio System MAC Address Port Key
CE BPDU L2
CE PE1
PE2
BPDU
MPLS
ESI Auto-Provisioning with MC-LAG
Bridge Prio Root Bridge MAC 0x0000

EVPN ACTIVE/STANDBY MULTI-HOMING:
CONFIGURATION
interfaces {
ge-1/0/1 {
esi {
00:10:11:00:00:00:00:00:00:01;
single-active;
}
unit 10 {
vlan-id 10;
family bridge;
interfaces {
ge-10/0/3 {
vlan-tagging;
esi {
00:20:22:00:00:00:00:00:00:02;
single-active;
}
unit 20 {
vlan-id 20;
family bridge;
PE1: PE2:
}
}
ge-1/0/2 {
esi {
00:20:22:00:00:00:00:00:00:02;
single-active;
}
unit 30 {
vlan-id 30;
family bridge;
}
}
}
}
}

EVPN NLRI
BGP AFI 25 (L2VPN)/ SAFI 70 (EVPN)
Format = Route Type : Length : Route-type Specific
EVPN Route Types
1) Ethernet Auto-Discovery (A-D) Route
Used for fast convergence (withdrawal), and active/active multi-homing (split-horizon
label)
2 variants: per ESI and per EVI
2) MAC/IP Advertisement Route
Used for remote MAC address learning, known unicast traffic
3) Inclusive Multicast Route
Used for BUM (broadcast, unknown unicast, multicast) traffic
4) Ethernet Segment Route
Used for auto-discovery of multi-homed Ethernet segments and Designated
Forwarder election

ETHERNET AUTO-DISCOVERY PER ESI – TYPE 1
BGP signaling on WAN
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS
9
Loop Avoidance via split horizon
Fast Convergence
Ethernet AD route
per ESI announces
ESI mode
• Ethernet AD route per ESI signals All active or single active mode of
operation for a multi-homed CE
• Advertises Split Horizon label for L2 BUM traffic
• Enables forwarding state for the advertised ESI
• On withdrawal of AD route per ESI, all PEs adjust NHs or invalidate MAC
routes associated with that ESI, allowing rapid convergence
Auto Discovery
message per L2
Segment
RD
ESI
Ethernet Tag
Service Tag

ETHERNET AUTO-DISCOVERY PER EVI – TYPE 1
MP BGP signaling between PEs
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS
9
Allows Load-balancing
Rapid convergence
Ethernet AD route
per EVI includes all
connected ESIs for
that EVI
• Each Multi-homed PE advertises AD route per EVI for all connected ESIs
to advertise “service label” (aka “aliasing label”)
• Ethernet A-D per EVI route is used for ’Aliasing’ (load-balancing)
• Remote PEs use AD per EVI route and MAC route together to load-balance
traffic
• Load balancing for L2 as well as L3 traffic
• AD route per EVI and AD route per ESI BOTH are reqd for multi-homing
Auto Discovery
message per EVI
RD of EVI
ESI
Ethernet Tag
Service Tag

EVPN ROUTE TYPE 1 –
ETHERNET AUTO-DISCOVERY ROUTE
juniper@mx-re1 show route table EVPN-1.evpn.0 detail
1:11.99.0.86:0::202200000000000002::0/304 (1 entry, 1 announced) Format =
Type:RD::ESI::Label/304
*BGP Preference: 170/-101
Route Distinguisher: 11.99.0.86:0 RD set to PE IP address followed by zero
[…]
Source: 11.99.0.86
Protocol next hop: 11.99.0.86
[…]
Local AS: 65320 Peer AS: 65320
Age: 35:05 Metric2: 1
Validation State: unverified
Task: BGP_65320.11.99.0.86+179
Announcement bits (1): 0-EVPN-1-evpn
AS path: I
Communities: target:65320:200 esi-label:100000(label 0) ESI Label:
flag = 0: active/active, flag = 1: active/standby
Import Accepted
Localpref: 100
Router ID: 11.99.0.86
Primary Routing Table bgp.evpn.0

MAC ROUTE – TYPE 2
Establishes Reachability MP BGP signaling between EVPN PEs
Each PE learns MAC on
PE2
PE1 (DF)
PE3
PE4
CE
CE-PE link and advertises
its reachability in EVPN
MAC route
CE
MPLS
9
MAC reachability
advertisement
RD of EVI
RT
ESI
Ethernet Tag
MAC Address
IPv4 or IPv6 Address
Service Tag
• Advertises host MAC (and host IP) reachability with “service
label”
• Allows Control Plane based MAC learning for remote PEs
• On MX, service label is same as one advertised in AD per EVI route
• Minimizes flooding across WAN
• Allows PE to do proxy-ARP for remote hosts locally
• IRB MAC address route has default GW extended community
• Used in VM motion when default GW of VM remains same
• If IRB MACs and IP are same across MH PEs, avoids flooding after node
failure

INCLUSIVE MULTICAST ROUTE – TYPE 3
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS
9
MP-BGP
Sets up path for BUM traffic
Per VLAN per EVI
Allows PE to send BUM traffic from a CE on a VLAN in an EVI, to all the
other PEs that span that VLAN in that EVPN instance
• Uses Existing MVPN defined constructs for signalling and transport
• P2MP Tunnel : If advertising PE uses a P-Multicast tree for EVPN,
the PMSI Tunnel attribute MUST contain tree identity
• Ingres Replication : Route includes PMSI Tunnel attribute with
Tunnel Type set to Ingress Replication and Tunnel ID as PE
address.
• Able to carry the traffic of more than one EVPN instance on the same
tree using ’Aggregation’
Inclusive multicast
Ethernet TAG route
RD of EVI
Eth TAG
Advertising PE IP
Next Hop (PE IP)
Route Target
PMSI Tunnel Attr

EVPN ROUTES TYPE 2 TYPE 3
juniper@mx-re1 show route table EVPN-1.evpn.0
2:11.99.0.13:200::200::00:00:0a:0a:02:01/304
*[EVPN/170] 00:04:36
Indirect
2:11.99.0.13:200::200::00:00:0b:0a:00:12/304
*[EVPN/170] 00:04:36
Indirect
2:11.99.0.13:200::200::00:00:0b:0a:01:11/304
*[EVPN/170] 00:04:36
Indirect
3:11.99.0.13:200::200::11.99.0.13/304
*[EVPN/170] 00:53:47
Indirect
Local MAC
Advertisement Routes
(Format = Type:RD::Eth-Tag-iD::
MAC/304)
Local Inclusive Multicast
Ethernet Tag Route
2:11.99.0.86:200::200::00:00:0b:0a:00:0a/304
*[BGP/170] 00:26:58, localpref 100, from 11.99.0.86
AS path: I, validation-state: unverified
to 11.0.100.18 via xe-2/0/0.10
to 11.0.100.22 via xe-2/0/1.10
2:11.99.0.86:200::200::00:00:0b:0a:00:0b/304
to 11.0.100.18 via xe-2/0/0.10
to 11.0.100.22 via xe-2/0/1.10
3:11.99.0.86:200::200::11.99.0.86/304
to 11.0.100.18 via xe-2/0/0.10
to 11.0.100.22 via xe-2/0/1.10
Remote MAC
Advertisement Routes
Remote Inclusive Multicast
Ethernet Tag Route

ETHERNET SEGMENT ROUTE – TYPE 4
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS
9
Simplifies Configuration
Loop Avoidance via DF selection
Ethernet Segment
route sent to every
EVPN peer for ESI
discovery
• Ethernet Segment Identifier allows multi-homing of CEs to PE
• PEs connected to the same Ethernet segment discover each
other by exchanging of Ethernet Segment route.
• Include ES-Import extended community with value auto-derived
from the MAC address portion of ESI
• Only PEs that host that ESI import this route
• DF selection is carried out based on ES routes
ES Route
RD
ESI
IP Addr Length
Originator’s IP Addr

EVPN ACTIVE/STANDBY MULTI-HOMING: ROUTE TYPE 4
– ETHERNET SEGMENT ROUTE
juniper@mx-re1 show route table bgp.evpn.0 detail
4:11.99.0.86:0::202200000000000002:11.99.0.86/304 (1 entry, 0 announced) Format =
Type:RD::ESI:Originating-Router-IP/304
*BGP Preference: 170/-101
Route Distinguisher: 11.99.0.86:0 RD set to PE IP address followed by zero
[…]
Source: 11.99.0.86
Protocol next hop: 11.99.0.86
[…]
Local AS: 65320 Peer AS: 65320
Age: 8:37 Metric2: 1
Validation State: unverified
Task: BGP_65320.11.99.0.86+179
AS path: I
Communities: es-import-target:22-0-0-0-0-0 ES-Import Route Target – auto-derived
from ESI (byte 3 to byte 8)
Import Accepted
Localpref: 100
Router ID: 11.99.0.86
Secondary Tables: __default_evpn__.evpn.0

LOOP PREVENTION ON MULTI-HOMED SEGMENTS
ESI Label is used to prevent loops on multi-homed ESI segments
ESI Label is distributed as part of Ethernet A-D Route (ESI Label
Extended Community)
ESI Label is downstream assigned MPLS label in case of ingress replication
ESI Label is upstream assigned in case of P2MP LSP
CE1
PE1
PE2
MPLS
PE3
PE4
CE2
LAG LAG

HOW TO PREVENT DUPLICATE COPIES ON MULTI-HOMED
SEGMENTS?
Designated Forwarder (DF) is elected for each EVI or entire
Ethernet Segment.
DF is responsible for forwarding of BUM traffic
Default procedure for DF election is ESI, EVI allowing to
load-balance BUM traffic (for different EVIs) across multiple
PEs
CE1
PE1
PE2
MPLS
PE3 CE2
LAG

HOW TO LOAD BALANCE TRAFFIC TOWARDS ALL A/A
PES ON THE ETHERNET SEGMENT?
EVPN introducing a concept of Aliasing.
Each PE signals that it has reachability to a given Ethernet
segment (using Ethernet A-D Route)
Remote PE should install all PEs as next-hop which are
attached to the same Ethernet Segment
ESI1 MAC1 - ESI1 - (PE1, PE2)
CE1
PE1
PE2
DF
MPLS
PE3 CE2
MAC1 LAG

FAST CONVERGENCE IN ACTIVE/BACKUP ATTACHED
ETHERNET SEGMENT ?
EVPN introducing a concept of Backup-Path.
Each PE signals that it has reachability to a given Ethernet
segment (using Ethernet A-D Route)
Remote PE should install backup paths to all further PEs which
have reachability to particular Ethernet Segment
ESI1 MAC1 - ESI1 - (PE1 BACKUP, PE2 ACTIVE)
CE1
PE1
PE2
DF
MPLS
PE3 CE2
MAC1 LAG

ARP PROXY
PE can snoop ARP messages for locally attached hosts.
MAC/IP binding can be then redistributed to other PEs by using
MAC Advertisement Route.
ARP REQUEST FOR IP3
CE1 PE1
PE2
DF
MPLS
PE3 CE3
MAC1, IP1
MAC2, IP2 CE2
MAC3, IP3
ARP REPLY FOR IP3
ARP REQUEST FOR IP3
ARP REPLY FOR IP3

MAC MOBILITY AND DUPLICATED MACS
Each time MAC moves to different Ethernet Segment
incremented Sequence Number is included in MAC
Advertisement Route by PE which is attached to the new
segment
Advertisement should be disabled if local PE learns same
address N times within M seconds
ESI1
CE1
MAC moves from ESI1 to ESI2
PE1
PE2
DF
MPLS
PE3 CE2
MAC1 LAG
MAC1
ESI2

MAC MOVE – BASED ON LATEST LEARNED MAC
ADVERTISEMENT ROUTE
MAC 00:00:0b:0a:01:11 initially connected to PE1:
juniper@mx-re1 show evpn mac-table
Routing instance : EVPN-1
MAC MAC Logical NH RTR
address flags interface Index ID
[…]
00:00:0b:0a:01:11 D,SE ge-1/0/1.10
MAC 00:00:0b:0a:01:11 moves to PE2:
juniper@mx2-re1 show evpn mac-table
Routing instance : EVPN-1
MAC MAC Logical NH RTR
address flags interface Index ID
[…]
00:00:0b:0a:01:11 D ge-10/0/3.20
PE2 advertises new MAC address. PE1 deletes MAC address from local table:
May 22 13:50:38.228221 EVPN instance EVPN-1 [VLAN: 200, Refcount: 3, Intfs: 2 (2 up), IRBs: 0 (0 up), Remote
PEs: 1, Flags: 0x8] Received MAC advertisement route (type 2) from BGP
May 22 13:50:38.228244 EVPN instance EVPN-1 [VLAN: 200, Refcount: 3, Intfs: 2 (2 up), IRBs: 0 (0 up), Remote
PEs: 1, Flags: 0x8] Processing ADD for MAC 00:00:0b:0a:01:11 from 11.99.0.86 with ESI 0, VLAN 200, label
301072
May 22 13:50:38.228282 EVPN MAC peer EVPN-1::200::00:00:0b:0a:01:11::11.99.0.86 [MAC: no, MAC+IPs: 0, Active:
yes] Created
May 22 13:50:38.228325 EVPN MAC 00:00:0b:0a:01:11 (remote) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 Adv]
Created and added to MAC database
May 22 13:50:38.731442 EVPN MAC 00:00:0b:0a:01:11 (local) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 Adv]
Deleting MAC advertisement route
May 22 13:50:38.731458 EVPN route (local) [Instance: EVPN-1, Type: MAC advertisement (2), ESI: 0, VLAN: 200]
Withdrawing MAC route
May 22 13:50:38.731543 EVPN MAC 00:00:0b:0a:01:11 (local) [Instance: EVPN-1, VLAN: 200, Flags: 0x10 Adv]
Deleted from MAC database

IRB SUPPORT WITHIN EVPN
IRB allows to forward not only L2 but L3 traffic as well on the
same PE
In case of multiple locations (e.g. DC locations) it is desired to
use local forwarding for L3 traffic to avoid trombone effect
Each PE that acts as a Default GW for a given EVPN should
advertise its Default GW IP and MAC address using MAC
Advertisement Route (with Default Gateway Extended
Community).
All receiving PE should reply to all ARP requests received to
this IP address and should forward traffic destined to this MAC
address locally

EVPN WITH IRB – EVPN MAC ROUTE WITH DEFAULT
GATEWAY
juniper@mx-re0 show route table EVPN-1.evpn.0
2:11.99.0.13:200::200::84:18:88:2a:5f:f0::11.10.0.62/304 (1 entry, 1 announced)
MAC route includes default gateway IP address
*EVPN Preference: 170
[…]
AS path: I
Communities: evpn-default-gateway Default Gateway Extended Community
Route Label: 303632
ESI: 00:00:00:00:00:00:00:00:00:00

EVPN IN OPERATION – TRAFFIC FLOW OVERVIEW
PE2
PE1 (DF)
PE3
PE4
CE
CE
MPLS
@7
PE 3 as a non-DF
@ 2
@1
@4
@5
9
PE2 Drops Traffic as
it’s originated from
same ESI segment
@6
@7
@7
DP
Learning
MPLS LABEL USED
FOR FORWARDING
- Label per EVI
- Per EVI+VLAN
- Per MAC
PE1 receives
broadcast traffic from
CE1. PE1 adds PSN
and IM label and
forwards 3 copies
PE 4 as DF will
forward BUM
traffic into
segment
BGP MAC ADV
ROUTE
• EVPN NRLI
• MAC M1 via
PE1
for a given VLAN
(EVI) will drop the
traffic
BGP MAC ADV
ROUTE
RD ESI
MAC IP LEN
ETH TAG MAC LEN
IP ADDR MPLS
LBL

VIRTUAL MOBILE TRAFFIC
OPTIMIZER

VM DEFAULT GATEWAY PROBLEM
Data Centre (A) Data Centre (B)
Default G/W
PKT
PK
T
• VM does not update default g/w IP or MAC address
• Need a mechanism to ensure traffic exits via nearest g/w

OPTIMIZING INTER-VLAN TRAFFIC FLOWS
Efficient Traffic Trombones without VMTO Routing with VMTO
PRIVATE MPLS WAN PRIVATE MPLS WAN
VLAN 10 VLAN 10 VLAN 10 VLAN 10

WITHOUT VMTO: EGRESS TROMBONE EFFECT
Server 1
VLAN 20
DC 1
20.20.20.100/24
DC 2 VLAN 10
Server 2 Server 3
10.10.10.100/24
DC 3
VLAN 10
10.10.10.200/24
PRIVATE MPLS WAN
Active VRRP
DG:
10.10.10.1
Standby
VRRP
DG:
10.10.10.1
Standby
VRRP
DG:
10.10.10.1
Standby
VRRP
DG:
10.10.10.1
Task:
Server 3 in Data Center 3 needs to send packets
to Server 1 in Data Center 1.
Problem:
Server 3’s active Default Gateway for VLAN 10
is in Data Center 2.
Effect:
1. Traffic must travel via Layer 2 from Data
Center 3 to Data Center 2 to reach VLAN 10’s
active Default Gateway.
2. The packet must reach the Default Gateway
in order to be routed towards Data Center 1.
This results in duplicate traffic on WAN links
and suboptimal routing – hence the “Egress
Trombone Effect.”

WITH VMTO: NO EGRESS TROMBONE EFFECT
Server 1
VLAN 20
DC 1
20.20.20.100/24
DC 2 VLAN 10
Server 2 Server 3
10.10.10.100/24
DC 3
VLAN 10
10.10.10.200/24
PRIVATE MPLS WAN
Active RVI
DG:
10.10.10.1
Active RVI
DG:
10.10.10.1
Active RVI
DG:
10.10.10.1
Active RVI
DG:
10.10.10.1
Task:
Solution:
Virtualize and distribute the Default Gateway
so it is active on every router that participates
in the VLAN.
Effect:
1. Egress packets can be sent to any router on
VLAN 10, allowing the routing to be done in
the local data center. This eliminates the
“Egress Trombone Effect” and creates the
most optimal forwarding path for the inter-data
center traffic.

VM EGRESS TRAFFIC OPTIMIZATION
EVPN advantages over VPLS:
- No need for VRRP, Multi-homing, MC-LAG (less machinery and
protocol dependencies)
- IRB within EVPN VRF is configured on all PEs with a same IP address
(copypaste IRB config on all PEs)
- Each PE has a mapping between Default GW IP and all PEs MACs
- If VM moves from DC1 to DC2 it continue to use “old” MAC address
from PE located in DC1. However, both PEs in DC2 forward traffic
destined to this MAC locally.
IRB MAC on MX240-4
IRB MAC on MX480-3
IRB MAC on MX480-4

WITHOUT VMTO: INGRESS TROMBONE EFFECT
Server 1
VLAN 20
DC 1
20.20.20.100/24
Route Mask Cost Next Hop
10.10.10.0 24 5 Datacenter 2
10.10.10.0 24 10 Datacenter 3
DC 1’s Edge Router Table Without VMTO
DC 2
VLAN 10
Server 2 Server 3
10.10.10.100/24
DC 3
VLAN 10
10.10.10.200/24
PRIVATE MPLS WAN
Task:
Problem:
Data Center 1’s edge router prefers the path to
Data Center 2 for the 10.10.10.0/24 subnet. It
has no knowledge of individual host IPs.
Effect:
1. Traffic from Server 1 is first routed across
the WAN to Data Center 2 due to a lower cost
route for the 10.10.10.0/24 subnet.
2. Then the edge router in Data Center 2 will
send the packet via Layer 2 to Data Center 3.
10.10.10.0/24 Cost
10.10.10.0/24 Cost 10
5

WITH VMTO: NO INGRESS TROMBONE EFFECT
Server 1
VLAN 20
DC 1
20.20.20.100/24
Route Mask Cost Next Hop
10.10.10.0 24 5 Datacenter 2
10.10.10.0 24 10 Datacenter 3
10.10.10.100 32 5 Datacenter 2
10.10.10.200 32 5 Datacenter 3
DC 1’s Edge Router Table WITH VMTO
10.10.10.100/32 Cost 5
10.10.10.200/32 Cost 5
DC 2
VLAN 10
Task:
Solution:
In addition to sending a summary route of
10.10.10.0/24, the data center edge routers also
send host routes which represent the location
of local servers.
Server 2 Server 3
10.10.10.100/24
DC 3
VLAN 10
10.10.10.200/24
PRIVATE MPLS WAN
Effect:
1. Ingress traffic destined for Server 3 is sent
directly across the WAN from Data Center 1 to
Data Center 3. This eliminates the “Ingress
Trombone Effect” and creates the most
optimal forwarding path for the inter-data
center traffic.
10.10.10.0/24 Cost
5
10.10.10.0/24 Cost
10

SUMMARY

EVPN FORWARDING SUMMARY
BGP Control Plane based learning on WAN
detours
MPLS or IP
DP learning
over LAN
MAC1…………MPLS
nexthop
MAC11…………...LAN ports
DP learning
over LAN
LAG
LAG
MAC1…….……...LAN Ports
MAC11………MPLS nexthop
VLAN 1
MAC1
VLAN 1
MAC11
VLAN 2
MAC2…….……...LAN Ports
MAC22….……MPLS nexthop
MAC2……..….MPLS nexthop
MAC22….……..…LAN ports
MX
Series
MX
Series
detours
MPLS transport label(s)
including detour or IP
transport label
Service label
Ethernet Frame
P2P
connections for
unicast traffic
P2MP
connections for
multicast or
unknown traffic
Hash based
LB on
Ethernet
switch
VLAN 2
MAC2
MAC22

EVPN VS VPLS
NEXTGEN Cloud DC Attributes for L2-Stretch EVPN VPLS
Flexible physical network topologies (hub-n-spoke, mesh, ring)
Scale to 100K+ hosts within and across multiple DCs
Active-Active points of attachment (hosts, routers)
VPN (secure isolation, overlapping MAC, IP addresses)
Near Hitless Host Mobility without renumbering L2 and L3 addresses
Ability to span VLANs across racks in different locations
Controlled learning with Policies
Minimize or eliminate flooding of unknown unicast
Fast convergence from edge failures based on local repair
Multicast at scale with ability to trade bandwidth vs. state
Value Adds: Auto-Cfg, Non-Ethernet links, FRR on transit links

PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center

PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center

Similar to PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center (20)

Recently uploaded

Recently uploaded (20)

PLNOG 13: Emil Gągała: EVPN – rozwiązanie nie tylko dla Data Center