EVPN notes
By Aravind
Some common terminologies
EVPN types
Type1 - Ethernet Auto-discovery Route Type2 - MAC/IP Advertisement Route Type3 - Inclusive Multicast Ethernet Tag Route Type4 - Ethernet Segment Route Type5 - IP Prefix Route Type6 - Selective Multicast Ethernet Tag Route Type7 - Multicast Join Sync Route Type8 - Multicast Leave Sync Route Type9 - Per-Region I-PMSI A-D route Type10 - S-PMSI A-D route Type11 - Leaf A-D route
ESI Lags
- Needed for multihoming
- Unique 10B value which can be re-used across nodes
- Supports Active Active and Active Standby models
- Can run LACP on top of ae bundles
ESI Types
- Type 0 (Manually hardcoded ESI value)
- Type 1 (Auto derived from LACP)
- Type 5 (IRB VGA gateway ESI encoded based on ASN)
How are they managed ?
- Managed from control plane using EVPN Type 4 routes and the ES import route target extended community
- Extended community is used within the type-4 Es route to signal using EVPN BGP messages when connecting to Leaf devices with same multihomed CE device
- For example LACP system ID associated would be last 6B
- example: ESI 00:03:03:03:03:03:03:03:03:01 LACP Id: 03:03:03:03:03:01
Types of DC architectures
- Centrally Routed Bridging (CRB) : Inter VNI routing occurs on spine switches
- provision evpn lags at leaf layer
- esi value and lacp system id should be used when connecting multiple leaf devices to the same server. Unique ESI values and lacp system Ids should be used per evpn lag
- Edge Routed Bridging (ERB): Inter VNI routing occurs on leaf switches
- Similar to CRB with difference in IP first hop gateway capability is moved to leaf using irb interface with anycast addressing
- offers ARP suppression capability complemented by advertisement of most specific /32 host type 5 routes from leaf towards spine. This reduces traffic flooding and creates a topology that is oftenj used to support virtual machine traffic optimization.
- Bridged Overlay (BO): Inter Vlan and Inter VNI routing occurs outside of EVPN-VXLAN fabric
- vlans are extended across leafs over a vxlan tunnel
- first hop IP gw are outside of the evpn vxlan fabric.
Evpn Aliasing
- ability of remote device to load balance Layer 2 unicast traffic across an ethernet segment towards end point .
Types of EVPN
- Vlan based
- Vlan bundle
- vlan aware bundle
- Port based
EVPN-Active-Active Loadbalancing ?
EVPN helps in A/A load balancing when CEs are multihomed across multiple PEs.
Below config shows an ae bundle towards the CE. The A/A config is part of ESI lags. The ESI ID should should be the same across all the PEs to which the CEs are multihomed . The below config is part of vlan aware bundle on MX series of routers which provides flexibility.
[edit interfaces]
ae1 {
description "to vsrx11";
flexible-vlan-tagging;
encapsulation flexible-ethernet-services;
## Warning: requires 'esi-lag' license
esi {
00:11:11:11:11:11:11:11:11:11;
all-active;
}
aggregated-ether-options {
minimum-links 1;
lacp {
active;
periodic fast;
system-id 11:11:11:11:11:11;
}
}
unit 10 {
encapsulation vlan-bridge;
vlan-id 10;
}
unit 20 {
encapsulation vlan-bridge;
vlan-id 20;
}
}
[edit]
routing-instances {
EVPN {
instance-type virtual-switch;
protocols {
evpn {
encapsulation mpls;
extended-vlan-list [ 10 20 ];
}
}
bridge-domains {
VLAN-10 {
domain-type bridge;
vlan-id 10;
interface ae1.10;
interface ae2.10;
interface ae3.10;
routing-interface irb.10;
}
VLAN-20 {
domain-type bridge;
vlan-id 20;
interface ae1.20;
interface ae2.20;
interface ae3.20;
routing-interface irb.20;
}
}
route-distinguisher 100:101;
vrf-target target:100:100;
}
}
[edit protocols bgp]
group EVPN {
type internal;
family evpn {
signaling;
}
neighbor 2.2.2.2 {
local-address 1.1.1.1;
}
}
How about EVPN on MX ?
MX routers support EVPN with both vxlan and mpls dataplane.
anycast IRB
Anycast IRB helps in acting as an anycast GW for the CEs which are multihomed. if using a /29 - /31 address to the IRB, then we dont need to necessarily use a virtual-gateway-v4-mac. It will use a mac address which is defined. if using anything else, it is recommended to define a unique mac address so that it doesnt conflict with other operations such as VRRP.
irb {
unit 10 {
virtual-gateway-accept-data;
family inet {
address 10.1.1.254/24 {
virtual-gateway-address 10.1.1.252;
}
}
virtual-gateway-v4-mac 00:11:02:00:01:02;
}
unit 20 {
virtual-gateway-accept-data;
family inet {
address 20.1.1.254/24 {
virtual-gateway-address 20.1.1.252;
}
}
virtual-gateway-v4-mac 00:01:02:00:01:02;
}
}
Operational commands
EVPN state
root@r1_re0# run show bgp summary
Warning: License key missing; requires 'bgp' license
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 2 Peers: 2 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
bgp.evpn.0
35 35 0 0 0 0
inet.0
13 13 0 0 0 0
bgp.l3vpn.0
14 14 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
2.2.2.2 65001 20298 20292 0 1 6d 8:38:48 Establ
bgp.evpn.0: 35/35/35/0
EVPN.evpn.0: 30/30/30/0
__default_evpn__.evpn.0: 5/5/5/0
192.168.2.2 65002 20290 20288 0 3 6d 8:40:57 Establ
inet.0: 13/13/13/0
bgp.l3vpn.0: 14/14/14/0
SANE-CUSTOMER.inet.0: 14/14/14/0
EVPN table
root@r1_re0# run show route table bgp.evpn.0
< --- snipped ---- >
2:100:102::10::00:11:02:00:01:02::10.1.1.252/304 MAC/IP
*[BGP/170] 6d 08:39:39, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 90
2:100:102::10::2c:6b:f5:df:88:70::10.1.1.253/304 MAC/IP
*[BGP/170] 6d 08:39:39, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 16
2:100:102::10::4c:96:14:aa:ef:80::10.1.1.11/304 MAC/IP
*[BGP/170] 6d 08:16:35, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 29
2:100:102::10::4c:96:14:bc:d7:80::10.1.1.13/304 MAC/IP
*[BGP/170] 6d 08:27:18, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 23
2:100:102::10::4c:96:14:bd:01:80::10.1.1.12/304 MAC/IP
*[BGP/170] 6d 08:16:27, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 26
2:100:102::20::00:01:02:00:01:02::20.1.1.252/304 MAC/IP
*[BGP/170] 6d 08:39:39, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 92
2:100:102::20::2c:6b:f5:df:88:70::20.1.1.253/304 MAC/IP
*[BGP/170] 6d 08:39:39, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 16
2:100:102::20::4c:96:14:aa:ef:80::20.1.1.11/304 MAC/IP
*[BGP/170] 6d 08:16:35, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 29
2:100:102::20::4c:96:14:bc:d7:80::20.1.1.13/304 MAC/IP
*[BGP/170] 6d 08:16:27, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 181.1.1.2 via et-0/0/4.0, Push 23
2:100:102::20::4c:96:14:bd:01:80::20.1.1.12/304 MAC/IP
*[BGP/170] 6d 08:16:27, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverifi
Bridge mac table
root@r1_re0# run show bridge mac-table
MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)
Routing instance : EVPN
Bridging domain : VLAN-10, VLAN : 10
MAC MAC Logical NH MAC active
address flags interface Index property source
2c:6b:f5:5f:c8:70 DC 1048579 2.2.2.2
4c:96:14:a2:9d:80 D ae2.10
4c:96:14:b5:29:80 D ae3.10
MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)
Routing instance : EVPN
Bridging domain : VLAN-20, VLAN : 20
MAC MAC Logical NH MAC active
address flags interface Index property source
2c:6b:f5:5f:c8:70 DC 1048579 2.2.2.2
4c:96:14:a2:9d:80 D ae2.20
4c:96:14:b5:29:80 D ae3.20
Bridge mac-ip-table
root@r1_re0# run show bridge mac-ip-table
MAC IP flags (S - Static, D - Dynamic, L - Local , R - Remote, Lp - Local Proxy,
Rp - Remote Proxy, K - Kernel, RT - Dest Route, (N)AD - (Not) Advt to remote,
RE - Re-ARP/ND, RO - Router, OV - Override, Ur - Unresolved,
RTS - Dest Route Skipped, RGw - Remote Gateway, RTF - Dest Route Forced,
SC - Static Config, P - Probe, NLC - No Local Config)
Routing instance : EVPN
Bridging domain : VLAN-10
IP MAC Flags GBP Logical Active
address address Tag Interface source
10.1.1.252 00:11:02:00:01:02 S,K irb.10
10.1.1.253 2c:6b:f5:5f:c8:70 SR,K,RT,RGw 2.2.2.2
10.1.1.254 2c:6b:f5:fd:c5:70 S,K irb.10
10.1.1.12 4c:96:14:a2:9d:80 DLRp,K,RT,AD ae2.10
10.1.1.13 4c:96:14:b5:29:80 DL,K,RT,AD ae3.10
MAC IP flags (S - Static, D - Dynamic, L - Local , R - Remote, Lp - Local Proxy,
Rp - Remote Proxy, K - Kernel, RT - Dest Route, (N)AD - (Not) Advt to remote,
RE - Re-ARP/ND, RO - Router, OV - Override, Ur - Unresolved,
RTS - Dest Route Skipped, RGw - Remote Gateway, RTF - Dest Route Forced,
SC - Static Config, P - Probe, NLC - No Local Config)
Routing instance : EVPN
Bridging domain : VLAN-20
IP MAC Flags GBP Logical Active
address address Tag Interface source
20.1.1.252 00:01:02:00:01:02 S,K irb.20
20.1.1.253 2c:6b:f5:5f:c8:70 SR,K,RT,RGw 2.2.2.2
20.1.1.254 2c:6b:f5:fd:c5:70 S,K irb.20
20.1.1.12 4c:96:14:a2:9d:80 DLRp,K,RT,AD ae2.20
20.1.1.13 4c:96:14:b5:29:80 DLRp,K,RT,AD ae3.20
Bridge flood
root@r1_re0# run show bridge flood
Name: EVPN
CEs: 6
VEs: 2
Bridging domain: VLAN-10
EVPN extended: Yes
Flood Routes:
Prefix Type Owner NhType NhIndex
0x30005/51 FLOOD_GRP_COMP_NH __ves__ comp 697
0x30003/51 FLOOD_GRP_COMP_NH __all_ces__ comp 682
0x30001/51 FLOOD_GRP_COMP_NH __re_flood__ comp 683
Bridging domain: VLAN-20
EVPN extended: Yes
Flood Routes:
Prefix Type Owner NhType NhIndex
0x30006/51 FLOOD_GRP_COMP_NH __ves__ comp 698
0x30004/51 FLOOD_GRP_COMP_NH __all_ces__ comp 684
0x30002/51 FLOOD_GRP_COMP_NH __re_flood__ comp 685
Name: default-switch
CEs: 0
VEs: 0
L2 learning interface
root@r1_re0# run show l2-learning interface
Routing Instance Name : EVPN
Logical Interface flags (DL -disable learning, AD -packet action drop,
LH - MAC limit hit, DN - Interface Down,
MP - MAC Pinning, MI - MAC+IP limit hit)
Logical BD MAC MAC+IP STP Logical
Interface Name Limit Limit State Interface flags
ae1.10 0 0
VLAN-.. 1024 0 Forwarding
Routing Instance Name : EVPN
Logical Interface flags (DL -disable learning, AD -packet action drop,
LH - MAC limit hit, DN - Interface Down,
MP - MAC Pinning, MI - MAC+IP limit hit)
Logical BD MAC MAC+IP STP Logical
Interface Name Limit Limit State Interface flags
ae1.20 0 0
VLAN-.. 1024 0 Forwarding
Routing Instance Name : EVPN
Logical Interface flags (DL -disable learning, AD -packet action drop,
LH - MAC limit hit, DN - Interface Down,
MP - MAC Pinning, MI - MAC+IP limit hit)
Logical BD MAC MAC+IP STP Logical
Interface Name Limit Limit State Interface flags
ae2.10 0 0
VLAN-.. 1024 0 Forwarding
Routing Instance Name : EVPN
Logical Interface flags (DL -disable learning, AD -packet action drop,
LH - MAC limit hit, DN - Interface Down,
MP - MAC Pinning, MI - MAC+IP limit hit)
Logical BD MAC MAC+IP STP Logical
Interface Name Limit Limit State Interface flags
ae2.20 0 0
VLAN-.. 1024 0 Forwarding
Routing Instance Name : EVPN
Logical Interface flags (DL -disable learning, AD -packet action drop,
LH - MAC limit hit, DN - Interface Down,
MP - MAC Pinning, MI - MAC+IP limit hit)
Logical BD MAC MAC+IP STP Logical
Interface Name Limit Limit State Interface flags
ae3.10 0 0
VLAN-.. 1024 0 Forwarding
Routing Instance Name : EVPN
Logical Interface flags (DL -disable learning, AD -packet action drop,
LH - MAC limit hit, DN - Interface Down,
MP - MAC Pinning, MI - MAC+IP limit hit)
Logical BD MAC MAC+IP STP Logical
Interface Name Limit Limit State Interface flags
ae3.20 0 0
VLAN-.. 1024 0 Forwarding
L2 instance learning
root@r1_re0# run show l2-learning instance
Information for routing instance and bridge domain:
Flags (DL -disable learning, SE -stats enabled,
AD -packet action drop, LH -mac limit hit,
MI - mac+ip limit hit)
Inst Logical Routing Bridging Index IRB Flags BD
Type System Instance Domain Index vlan
RTT Default EVPN 1803
Default EVPN VLAN-10 2 343 10
RTT Default EVPN VLAN-20 3 344 20
Default __juniper_private1__ 1
RTT Default __juniper_private1__ ____juniper_private1____ 1 NA
Default default-switch 1802
References
- juniper-multihoming
- juniper-docs-best-practices
- proxy-arp
- mac-mobility
- core-isolation
- evpn-mx-config
- evpn-configs
junos switching