Integrated Routing and Bridging Overlay
In the previous section, we used VXLAN to extend the broadcast domain between leaf switches. EVPN was used to distribute to MAC address information between the leaves.
What we don't have is a way to route outside of the subnet. In this section we discuss the configuration of the integrated routing and bridging (IRB) aspects of the VXLAN and EVPN configuration.
Layer 3 Overlay VLAN
A Vlan and Vlan interface are required to be the 'overlay' VLAN from a layer 3 perspective. The Vlan interface doesn't need an IP address, however the ip forward command is required so that the switch can perform an IP based lookup even when the interface VLAN has no IP address defined.
The Vlan interface is added as a member of the vrf that will be created in the next step.
vlan 512
vn-segment 10512
interface Vlan512
no shutdown
vrf member tennant_1
ip forward
VRF Creation
A VRF is created to separate the layer 3 domain. The VNI is assigned, and it must match the VNI given to the overlay Vlan. We use the auto keyword to automatically define the route distinguisher and the route targets. The route distinguisher becomes <router_id>:<vrf_id>, and the route-target is <router_id>:<l3_vni>.
vrf context tennant_1
vni 10512
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
Anycast Gateways and SVIs
We now add SVIs for each of out VLANs. We're going to use the anycast gateway functionality, which means that all of the SVIs will have the same IP address and MAC address. This allows hosts to move between leaf switches while still maintaining their default gateway and ARP table entry for the default gateway.
# Leaf01
fabric forwarding anycast-gateway-mac 0001.0002.0003
interface Vlan128
no shutdown
vrf member tennant_1
ip address 203.0.113.1/25
fabric forwarding mode anycast-gateway
interface Vlan129
no shutdown
vrf member tennant_1
ip address 203.0.113.129/25
fabric forwarding mode anycast-gateway
# Leaf02
fabric forwarding anycast-gateway-mac 0001.0002.0003
interface Vlan256
no shutdown
vrf member tennant_1
ip address 203.0.113.1/25
fabric forwarding mode anycast-gateway
interface Vlan257
no shutdown
vrf member tennant_1
ip address 203.0.113.129/25
fabric forwarding mode anycast-gateway
NVE Layer 3
The layer 3 VNI is added to the NVE interface and associated with its VRF.
interface nve 1
member vni 10512 associate-vrf
BGP Configuration
Within the BGP configuration, we move into the VRF configuration, and under the ipv4 unicast address family we announce the connected networks and advertise the EVPN information.
# Leaf01
router bgp 65000
vrf tennant_1
address-family ipv4 unicast
network 203.0.113.0/25
network 203.0.113.128/25
advertise l2vpn evpn
# Leaf02
router bgp 65000
vrf tennant_1
address-family ipv4 unicast
network 203.0.113.0/25
network 203.0.113.128/25
advertise l2vpn evpn
Confirmation
In this scenario, we have two hosts connected to two different VLANs on two different leaf switches:
- Host A has an IP of 203.0.113.2, connected to Leaf01 in Vlan 128 (VNI 1024).
- Host B has an IP of 203.0.113.130, connected to Leaf02 in Vlan 257 (VNI 1025).
Before looking at any of the MAC or IP specific informaition, let's make sure out NVE interface is up, the VNIs have been added, and they're up.
Leaf01# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 1024 239.1.1.1 Up CP L2 [128] SA
nve1 1025 239.1.1.1 Up CP L2 [129] SA
nve1 10512 n/a Up CP L3 [tennant_1]
MAC/ARP Tables & L2Routing Information
We first perform some normal, non-EVPN related checks: looking at the MAC table and ARP table. We can see the locally attached host in both of these tables (for brevity I've only shown Leaf01):
Leaf03# show mac address-table vlan 128
...
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 128 a46c.2a77.5801 dynamic 0 F F Eth1/25
G 128 64f6.9d5a.cb89 static - F F sup-eth1(R)
Leaf01# show ip arp vrf tennant_1
...
Address Age MAC Address Interface Flags
203.0.113.2 00:14:30 a46c.2a77.5801 Vlan128
We now take a look at the l2routing table from a MAC (analogous to the MAC address table), and a MAC-IP perspective (analogous to the ARP table):
Leaf03# show l2route mac all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (O):Re-Originated
Topology Mac Address Prod Flags Seq No Next-Hops
----------- -------------- ------ ------------- ---------- ----------------
128 a46c.2a77.5801 Local L, 0 Eth1/25
129 a46c.2a77.5800 BGP SplRcv 0 192.0.2.4
512 64f6.9d5a.cfe5 VXLAN Rmac 0 192.0.2.4
Leaf03# show l2route mac-ip all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (Ro):Re-Originated
Topology Mac Address Prod Flags Seq No Host IP Next-Hops
----------- -------------- ------ ---------- --------------- ---------------
128 a46c.2a77.5801 HMM -- 0 203.0.113.2 Local
129 a46c.2a77.5800 BGP -- 0 203.0.113.130 192.0.2.4
Under that L2 routing MAC table, we see our local MAC has been imported, and we've learn't the remote host's MAC address. We've also learnt the MAC address for the oher leaf switch's L3 VNI interface. This is used to address the VXLAN encapsulated Ethernet frame when we route outside of our subnet to the other leaf switch.
The ARP table contains out locally learnt ARP entry, and the ARP entry learnt from Leaf02.
BGP Table
We now take a look at the BGP table. It's expanded signifcantly since we last looked at the pure layer 2 configuration. We'll break it down into the slightly different type 2 prefixes, and the new type 5 prefixes.
Leaf03# show bgp l2vpn evpn
es extcommunity extcommunity-list
Leaf03# show bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 24, local router ID is 192.0.2.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 192.0.2.3:32895 (L2VNI 1024)
*>l[2]:[0]:[0]:[48]:[a46c.2a77.5801]:[0]:[0.0.0.0]/216
192.0.2.3 100 32768 i
*>l[2]:[0]:[0]:[48]:[a46c.2a77.5801]:[32]:[203.0.113.2]/272
192.0.2.3 100 32768 i
Route Distinguisher: 192.0.2.3:32896 (L2VNI 1025)
*>i[2]:[0]:[0]:[48]:[a46c.2a77.5800]:[0]:[0.0.0.0]/216
192.0.2.4 100 0 i
*>i[2]:[0]:[0]:[48]:[a46c.2a77.5800]:[32]:[203.0.113.130]/272
192.0.2.4 100 0 i
Route Distinguisher: 192.0.2.4:5
*>i[5]:[0]:[0]:[25]:[203.0.113.0]:[0.0.0.0]/224
192.0.2.4 100 0 i
*>i[5]:[0]:[0]:[25]:[203.0.113.128]:[0.0.0.0]/224
192.0.2.4 100 0 i
Route Distinguisher: 192.0.2.4:33024
*>i[2]:[0]:[0]:[48]:[a46c.2a77.5800]:[0]:[0.0.0.0]/216
192.0.2.4 100 0 i
*>i[2]:[0]:[0]:[48]:[a46c.2a77.5800]:[32]:[203.0.113.130]/272
192.0.2.4 100 0 i
Route Distinguisher: 192.0.2.3:3 (L3VNI 10512)
*>i[2]:[0]:[0]:[48]:[a46c.2a77.5800]:[32]:[203.0.113.130]/272
192.0.2.4 100 0 i
* i[5]:[0]:[0]:[25]:[203.0.113.0]:[0.0.0.0]/224
192.0.2.4 100 0 i
*>l 192.0.2.3 100 32768 i
*>l[5]:[0]:[0]:[25]:[203.0.113.128]:[0.0.0.0]/224
192.0.2.3 100 32768 i
* i 192.0.2.4 100 0 i
Type 2 MAC/IP Entries
Type 5 IP IP Prefix Entries
Routing Table
Leaf01# show ip route vrf tennant_1
IP Route Table for VRF "tennant_1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
203.0.113.0/25, ubest/mbest: 1/0, attached
*via 203.0.113.1, Vlan128, [0/0], 10:39:47, direct
203.0.113.1/32, ubest/mbest: 1/0, attached
*via 203.0.113.1, Vlan128, [0/0], 10:39:47, local
203.0.113.2/32, ubest/mbest: 1/0, attached
*via 203.0.113.2, Vlan128, [190/0], 10:39:47, hmm
203.0.113.128/25, ubest/mbest: 1/0
*via 192.0.2.4%default, [200/0], 00:41:38, bgp-65000, internal, tag 65000 (evpn) segid: 10512 tunnelid: 0xc0000204 encap: VXLAN
203.0.113.130/32, ubest/mbest: 1/0
*via 192.0.2.4%default, [200/0], 00:37:19, bgp-65000, internal, tag 65000 (evpn) segid: 10512 tunnelid: 0xc0000204 encap: VXLAN