Hi all,

I have simple two node setup with below configuration on each node. I am trying to verify functionality related to EVPN Type-5 routes and observing few issues as listed below.

Node A ------------------ Node B

Most of the configuration is from

source: https://docs.cumulusnetworks.com/display/DOCS/Ethernet+Virtual+Private+Network+-+EVPN

Node A

router bgp 65000

bgp router-id 203.0.113.1

no bgp default ipv4-unicast

neighbor fabric peer-group

neighbor fabric remote-as 65001

neighbor fabric capability extended-nexthop

neighbor 203.0.113.2 peer-group fabric

address-family ipv4 unicast

network 44.44.44.0/24

exit-address-family

address-family l2vpn evpn

neighbor fabric activate

advertise-all-vni

advertise ipv4 unicast

exit-address-family

router bgp 65000 vrf Sri

bgp router-id 203.0.113.1

address-family ipv4 unicast

network 45.45.45.0/24

exit-address-family

address-family l2vpn evpn

advertise-all-vni

advertise ipv4 unicast

exit-address-family

line vty

Node B

router bgp 65001

bgp router-id 203.0.113.2

no bgp default ipv4-unicast

neighbor fabric peer-group

neighbor fabric remote-as 65000

neighbor fabric capability extended-nexthop

neighbor 203.0.113.1 peer-group fabric

address-family ipv4 unicast

network 54.54.54.0/24

exit-address-family

address-family l2vpn evpn

neighbor fabric activate

advertise-all-vni

advertise ipv4 unicast

exit-address-family

router bgp 65001 vrf Sri

bgp router-id 203.0.113.2

address-family ipv4 unicast

network 55.55.55.0/24

exit-address-family

address-family l2vpn evpn

advertise-all-vni

advertise ipv4 unicast

exit-address-family

line vty

When I had this configuration in frr.conf and rebooted both the nodes, no Type-5 routes were advertised and the only routes I see are Type-3 and Type-2 routes and the output is as below.

Node_A# show bgp evpn route

BGP table version is 2, local router ID is 203.0.113.1

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete

EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]

EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]

EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]

EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

Network Next Hop Metric LocPrf Weight Path

Route Distinguisher: 203.0.113.1:3

*> [2]:[0]:[48]:[00:50:56:b5:b4:53]

203.0.113.1 32768 i

*> [3]:[0]:[32]:[203.0.113.1]

203.0.113.1 32768 i

Route Distinguisher: 203.0.113.1:4

*> [3]:[0]:[32]:[203.0.113.1]

203.0.113.1 32768 i

Route Distinguisher: 203.0.113.2:3

*> [3]:[0]:[32]:[203.0.113.2]

203.0.113.2 0 65001 i

Route Distinguisher: 203.0.113.2:4

*> [3]:[0]:[32]:[203.0.113.2]

203.0.113.2 0 65001 i

Displayed 5 prefixes (5 paths)

dev#

I remember about an issue that was discussed by Donald and others earlier, where VRF configuration sequence does matter for Type-5 Routes. The mail thread that talks about this issue is @

https://lists.frrouting.org/pipermail/frog/2018-July/000205.html

So I removed the bgp configuration from both the nodes as below

Node_A# conf t

Node_A (config)# no router bgp 65000

Node_A (config)# no router bgp 65000 vrf Sri

Node_A (config)#

Node_B# conf t

Node_B (config)# no router bgp 65001

Node_B (config)# no router bgp 65001 vrf Sri

Node_B (config)#

Configured bgp and associated VRF again on the two nodes (vrf Sri configuration was not removed so honoring the sequence requirements from the above thread). After this the bgp evpn routes on each node are as below.

Node A

Node_A# show bgp evpn route

BGP table version is 1, local router ID is 203.0.113.1

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete

EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]

EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]

EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]

EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

Network Next Hop Metric LocPrf Weight Path

Route Distinguisher: 203.0.113.1:5

*> [5]:[0]:[24]:[45.45.45.0]

0.0.0.0 0 32768 i

Displayed 1 prefixes (1 paths)

Node B

Node_B# show bgp evpn route

BGP table version is 1, local router ID is 203.0.113.2

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal

Origin codes: i - IGP, e - EGP, ? - incomplete

EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]

EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]

EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]

EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

Network Next Hop Metric LocPrf Weight Path

Route Distinguisher: 203.0.113.1:5

*> [5]:[0]:[24]:[45.45.45.0]

203.0.113.1 0 0 65000 i

Displayed 1 prefixes (1 paths)

EdgeRouter-2#

Some of the observations:

When the two nodes were brought for the first time, no Type-5 routes were advertised initially. Only Type-3 and Type-2.
Once I unconfigure BGP and configure again, neither Type-3 nor Type-2 routes were showing up.
After this, It looks like Node-A is advertising Type-5 route for one of the prefix that was added through network command and this indeed received by Node-B. But this is not happening from Node B to Node A.
IP-Prefix from VRF Sri was advertised by Node A, but this is not happening for the IP-Prefix that was available at global level.

I am trying to understand the reason for above behavior. Can you help me to identify if I am missing anything in configuration ? If not what am I missing here ? Unfortunately, not many log messages were available in this part of the code and I am trying to add more and debug, but any quick help on this would be really useful.

This is latest FRR (may be a month old). I do have the log files for both the nodes with all necessary BGP debugs enabled.

Let me know if you need any more details on this.

Thanks,

Mohan