[FROG] VxLAN EVPN VTEP on Linux host with frr 8.3.1 no communication

[0xbabaf00l]

Hi list,

I'm trying to run a VxLAN EVPN VTEP (multicast BUM) on a Linux host
running frr 8.3.1.

The interface is up, but there is no traffic going in or coming out the VTEP.

I get these messages in frr.log:
zebra[68]: [VB5HM-D56CP] remote es 03:44:38:39:fe:ed:01:00:00:01 vtep
10.224.4.1 add  with unsupported df_alg 0
zebra[68]: [VB5HM-D56CP] remote es 03:44:38:39:fe:ed:01:00:00:01 vtep
10.224.4.2 add  with unsupported df_alg 0
zebra[68]: [VB5HM-D56CP] remote es 03:44:38:39:be:ef:03:00:00:01 vtep
10.224.5.31 add  with unsupported df_alg 0
zebra[68]: [VB5HM-D56CP] remote es 03:44:38:39:be:ef:03:00:00:01 vtep
10.224.5.32 add  with unsupported df_alg 0

'ip mroute' doesn't show any entries at all.
I tried to subscribe to a random multicast group using 'ip addr add
239.255.55.55/32 dev lo autojoin'
to verify that multicast routing is working in general and it does
create entries in 'ip mroute' output
on the Linux host and the PIM routers connected to. So for me,
multicast seems to be working.

The other VTEPs are on Mellanox switches running Cumulus Linux 5.2.1.
The other VTEPs can reach each other.

Any idea whats wrong?

The config files are mostly identical to the ones on the Cumulus switches.


My setup:
- Debian Linux 11 bullseye
- Kernel 5.18.16-1~bpo11+1
- frr 8.3.1 running in privileged docker container
- ifupdown2 3.1.0-1
- network interfaces (2) unnumbered via eBGP


/etc/network/interfaces:

auto lo
iface lo inet loopback
    address 10.224.6.11/32
    address xxxx:yyyy:zzzz:20ff:1:1:6:11/128

auto 100g-1
iface 100g-1
    mtu 9200

auto 100g-1.11
iface 100g-1.11
    mtu 9200

auto 100g-2
iface 100g-2
    mtu 9200

auto 100g-2.11
iface 100g-2.11
    mtu 9200

auto bridge
iface bridge
    bridge-ports vxlan-3560
    bridge-vlan-aware yes
    bridge-vids 3560

auto vxlan-3560
iface vxlan-3560
    vxlan-id 3560
    vxlan-mcastgrp 239.255.35.60
    bridge-access 3560
    bridge-learning off

auto vlan-3560
iface vlan-3560
    vlan-raw-device bridge
    vlan-id 3560
    address 10.4.31.100/24

/etc/frr/frr.conf:
service integrated-vtysh-config
frr defaults datacenter
log syslog debugging
ip forwarding
ipv6 forwarding
!
interface lo vrf default
 ip pim
 ip igmp
!
interface 100g-1.11 vrf default
 ipv6 nd ra-interval 5
 no ipv6 nd suppress-ra
 ip pim
 ip pim bfd
!
interface 100g-2.11 vrf default
 ipv6 nd ra-interval 5
 no ipv6 nd suppress-ra
 ip pim
 ip pim bfd
!
ip pim ecmp
ip pim rp 10.224.255.1 239.255.0.0/16
!
router bgp 4242000601 vrf default
 bgp router-id 10.224.6.11
 bgp bestpath as-path multipath-relax
 bgp bestpath compare-routerid
 neighbor firewall.underlay peer-group
 neighbor firewall.underlay remote-as external
 neighbor firewall.underlay capability extended-nexthop
 neighbor firewall.underlay bfd
 neighbor firewall.underlay password XXXXXXXXXXXX
 neighbor 100g-1.11 interface peer-group firewall.underlay
 neighbor 100g-2.11 interface peer-group firewall.underlay
!
 address-family ipv4 unicast
  network 10.224.6.11/32
 exit-address-family
!
 address-family ipv6 unicast
  neighbor firewall.underlay activate
  network xxxx:yyyy:zzzz:20ff:1:1:6:11/128
 exit-address-family
!
 address-family l2vpn evpn
  neighbor firewall.underlay activate
  advertise-all-vni
 exit-address-family

vtysh# sh bgp l2vpn evpn summary
BGP router identifier 10.224.6.11, local AS number 4242000601 vrf-id 0
BGP table version 0
RIB entries 1907, using 358 KiB of memory
Peers 2, using 1447 KiB of memory
Peer groups 1, using 64 bytes of memory

Neighbor                       V         AS   MsgRcvd   MsgSent
TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
1-rtr-leaf-core-cit(100g-1.11) 4 4242000101     12114     12088
0    0    0 09:10:56          974      974 N/A
2-rtr-leaf-core-cit(100g-2.11) 4 4242000102     12131     12087
0    0    0 09:10:53          974      974 N/A

Total number of neighbors 2

vtysh# sh evpn vni
detail  json
1a-fw-vnt-core-cit# sh evpn vni
VNI        Type VxLAN IF              # MACs   # ARPs   # Remote VTEPs
 Tenant VRF
3560       L2   vxlan-3560            2        0        0               default

vtysh# sh evpn l2-nh
VTEP             NH id      #ES
10.224.5.31      268435461  1
10.224.4.2       268435459  1
10.224.5.32      268435462  1
10.224.4.1       268435458  1

vtysh# sh evpn es
Type: B bypass, L local, R remote, N non-DF
ESI                            Type ES-IF                 VTEPs
03:44:38:39:be:ef:03:00:00:01  R    -
10.224.5.31,10.224.5.32
03:44:38:39:fe:ed:01:00:00:01  R    -                     10.224.4.1,10.224.4.2

vtysh# sh evpn next-hops vni all
[nothing]

Cheers,
Sascha