Are you using zebra? BGP has received a v6 LL address and it has absolutely no way to know what interface it has come in on as that it is depending on zebra for this data. donald On Mon, Oct 25, 2021 at 7:15 PM Michael H Lambert <lambert@psc.edu> wrote:

I’m trying to debug a rather strange issue I’m seeing with bgpd from FRR 7.5.1 (latest from FreeBSD 13.0 packages) used as a route server. The single peer for this test case is a VRF on a Cisco 6509 (although the problem also exists with Brocade and a couple of flavors of Junos). Peering is over a tagged (VLAN) subnet, ie, a subinterface on a physical Ethernet.

The motivation is a smooth (only some minor syntactic changes needed) migration from the Euro-IX fork of quagga to FRR. The alternatives (BIRD or OpenBGPD) would probably require semantic changes in addition.

[Note: IP addresses and ASNs have been obfuscated so our security folks won’t yell]

The test configuration is straightforward:

==========

Current configuration: ! frr version 7.5.1 frr defaults traditional ! hostname rs-dev-bgpd password password enable password password log syslog ! debug bgp updates in debug bgp updates out ! ! ! router bgp 64512 view TESTING bgp router-id 198.51.100.247 bgp log-neighbor-changes no bgp ebgp-requires-policy no bgp default ipv4-unicast no bgp network import-check neighbor 198.51.100.248 remote-as 64536 neighbor 198.51.100.248 description Peering with TESTING (VRF) neighbor 2001:db8:0:fe1f::18 remote-as 64536 neighbor 2001:db8:0:fe1f::18 description Peering with TESTING (VRF) ! address-family ipv4 unicast neighbor 198.51.100.248 activate neighbor 198.51.100.248 soft-reconfiguration inbound neighbor 198.51.100.248 route-server-client neighbor 198.51.100.248 route-map PERMIT_ANY in neighbor 198.51.100.248 route-map DENY_ANY out neighbor 198.51.100.248 attribute-unchanged next-hop exit-address-family ! address-family ipv6 unicast neighbor 2001:db8:0:fe1f::18 activate neighbor 2001:db8:0:fe1f::18 soft-reconfiguration inbound neighbor 2001:db8:0:fe1f::18 route-server-client neighbor 2001:db8:0:fe1f::18 route-map PERMIT_ANY in neighbor 2001:db8:0:fe1f::18 route-map DENY_ANY out neighbor 2001:db8:0:fe1f::18 attribute-unchanged next-hop exit-address-family ! ! ! route-map DENY_ANY deny 100 ! route-map PERMIT_ANY permit 100 ! ! line vty exec-timeout 60 0 ! end

==========

I’m checking IPv4 first. The Cisco thinks its announcements are

cisco#show bgp vrf TESTING vpnv4 unicast neighbors 198.51.100.247 advertised-routes BGP table version is 1867871, local router ID is 198.51.100.249 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 64536:1 (default for vrf TESTING) VRF Router ID 198.51.100.249 *> 192.0.2.0/24 0.0.0.0 0 32768 ? *> 198.51.100.224/27 0.0.0.0 0 32768 ?

Total number of prefixes 2

==========

The BGP summary on FRR/bgpd shows the expected two routes:

==========

rs-dev-bgpd# show bgp view TESTING ipv4 unicast summary BGP router identifier 198.51.100.247, local AS number 64512 vrf-id 1 BGP table version 2 RIB entries 3, using 576 bytes of memory Peers 1, using 14 KiB of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt 198.51.100.248 4 64536 16 12 0 0 0 00:10:05 2 0

Total number of neighbors 1

==========

And they are also the expected routes, matching what the Cisco thinks it is sending.

==========

rs-dev-bgpd# show bgp view TESTING ipv4 unicast BGP table version is 2, local router ID is 198.51.100.247, vrf id 1 Default local pref 100, local AS 64512 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path *> 192.0.2.0/24 198.51.100.248 0 0 64536 ? *> 198.51.100.224/27 198.51.100.248 0 0 64536 ?

Displayed 2 routes and 2 total paths

==========

From more extensive testing, IPv4 seems to be behaving as I think it should.

Now I turn my attention to IPv6. Again, the Cisco seems happy:

==========

cisco#show bgp vrf TESTING vpnv6 unicast neighbors 2001:DB8:0:FE1F::17 advertised-routes BGP table version is 1642349, local router ID is 198.51.100.249 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 64536:1 (default for vrf TESTING) VRF Router ID 198.51.100.249 *> 2001:DB8:0:FE1F::/64 :: 0 32768 ? *> 2001:DB8:FFFF:9F::/64 :: 0 32768 ?

Total number of prefixes 2

==========

Two prefixes are being announced over the established BGP session.

Indeed, bgpd sees the session as established, but it’s not receiving prefixes:

rs-dev-bgpd# show bgp view TESTING ipv6 unicast summary BGP router identifier 198.51.100.247, local AS number 64512 vrf-id 1 BGP table version 0 RIB entries 0, using 0 bytes of memory Peers 1, using 14 KiB of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt 2001:db8:0:fe1f::18 4 64536 16 12 0 0 0 00:10:16 0 0

Total number of neighbors 1 rs-dev-bgpd# show bgp view TESTING ipv6 unicast No BGP prefixes displayed, 0 exist

==========

So what’s going on here?

In syslog (daemon.log), I see a few lines, two of which seem pretty strange:

==========

Oct 25 14:29:50 rs-dev bgpd[59414]: [EC 33554503] 2001:db8:0:fe1f::18 unrecognized capability code: 70 - ignored Oct 25 14:29:50 rs-dev bgpd[59414]: %ADJCHANGE: neighbor 2001:db8:0:fe1f::18(Unknown) in vrf Up Oct 25 14:29:50 rs-dev bgpd[59414]: 2001:db8:0:fe1f::18 sent a v6 LL next-hop and there's no peer interface information. Hence, withdrawing Oct 25 14:29:50 rs-dev bgpd[59414]: [EC 33554487] 2001:db8:0:fe1f::18: Attribute MP_REACH_NLRI, parse error - treating as withdrawal Oct 25 14:29:50 rs-dev bgpd[59414]: [EC 33554454] 2001:db8:0:fe1f::18 rcvd UPDATE with errors in attr(s)!! Withdrawing route. Oct 25 14:29:50 rs-dev bgpd[59414]: 2001:db8:0:fe1f::18 [Info] UPDATE for non-enabled AFI/SAFI 1/1 Oct 25 14:29:50 rs-dev bgpd[59414]: bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from 2001:db8:0:fe1f::18 in vrf default

==========

Several comments/observations:

1) The link-local next-hop and AFI/SAFI messages are present whether the peer is the Cisco or a Juniper or Brocade/Extreme. These are particularly befuddling, since they have no trouble talking to quagga or to each other.

2) I tried assigning a different MAC address to the subinterface than that which was on the physical interface with the same results. I thought this might take care of link-local addresses and neighbor discovery issues.

3) On Friday, I built the code from the head of the git master and tested it, again with the same results.

4) If I mock this up on VMs under VirtualBox, using (emulated) physical interfaces (not tagged subinterfaces) and OpenBSD/OpenBGPD speakers instead of vendor routers, IPv4 and IPv6 both work as expected. I will note that I’m using ULAs and not global unicast addresses. In this case, it’s the same version of FreeBSD and FRR as in the physical test.

At this point, my best guess is that I’m running into an interaction issue between FRR/bgpd and tagged subinterfaces. If that is the case, is it across all platforms? Restricted to FreeBSD? Specific to FreeBSD 13? Am I just missing another knob on bgpd for running as a route server? Do I need to run vtysh to have the interfaces handled properly? Do I need to do something with zebra even though there is no forwarding involved, just routing?

Any insights would be most appreciated.

Michael

----- Michael H Lambert, GigaPoP Manager Phone: +1 412 268-4960 Pittsburgh Supercomputing Center/3ROX FAX: +1 412 268-5832 300 S Craig St, Pittsburgh, PA 15213 USA lambert@psc.edu

_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog