[FROG] Next-Hop/Route Installation Issues with IPv6 on Route Server

Michael H Lambert lambert at psc.edu
Mon Oct 25 21:16:06 UTC 2021


I’m trying to debug a rather strange issue I’m seeing with bgpd from FRR 7.5.1 (latest from FreeBSD 13.0 packages) used as a route server. The single peer for this test case is a VRF on a Cisco 6509 (although the problem also exists with Brocade and a couple of flavors of Junos). Peering is over a tagged (VLAN) subnet, ie, a subinterface on a physical Ethernet.

The motivation is a smooth (only some minor syntactic changes needed) migration from the Euro-IX fork of quagga to FRR. The alternatives (BIRD or OpenBGPD) would probably require semantic changes in addition.

[Note: IP addresses and ASNs have been obfuscated so our security folks won’t yell]

The test configuration is straightforward:

==========

Current configuration:
!
frr version 7.5.1
frr defaults traditional
!
hostname rs-dev-bgpd
password password
enable password password
log syslog
!
debug bgp updates in
debug bgp updates out
!
!
!
router bgp 64512 view TESTING
 bgp router-id 198.51.100.247
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 neighbor 198.51.100.248 remote-as 64536
 neighbor 198.51.100.248 description Peering with TESTING (VRF)
 neighbor 2001:db8:0:fe1f::18 remote-as 64536
 neighbor 2001:db8:0:fe1f::18 description Peering with TESTING (VRF)
 !
 address-family ipv4 unicast
  neighbor 198.51.100.248 activate
  neighbor 198.51.100.248 soft-reconfiguration inbound
  neighbor 198.51.100.248 route-server-client
  neighbor 198.51.100.248 route-map PERMIT_ANY in
  neighbor 198.51.100.248 route-map DENY_ANY out
  neighbor 198.51.100.248 attribute-unchanged next-hop
 exit-address-family
 !
 address-family ipv6 unicast
  neighbor 2001:db8:0:fe1f::18 activate
  neighbor 2001:db8:0:fe1f::18 soft-reconfiguration inbound
  neighbor 2001:db8:0:fe1f::18 route-server-client
  neighbor 2001:db8:0:fe1f::18 route-map PERMIT_ANY in
  neighbor 2001:db8:0:fe1f::18 route-map DENY_ANY out
  neighbor 2001:db8:0:fe1f::18 attribute-unchanged next-hop
 exit-address-family
!
!
!
route-map DENY_ANY deny 100
!
route-map PERMIT_ANY permit 100
!
!
line vty
 exec-timeout 60 0
!
end

==========

I’m checking IPv4 first. The Cisco thinks its announcements are

cisco#show bgp vrf TESTING vpnv4 unicast neighbors 198.51.100.247 advertised-routes
BGP table version is 1867871, local router ID is 198.51.100.249
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network           Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 64536:1 (default for vrf TESTING) VRF Router ID 198.51.100.249
 *>  192.0.2.0/24      0.0.0.0                  0         32768 ?
 *>  198.51.100.224/27 0.0.0.0                  0         32768 ?

Total number of prefixes 2 

==========

The BGP summary on FRR/bgpd shows the expected two routes:

==========

rs-dev-bgpd# show bgp view TESTING ipv4 unicast summary 
BGP router identifier 198.51.100.247, local AS number 64512 vrf-id 1
BGP table version 2
RIB entries 3, using 576 bytes of memory
Peers 1, using 14 KiB of memory

Neighbor         V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
198.51.100.248   4      64536        16        12        0    0    0 00:10:05            2        0

Total number of neighbors 1

==========

And they are also the expected routes, matching what the Cisco thinks it is sending.

==========

rs-dev-bgpd# show bgp view TESTING ipv4 unicast         
BGP table version is 2, local router ID is 198.51.100.247, vrf id 1
Default local pref 100, local AS 64512
Status codes:  s suppressed, d damped, h history, * valid, > best, = multipath,
               i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes:  i - IGP, e - EGP, ? - incomplete

   Network            Next Hop            Metric LocPrf Weight Path
*> 192.0.2.0/24      198.51.100.248            0             0 64536 ?
*> 198.51.100.224/27 198.51.100.248            0             0 64536 ?

Displayed  2 routes and 2 total paths

==========

From more extensive testing, IPv4 seems to be behaving as I think it should.

Now I turn my attention to IPv6. Again, the Cisco seems happy:

==========

cisco#show bgp vrf TESTING vpnv6 unicast neighbors 2001:DB8:0:FE1F::17 advertised-routes
BGP table version is 1642349, local router ID is 198.51.100.249
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 64536:1 (default for vrf TESTING) VRF Router ID 198.51.100.249
 *>  2001:DB8:0:FE1F::/64
                       ::                       0         32768 ?
 *>  2001:DB8:FFFF:9F::/64
                       ::                       0         32768 ?

Total number of prefixes 2 

==========

Two prefixes are being announced over the established BGP session.

Indeed, bgpd sees the session as established, but it’s not receiving prefixes:

rs-dev-bgpd# show bgp view TESTING ipv6 unicast summary 
BGP router identifier 198.51.100.247, local AS number 64512 vrf-id 1
BGP table version 0
RIB entries 0, using 0 bytes of memory
Peers 1, using 14 KiB of memory

Neighbor            V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt
2001:db8:0:fe1f::18 4      64536        16        12        0    0    0 00:10:16            0        0

Total number of neighbors 1
rs-dev-bgpd# show bgp view TESTING ipv6 unicast
No BGP prefixes displayed, 0 exist

==========

So what’s going on here?

In syslog (daemon.log), I see a few lines, two of which seem pretty strange:

==========

Oct 25 14:29:50 rs-dev bgpd[59414]: [EC 33554503] 2001:db8:0:fe1f::18 unrecognized capability code: 70 - ignored
Oct 25 14:29:50 rs-dev bgpd[59414]: %ADJCHANGE: neighbor 2001:db8:0:fe1f::18(Unknown) in vrf  Up
Oct 25 14:29:50 rs-dev bgpd[59414]: 2001:db8:0:fe1f::18 sent a v6 LL next-hop and there's no peer interface information. Hence, withdrawing
Oct 25 14:29:50 rs-dev bgpd[59414]: [EC 33554487] 2001:db8:0:fe1f::18: Attribute MP_REACH_NLRI, parse error - treating as withdrawal
Oct 25 14:29:50 rs-dev bgpd[59414]: [EC 33554454] 2001:db8:0:fe1f::18 rcvd UPDATE with errors in attr(s)!! Withdrawing route.
Oct 25 14:29:50 rs-dev bgpd[59414]: 2001:db8:0:fe1f::18 [Info] UPDATE for non-enabled AFI/SAFI 1/1
Oct 25 14:29:50 rs-dev bgpd[59414]: bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from 2001:db8:0:fe1f::18 in vrf default

==========

Several comments/observations:

1) The link-local next-hop and AFI/SAFI messages are present whether the peer is the Cisco or a Juniper or Brocade/Extreme. These are particularly befuddling, since they have no trouble talking to quagga or to each other.

2) I tried assigning a different MAC address to the subinterface than that which was on the physical interface with the same results. I thought this might take care of link-local addresses and neighbor discovery issues.

3) On Friday, I built the code from the head of the git master and tested it, again with the same results.

4) If I mock this up on VMs under VirtualBox, using (emulated) physical interfaces (not tagged subinterfaces) and OpenBSD/OpenBGPD speakers instead of vendor routers, IPv4 and IPv6 both work as expected. I will note that I’m using ULAs and not global unicast addresses. In this case, it’s the same version of FreeBSD and FRR as in the physical test.

At this point, my best guess is that I’m running into an interaction issue between FRR/bgpd and tagged subinterfaces. If that is the case, is it across all platforms? Restricted to FreeBSD? Specific to FreeBSD 13? Am I just missing another knob on bgpd for running as a route server? Do I need to run vtysh to have the interfaces handled properly? Do I need to do something with zebra even though there is no forwarding involved, just routing?

Any insights would be most appreciated.

Michael

-----
Michael H Lambert, GigaPoP Manager             Phone: +1 412 268-4960
Pittsburgh Supercomputing Center/3ROX          FAX:   +1 412 268-5832
300 S Craig St, Pittsburgh, PA  15213 USA      lambert at psc.edu





More information about the frog mailing list