Hey peeps hoping someone here has a solution or some idea on this, and
maybe this just isn't possible but I figured id let people with much more
FRR skill than me let me know..
So, I am trying to solve an issue with IPv6 /128 peers and VRF importing.
This is on Ubuntu 20.04.1 with FRR 7.5-0~ubuntu20.04
I have 2 VRFs, the default and dn42 (table 42), currently I import both
VRFs into each other for IPv4 and IPv6.
In my default VRF I only have 1 downstream peer that is my own, all of my
other peers are in the dn42 VRF.
I have no issues when my BGP peers are in the same subnet as me, such as
links using /64s to /127s etc. All routes import and get installed on this
host in both VRFs
But I do have an issue when usings links with /128s, peering comes up
routes are exchanged and no issues using any of the received routes on the
dn42 VRF, they are installed and work perfectly in the dn42 VRF.
However the routes are never installed in the default VRF table of the
host, but they are announced to my downstream peer by FRR, but when a
packet comes in for the affected route it follows the default route instead
of getting routed into the VRF and beyond because there is no route in the
default table for it.
I will note these are Multiprotocol peering sessions and IPv4 is working
fine, this only affects IPv6. And static routes for the /128 are installed
via "ip route add..", also tried adding via FRR as a test, no change of
course.
Also note, I have 3 peers with /128 links, this affects them all, just
including one example.
I have included a broken peer using the /128s, and a working peer using a
/127, though again peers/links using /64 - /127s are fine.
Broken Peer
Peer AS: 4242421080
Interface: wg1080
My IPv6: fda9:26a9:1c47:d42:42:42:1080:1050/128
Peer Ipv6: fd86:bad:11b7:9::1/128
Working Peer
Peer AS: 4242422601
Interface: wg2601
My IPv6: fda9:26a9:1c47:d42:42:42:2601:1050/127
Peer Ipv6: fda9:26a9:1c47:d42:42:42:2601:1051/127
***********Snipped relevant peers***********
ATL1-US.napshome.dn42# show ipv6 nht vrf dn42
fd86:bad:11b7:9::1
resolved via static
is directly connected, wg1080
Client list: bgp(fd 16)
fda9:26a9:1c47:d42:42:42:2601:1051(Connected)
resolved via connected
is directly connected, wg2601
Client list: bgp(fd 16)
*************************************
*********Broken Route****************
ATL1-US.napshome.dn42# show ipv6 route vrf dn42 fd86:bad:11b7::/48
Routing entry for fd86:bad:11b7::/48
Known via "bgp", distance 20, metric 0, vrf dn42, best
Last update 02:04:19 ago
fd86:bad:11b7:9::1 (recursive), weight 1
* fd86:bad:11b7:9::1, via wg1080 onlink, weight 1
ATL1-US.napshome.dn42# show ipv6 route fd86:bad:11b7::/48
Routing entry for fd86:bad:11b7::/48
Known via "bgp", distance 200, metric 0
Last update 2d00h44m ago
fd86:bad:11b7:9::1(vrf dn42) inactive, weight 1
*************************************
Note: inactive in default VRF
*********Broken Route****************
ip -6 route list fd86:bad:11b7::/48
<Blank/Nothing>
ip -6 route list table 42 fd86:bad:11b7::/48
fd86:bad:11b7::/48 nhid 2095953 via fd86:bad:11b7:9::1 dev wg1080 proto bgp
metric 20 onlink pref medium
*************************************
*********Working Route***************
ATL1-US.napshome.dn42# show ipv6 route vrf dn42 fd42:4242:2601::/48
Routing entry for fd42:4242:2601::/48
Known via "bgp", distance 20, metric 0, vrf dn42, best
Last update 02:07:13 ago
* fe80::42:2601:2e:1, via wg2601, weight 1
ATL1-US.napshome.dn42# show ipv6 route fd42:4242:2601::/48
Routing entry for fd42:4242:2601::/48
Known via "bgp", distance 200, metric 0, best
Last update 2d00h47m ago
* fe80::42:2601:2e:1, via wg2601(vrf dn42), weight 1
*************************************
*********Working Route***************
ip -6 route list fd42:4242:2601::/48
fd42:4242:2601::/48 nhid 2094697 via fe80::42:2601:2e:1 dev wg2601 proto
bgp metric 20 pref medium
ip -6 route list table 42 fd42:4242:2601::/48
fd42:4242:2601::/48 nhid 2094697 via fe80::42:2601:2e:1 dev wg2601 proto
bgp metric 20 pref medium
*************************************
*********Broken Peer***************
ATL1-US.napshome.dn42# show bgp vrf dn42 neighbors fd86:bad:11b7:9::1
BGP neighbor is fd86:bad:11b7:9::1, remote AS 4242421080, local AS
4242421050, external link
Description: jlu5-dn42-ATL
Member of peer-group DN42fullpeer for session parameters
BGP version 4, remote router ID 172.20.229.119, local router ID
172.22.169.128
BGP state = Established, up for 2d00h48m
Last read 00:00:19, Last write 00:00:02
Hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast
IPv6 Unicast: RX advertised IPv6 Unicast
Route refresh: advertised and received(new)
Address Family IPv4 Unicast: advertised and received
Address Family IPv6 Unicast: advertised and received
Hostname Capability: advertised (name: ATL1-US.napshome.dn42,domain
name: n/a) not received
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, IPv6 Unicast
End-of-RIB received: IPv4 Unicast, IPv6 Unicast
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
IPv6 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 3 3
Notifications: 2 0
Updates: 1697133 320409
Keepalives: 5638 6440
Route Refresh: 1 6
Capability: 0 0
Total: 1702777 326858
Minimum time between advertisement runs is 0 seconds
Update source is wg1080
For address family: IPv4 Unicast
DN42fullpeer peer-group member
Update group 3, subgroup 57
Packet Queue length 0
MED is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
Inbound path policy configured
Outbound path policy configured
Route map for incoming advertisements is *rm-fullpeer-in
Route map for outgoing advertisements is *rm-fullpeer-out
444 accepted prefixes
For address family: IPv6 Unicast
DN42fullpeer peer-group member
Update group 4, subgroup 58
Packet Queue length 0
Inbound soft reconfiguration allowed
MED is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
Inbound path policy configured
Outbound path policy configured
Route map for incoming advertisements is *rm-fullpeer-in
Route map for outgoing advertisements is *rm-fullpeer-out
335 accepted prefixes
Connections established 3; dropped 2
Last reset 2d00h48m, Peer closed the session
Local host: fda9:26a9:1c47:d42:42:42:1080:1050, Local port: 40287
Foreign host: fd86:bad:11b7:9::1, Foreign port: 179
Nexthop: 172.22.169.128
Nexthop global: fda9:26a9:1c47:d42:42:42:1080:1050
Nexthop local: ::
BGP connection: non shared network
BGP Connect Retry Timer in Seconds: 30
Estimated round trip time: 8 ms
Read thread: on Write thread: on FD used: 49
*************************************
*********Working Peer***************
ATL1-US.napshome.dn42# show bgp vrf dn42 neighbors
fda9:26a9:1c47:d42:42:42:2601:1051
BGP neighbor is fda9:26a9:1c47:d42:42:42:2601:1051, remote AS 4242422601,
local AS 4242421050, external link
Description: Burble-dn42-us-CHI1
Member of peer-group DN42fullpeer for session parameters
BGP version 4, remote router ID 193.29.63.150, local router ID
172.22.169.128
BGP state = Established, up for 2d00h48m
Last read 00:00:29, Last write 00:00:10
Hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
4 Byte AS: advertised and received
AddPath:
IPv4 Unicast: RX advertised IPv4 Unicast
IPv6 Unicast: RX advertised IPv6 Unicast
Route refresh: advertised and received(new)
Address Family IPv4 Unicast: advertised and received
Address Family IPv6 Unicast: advertised and received
Hostname Capability: advertised (name: ATL1-US.napshome.dn42,domain
name: n/a) not received
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families by peer:
none
Graceful restart information:
End-of-RIB send: IPv4 Unicast, IPv6 Unicast
End-of-RIB received: IPv4 Unicast, IPv6 Unicast
Local GR Mode: Helper*
Remote GR Mode: Helper
R bit: False
Timers:
Configured Restart Time(sec): 120
Received Restart Time(sec): 120
IPv4 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
IPv6 Unicast:
F bit: False
End-of-RIB sent: Yes
End-of-RIB sent after update: No
End-of-RIB received: Yes
Timers:
Configured Stale Path Time(sec): 360
Message statistics:
Inq depth is 0
Outq depth is 0
Sent Rcvd
Opens: 3 2
Notifications: 0 2
Updates: 1702783 136850
Keepalives: 5637 6445
Route Refresh: 0 10
Capability: 0 0
Total: 1708423 143309
Minimum time between advertisement runs is 0 seconds
Update source is wg2601
For address family: IPv4 Unicast
DN42fullpeer peer-group member
Update group 3, subgroup 57
Packet Queue length 0
MED is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
Inbound path policy configured
Outbound path policy configured
Route map for incoming advertisements is *rm-fullpeer-in
Route map for outgoing advertisements is *rm-fullpeer-out
454 accepted prefixes
For address family: IPv6 Unicast
DN42fullpeer peer-group member
Update group 7, subgroup 56
Packet Queue length 0
Inbound soft reconfiguration allowed
MED is propagated unchanged to this neighbor
Community attribute sent to this neighbor(all)
Inbound path policy configured
Outbound path policy configured
Route map for incoming advertisements is *rm-fullpeer-in
Route map for outgoing advertisements is *rm-fullpeer-out
339 accepted prefixes
Connections established 2; dropped 1
Last reset 2d00h48m, No AFI/SAFI activated for peer
Local host: fda9:26a9:1c47:d42:42:42:2601:1050, Local port: 179
Foreign host: fda9:26a9:1c47:d42:42:42:2601:1051, Foreign port: 54219
Nexthop: 172.22.169.128
Nexthop global: fda9:26a9:1c47:d42:42:42:2601:1050
Nexthop local: fe80::1050
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 30
Estimated round trip time: 20 ms
Read thread: on Write thread: on FD used: 38
*************************************
----------------------------------
Brandon Jackson
bjackson(a)napshome.net
