-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I figured out what's happening. FRR (I'm assuming zebra) really doesn't like having routes bound to VRRP interfaces. It works so long as everything is up, but if there's a VRRP state transition and the VRRP interface or the underlying interface go away, zebra stops redistributing any routes into the kernel table at all until FRR is completely stopped and started again. It does still run the protocols, I could see static and connected routes on my problem box in the OSPF routes on its neighbour, it just wouldn't put anything in the kernel table on the problem box until I removed the static routes and restarted everything. Adding the static routes without an interface specified works fine, but doesn't quite do what I want, which is for the routes to be withdrawn when the VRRP goes down. I guess I'll have to script that somehow. Thanks. Alasdair Muckart via frog <frog@lists.frrouting.org> writes:

[1. message/rfc822] From: Alasdair Muckart <alasdairmuckart@catalyst.net.nz> Subject: Re: [FROG] Static route stuck in 'inactive' state To: frog@lists.frrouting.org Date: Tue, 11 Jun 2024 19:09:52 +1200 (14 hours, 57 minutes, 32 seconds ago) Reply-To: alasdairmuckart@catalyst.net.nz Flags: seen, list, personal Maildir: /catalyst/Lists/Frog

[[PGP Signed Part:Undecided]] [2. text/plain]

And now all three routes are stuck in "inactive" state, this persists even if I restart FRR.

Advice on how to debug this further would be greatly appreciated. As it stands, FRR is proving far too brittle for me to put into production even in the very basic use case I've got it deployed in now and I'm not sure why.

Thanks.

--- cut --- S 10.0.0.0/8 [1/0] via 192.168.146.134, vrrp4-1-2 inactive, weight 1, 00:00:08 S 172.16.0.0/12 [1/0] via 192.168.146.134, vrrp4-1-2 inactive, weight 1, 00:00:08 S 192.168.0.0/16 [1/0] via 192.168.146.134, vrrp4-1-2 inactive, weight 1, 00:00:08 --- cut ---

FRR knows about the next-hop: --- cut --- # show ip route 192.168.146.134 Routing entry for 192.168.146.128/29 Known via "connected", distance 0, metric 0, best Last update 00:37:33 ago * directly connected, enp6s0f3 --- cut ---

VRRP is working, and this node is the master --- cut --- # show vrrp 2

Virtual Router ID 2 Protocol Version 3 Autoconfigured No Shutdown No Interface enp6s0f3 VRRP interface (v4) vrrp4-1-2 VRRP interface (v6) None Primary IP (v4) 192.168.146.130 Primary IP (v6) :: Virtual MAC (v4) 00:00:5e:00:01:02 Virtual MAC (v6) 00:00:5e:00:02:02 Status (v4) Master Status (v6) Initialize Priority 110 Effective Priority (v4) 110 Effective Priority (v6) 110 Preempt Mode Yes Accept Mode Yes Checksum with IPv4 Pseudoheader Yes Advertisement Interval 1000 ms Master Advertisement Interval (v4) Rx 1000 ms (stale) Master Advertisement Interval (v6) Rx 0 ms (stale) Advertisements Tx (v4) 8654 Advertisements Tx (v6) 0 Advertisements Rx (v4) 28 Advertisements Rx (v6) 0 Gratuitous ARP Tx (v4) 2 Neigh. Adverts Tx (v6) 0 State transitions (v4) 4 State transitions (v6) 0 Skew Time (v4) 570 ms Skew Time (v6) 0 ms Master Down Interval (v4) 3570 ms Master Down Interval (v6) 0 ms IPv4 Addresses 1 .................................. 192.168.146.129 IPv6 Addresses 0 --- cut ---

Alasdair Muckart via frog <frog@lists.frrouting.org> writes:

...

[1. message/rfc822] From: Alasdair Muckart <alasdairmuckart@catalyst.net.nz> Subject: Re: [FROG] Static route stuck in 'inactive' state To: frog@lists.frrouting.org Date: Tue, 11 Jun 2024 09:53:26 +1200 (9 hours, 16 minutes, 26 seconds ago) Reply-To: alasdairmuckart@catalyst.net.nz Flags: seen, list, personal Maildir: /catalyst/Lists/Frog

[[PGP Signed Part:Undecided]] [2. text/plain]

If I delete the route and re-add it, it works so I'm not sure what dropped it into an "inactive" state.

Interface information:

Configuration: --- cut --- interface enp6s0f3 vrrp 2 vrrp 2 priority 110 vrrp 2 ip 192.168.146.129 exit --- cut ---

Show interfaces from vtysh --- cut --- # show interface enp6s0f3 Interface enp6s0f3 is up, line protocol is up Link ups: 15 last: 2024/06/10 17:29:11.53 Link downs: 2 last: 2024/06/08 04:35:15.39 vrf: default Description: to cat-wlgwil-prod-ffw via old network index 9 metric 0 mtu 1500 speed 1000 txqlen 1000 flags: <UP,BROADCAST,RUNNING,MULTICAST> Type: Ethernet HWaddr: 90:3c:b3:3f:da:16 inet 192.168.146.130/29 inet6 fe80::923c:b3ff:fe3f:da16/64 Interface Type Other Interface Slave Type None protodown: off

# show interface vrrp4-1-2 Interface vrrp4-1-2 is up, line protocol is up Link ups: 10 last: 2024/06/10 17:22:50.29 Link downs: 5 last: 2024/06/08 04:35:22.37 vrf: default Description: VRRP facing cat-wlgwil-prod-ffw on old network index 24 metric 0 mtu 1500 speed 1000 txqlen 1000 flags: <UP,BROADCAST,RUNNING,MULTICAST> Type: Ethernet HWaddr: 00:00:5e:00:01:02 inet 192.168.146.129/32 unnumbered inet6 fe80::200:5eff:fe00:102/64 Interface Type macvlan Interface Slave Type None protodown: off Parent interface: enp6s0f3 --- cut ---

IP link and address info from OS. --- cut --- # ip link show dev enp6s0f3 9: enp6s0f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group 300 qlen 1000 link/ether 90:3c:b3:3f:da:16 brd ff:ff:ff:ff:ff:ff

# ip a show dev enp6s0f3 9: enp6s0f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group 300 qlen 1000 link/ether 90:3c:b3:3f:da:16 brd ff:ff:ff:ff:ff:ff inet 192.168.146.130/29 brd 192.168.146.135 scope global enp6s0f3 valid_lft forever preferred_lft forever inet6 fe80::923c:b3ff:fe3f:da16/64 scope link valid_lft forever preferred_lft forever

# ip link show dev vrrp4-1-2 24: vrrp4-1-2@enp6s0f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group 300 qlen 1000 link/ether 00:00:5e:00:01:02 brd ff:ff:ff:ff:ff:ff

# ip a show dev vrrp4-1-2 24: vrrp4-1-2@enp6s0f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group 300 qlen 1000 link/ether 00:00:5e:00:01:02 brd ff:ff:ff:ff:ff:ff inet 192.168.146.129/32 scope global vrrp4-1-2 valid_lft forever preferred_lft forever inet6 fe80::200:5eff:fe00:102/64 scope link valid_lft forever preferred_lft forever --- cut ---

Alasdair Muckart via frog <frog@lists.frrouting.org> writes:

...

[1. message/rfc822] From: Alasdair Muckart <alasdairmuckart@catalyst.net.nz> Subject: Static route stuck in 'inactive' state To: frog@lists.frrouting.org Date: Mon, 10 Jun 2024 17:10:17 +1200 (16 hours, 43 minutes, 9 seconds ago) Reply-To: alasdairmuckart@catalyst.net.nz Flags: seen, list, personal Maildir: /catalyst/Lists/Frog

[[PGP Signed Part:Undecided]] [2. text/plain] I have a static route that's stuck in 'inactive' state and not being inserted into the kernel routing table, and I can't figure out why.

Can anyone shed light on this?

FRRouting 10.0 on Linux(5.15.0-112-generic), Ubuntu 22.04.

I have three static routes in my configuration:

--- cut --- ip route 10.0.0.0/8 192.168.146.134 vrrp4-1-2 ip route 172.16.0.0/12 192.168.146.134 vrrp4-1-2 ip route 192.168.0.0/16 192.168.146.134 vrrp4-1-2 --- cut ---

Two of them work but the 192.168.0.0/16 route is stuck "inactive" in the output of 'show ip route static':

--- cut --- Codes: K - kernel route, C - connected, L - local, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR, f - OpenFabric, t - Table-Direct, > - selected route, * - FIB route, q - queued, r - rejected, b - backup t - trapped, o - offload failure

S> 10.0.0.0/8 [1/0] via 192.168.146.134, vrrp4-1-2 (recursive), weight 1, 01:22:16 via 192.168.146.134, vrrp4-1-2 onlink, weight 1, 01:22:16 S> 172.16.0.0/12 [1/0] via 192.168.146.134, vrrp4-1-2 (recursive), weight 1, 01:22:16 via 192.168.146.134, vrrp4-1-2 onlink, weight 1, 01:22:16 S 192.168.0.0/16 [1/0] via 192.168.146.134, vrrp4-1-2 inactive, weight 1, 01:22:16 --- cut ---

Looking at the routes on the machine, I see routes for the 10/8 and 172.16/12, but not 192.168/16:

--- cut --- 10.0.0.0/8 nhid 84 via 192.168.146.134 dev vrrp4-1-2 proto static metric 20 onlink 172.16.0.0/12 nhid 84 via 192.168.146.134 dev vrrp4-1-2 proto static metric 20 onlink --- cut ---

Thanks.

- -- Alasdair Muckart (he/him) Network Infrastructure Architect Catalyst.Net Limited - Expert Open Source Solutions Catalyst.Net Ltd - a Catalyst IT group company DDI: +64 4 897 7794 | Mobile: +64 22 638 5141 | Tel: +64 4 499 2267 | www.catalyst.net.nz CONFIDENTIALITY NOTICE: This email is intended for the named recipients only. It may contain privileged, confidential or copyright information. If you are not the named recipient, any use, reliance upon, disclosure or copying of this email or its attachments is unauthorised. If you have received this email in error, please reply via email or call +64 4 499 2267. -----BEGIN PGP SIGNATURE----- iQJUBAEBCgA+FiEEu4g3jwJ68cPCdgH9iBAgH4ERwwMFAmZo4bcgHGFsYXNkYWly bXVja2FydEBjYXRhbHlzdC5uZXQubnoACgkQiBAgH4ERwwPwOw/6A0lZIwxvYSso rqUgR8WCjWgbfD0z9f1IOtIwtMTvLL50v8YOCORf0FOxpQr5PSkFTweBr7Kt3W61 u9D0M2a3UxS3mHcuEMqnk5n+00rDc8fyL0V0d1MADScmurejjK4iw1Yw/itR+m+/ v9mod6ctvYtJZhqinE8MKoVGBqgeCXoF8Mb5UEIZhUkFJdpe7m7+6fgMdUFS0g/J MajkebRZAkDAlS4BsNL239YhPTqYqypJTe4btiZkSE2To9HBDQzFcQiJSJu+9vns KjEqs55i8EGjQaR5o/XPu11uNhHCHrKW/c9QCoIqdIBv81S7m5/wONaYPdxw/7zs uYfWMulCcD2eKfBb7wt2c4j5nG7F5Xt1w+TGqIGDvSLZ2+TF8g46UjyaoVPlBD19 +0dNKaUMjUe7FISbK1bjnJgpghyy0d+oGm/zD5kbLRkYTKqL6JFI09Sk4ev1E7WJ 5BnV2QMm1nvH75Rb4AiOEOAQt1OpfHbfpPGPkvLii0ExNXxEeCDfE9gsDks/5bdu 19Vkl9GQYgAfLFvgJJV48rlYvlYZuM8JIEOldiO+dHm99+2dkwXup1Z370Fz0APl iuTJ9sAMCvlsT2BSCRbOzCC12Nqq5TVRPCFUKo3LBjJq3aMhyDdHkNiWmKF3NGqc IqylC7kVnepPVUlZ8knJd8aAP6Shiwo= =dbH/ -----END PGP SIGNATURE-----