FRR Adding IPv6 Routes
Hello, I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga. A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command "ip -6 ro sh". Quagga ---------- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium FRR ----- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium I have a simple BGP configuration using Autonomous System Border Router (ASBR). I am running the daemons (zebra, staticd, bgpd) through the all-encompassing config file frr.conf, and I have also tried separating each daemon out to a separate config file and running them separately. I used the same commands from the Quagga confile files, and I added the additional staticd commands. I've looked at all the generated log files, and I can't seem to find anything substantial to investigation. Would anyone have an idea why these IPv6 routes keep toggling from being added and removed? Any additional suggestions on troubleshooting steps? Thank you, Katherine
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below. I suspect this is not why routes are being added and withdrawn. $ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp Not sure what else to say. Can you post some more config? if you type do show ipv6 route <a prefix> what comes out? Is the next hop accessible? I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
Hello, Also, would someone be able to add me to the mailing list group? I have tried to subscribe but haven't heard a response... The next hop toggles between being accessible and not accessible. I run the "show ipv6 route" command at two different times and the IPv6 routes toggles between being added and removed. Vtysh terminal output of the Relay ----------------------------------------- Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00 C>* fd80::1:0/112 is directly connected, eth2, 00:00:21 B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00 B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00 C>* fd80::2:0/112 is directly connected, eth3, 00:00:21 B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00 C * fe80::/10 is directly connected, eth3, 00:00:20 C * fe80::/10 is directly connected, eth2, 00:00:20 C * fe80::/10 is directly connected, eth3, 00:00:21 C>* fe80::/10 is directly connected, eth2, 00:00:21 C * fe80::/64 is directly connected, eth1, 00:00:17 C>* fe80::/64 is directly connected, eth0, 00:00:17 Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route C>* fd80::1:0/112 is directly connected, eth2, 00:02:13 C>* fd80::2:0/112 is directly connected, eth3, 00:02:13 C * fe80::/10 is directly connected, eth3, 00:02:12 C * fe80::/10 is directly connected, eth2, 00:02:12 C * fe80::/10 is directly connected, eth3, 00:02:13 C>* fe80::/10 is directly connected, eth2, 00:02:13 C * fe80::/64 is directly connected, eth1, 00:02:09 C>* fe80::/64 is directly connected, eth0, 00:02:09 Below are the frr.conf config files for my configuration. The Relay, Server 1 and Server 2 are all connected to N3. Relay Config File -------------------- log file /var/log/frr/frr.log debugging ! interface eth0 ip address 10.0.4.2/24 ! interface eth1 ipv6 address 2001::1/64 ! interface aero2 ipv6 address fe80::1:1/10 ipv6 address fd80::1:1/112 ! interface aero3 ipv6 address fe80::2:1/10 ipv6 address fd80::2:1/112 ! ! Static routes ip route 0.0.0.0/0 10.0.4.1 ! ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 1 bgp router-id 10.0.4.2 no bgp default ipv4-unicast neighbor fd80::1:2 remote-as 2 neighbor fd80::1:2 interface aero2 neighbor fd80::2:2 remote-as 3 neighbor fd80::2:2 interface aero3 address-family ipv6 neighbor fd80::1:2 activate neighbor fd80::1:2 next-hop-self neighbor fd80::1:2 default-originate neighbor fd80::1:2 distribute-list blackhole out neighbor fd80::2:2 activate neighbor fd80::2:2 next-hop-self neighbor fd80::2:2 default-originate neighbor fd80::2:2 distribute-list blackhole out ! access-list blackhole deny any exit-address-family ! ipv6 access-list blackhole deny any ! Server1 config file ---------------------- log file /var/log/frr/frr.log debugging interface eth0 ip address 10.0.2.2/24 ! interface aero0 ipv6 address fe80::1:2/10 ipv6 address fd80::1:2/112 ! ! Static routes ! ip route 0.0.0.0/0 10.0.2.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 2 bgp router-id 10.0.2.2 no bgp default ipv4-unicast neighbor fd80::1:1 remote-as 1 neighbor fd80::1:1 interface aero0 neighbor fd80::1:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::1:1 activate neighbor fd80::1:1 next-hop-self neighbor fd80::1:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any ! Server2 config file ---------------------- interface eth0 ip address 10.0.3.2/24 ! interface aero0 ipv6 address fe80::2:2/10 ipv6 address fd80::2:2/112 ! ! Static routes ip route 0.0.0.0/0 10.0.3.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 3 bgp router-id 10.0.3.2 no bgp default ipv4-unicast neighbor fd80::2:1 remote-as 1 neighbor fd80::2:1 interface aero0 neighbor fd80::2:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::2:1 activate neighbor fd80::2:1 next-hop-self neighbor fd80::2:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any N3 config file --------------------- interface eth2 ! interface eth3 ! interface eth4 ! interface eth5 ! interface eth0 ! interface eth1 ! Thank you, Katherine From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes On 02/04/2020 22:25, Tran (US), Katherine K wrote: I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga. A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command "ip -6 ro sh". Quagga ---------- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium FRR ----- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium I think, the display of 186 it just because you need the `friendly` names specifying You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below. I suspect this is not why routes are being added and withdrawn. $ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp Not sure what else to say. Can you post some more config? if you type do show ipv6 route <a prefix> what comes out? Is the next hop accessible? I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
Katherine - You can self subscribe here: https://lists.frrouting.org/listinfo/frog Instructions should be pretty self explanatory I think. thanks! donald On Wed, Apr 8, 2020 at 8:13 AM Tran (US), Katherine K <katherine.k.tran@boeing.com> wrote:
Hello,
Also, would someone be able to add me to the mailing list group? I have tried to subscribe but haven’t heard a response…
The next hop toggles between being accessible and not accessible. I run the “show ipv6 route” command at two different times and the IPv6 routes toggles between being added and removed.
Vtysh terminal output of the Relay
-----------------------------------------
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
C * fe80::/10 is directly connected, eth3, 00:00:20
C * fe80::/10 is directly connected, eth2, 00:00:20
C * fe80::/10 is directly connected, eth3, 00:00:21
C>* fe80::/10 is directly connected, eth2, 00:00:21
C * fe80::/64 is directly connected, eth1, 00:00:17
C>* fe80::/64 is directly connected, eth0, 00:00:17
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
C * fe80::/10 is directly connected, eth3, 00:02:12
C * fe80::/10 is directly connected, eth2, 00:02:12
C * fe80::/10 is directly connected, eth3, 00:02:13
C>* fe80::/10 is directly connected, eth2, 00:02:13
C * fe80::/64 is directly connected, eth1, 00:02:09
C>* fe80::/64 is directly connected, eth0, 00:02:09
Below are the frr.conf config files for my configuration.
The Relay, Server 1 and Server 2 are all connected to N3.
Relay Config File
--------------------
log file /var/log/frr/frr.log debugging
!
interface eth0
ip address 10.0.4.2/24
!
interface eth1
ipv6 address 2001::1/64
!
interface aero2
ipv6 address fe80::1:1/10
ipv6 address fd80::1:1/112
!
interface aero3
ipv6 address fe80::2:1/10
ipv6 address fd80::2:1/112
!
! Static routes
ip route 0.0.0.0/0 10.0.4.1
!
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
! access-list blackhole deny any
exit-address-family
!
ipv6 access-list blackhole deny any
!
Server1 config file
----------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.2.2/24
!
interface aero0
ipv6 address fe80::1:2/10
ipv6 address fd80::1:2/112
!
! Static routes
!
ip route 0.0.0.0/0 10.0.2.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 2
bgp router-id 10.0.2.2
no bgp default ipv4-unicast
neighbor fd80::1:1 remote-as 1
neighbor fd80::1:1 interface aero0
neighbor fd80::1:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::1:1 activate
neighbor fd80::1:1 next-hop-self
neighbor fd80::1:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
!
Server2 config file
----------------------
interface eth0
ip address 10.0.3.2/24
!
interface aero0
ipv6 address fe80::2:2/10
ipv6 address fd80::2:2/112
!
! Static routes
ip route 0.0.0.0/0 10.0.3.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 3
bgp router-id 10.0.3.2
no bgp default ipv4-unicast
neighbor fd80::2:1 remote-as 1
neighbor fd80::2:1 interface aero0
neighbor fd80::2:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::2:1 activate
neighbor fd80::2:1 next-hop-self
neighbor fd80::2:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
N3 config file
---------------------
interface eth2
!
interface eth3
!
interface eth4
!
interface eth5
!
interface eth0
!
interface eth1
!
Thank you,
Katherine
From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying
You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
I suspect this is not why routes are being added and withdrawn.
$ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp
Not sure what else to say. Can you post some more config?
if you type
do show ipv6 route <a prefix>
what comes out?
Is the next hop accessible?
I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
On Wed, 8 Apr 2020 at 14:17, Donald Sharp <sharpd@cumulusnetworks.com> wrote:
Katherine - You can self subscribe here: https://lists.frrouting.org/listinfo/frog Instructions should be pretty self-explanatory I think.
or the usual suspects of using mailman based lists: - to: frog-subscribe@lists.frrouting.org - to: frog-request@lists.frrouting.org, subject: "subscribe" (then confirm the message that is returned)
Hello, The next hop toggles between being accessible and not accessible. I run the "show ipv6 route" command at two different times and the IPv6 routes toggles between being added and removed. Vtysh terminal output of the Relay ----------------------------------------- Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00 C>* fd80::1:0/112 is directly connected, eth2, 00:00:21 B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00 B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00 C>* fd80::2:0/112 is directly connected, eth3, 00:00:21 B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00 C * fe80::/10 is directly connected, eth3, 00:00:20 C * fe80::/10 is directly connected, eth2, 00:00:20 C * fe80::/10 is directly connected, eth3, 00:00:21 C>* fe80::/10 is directly connected, eth2, 00:00:21 C * fe80::/64 is directly connected, eth1, 00:00:17 C>* fe80::/64 is directly connected, eth0, 00:00:17 Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route C>* fd80::1:0/112 is directly connected, eth2, 00:02:13 C>* fd80::2:0/112 is directly connected, eth3, 00:02:13 C * fe80::/10 is directly connected, eth3, 00:02:12 C * fe80::/10 is directly connected, eth2, 00:02:12 C * fe80::/10 is directly connected, eth3, 00:02:13 C>* fe80::/10 is directly connected, eth2, 00:02:13 C * fe80::/64 is directly connected, eth1, 00:02:09 C>* fe80::/64 is directly connected, eth0, 00:02:09 Below are the frr.conf config files for my configuration. The Relay, Server 1 and Server 2 are all connected to N3. Relay Config File -------------------- log file /var/log/frr/frr.log debugging ! interface eth0 ip address 10.0.4.2/24 ! interface eth1 ipv6 address 2001::1/64 ! interface aero2 ipv6 address fe80::1:1/10 ipv6 address fd80::1:1/112 ! interface aero3 ipv6 address fe80::2:1/10 ipv6 address fd80::2:1/112 ! ! Static routes ip route 0.0.0.0/0 10.0.4.1 ! ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 1 bgp router-id 10.0.4.2 no bgp default ipv4-unicast neighbor fd80::1:2 remote-as 2 neighbor fd80::1:2 interface aero2 neighbor fd80::2:2 remote-as 3 neighbor fd80::2:2 interface aero3 address-family ipv6 neighbor fd80::1:2 activate neighbor fd80::1:2 next-hop-self neighbor fd80::1:2 default-originate neighbor fd80::1:2 distribute-list blackhole out neighbor fd80::2:2 activate neighbor fd80::2:2 next-hop-self neighbor fd80::2:2 default-originate neighbor fd80::2:2 distribute-list blackhole out ! access-list blackhole deny any exit-address-family ! ipv6 access-list blackhole deny any ! Server1 config file ---------------------- log file /var/log/frr/frr.log debugging interface eth0 ip address 10.0.2.2/24 ! interface aero0 ipv6 address fe80::1:2/10 ipv6 address fd80::1:2/112 ! ! Static routes ! ip route 0.0.0.0/0 10.0.2.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 2 bgp router-id 10.0.2.2 no bgp default ipv4-unicast neighbor fd80::1:1 remote-as 1 neighbor fd80::1:1 interface aero0 neighbor fd80::1:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::1:1 activate neighbor fd80::1:1 next-hop-self neighbor fd80::1:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any ! Server2 config file ---------------------- interface eth0 ip address 10.0.3.2/24 ! interface aero0 ipv6 address fe80::2:2/10 ipv6 address fd80::2:2/112 ! ! Static routes ip route 0.0.0.0/0 10.0.3.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 3 bgp router-id 10.0.3.2 no bgp default ipv4-unicast neighbor fd80::2:1 remote-as 1 neighbor fd80::2:1 interface aero0 neighbor fd80::2:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::2:1 activate neighbor fd80::2:1 next-hop-self neighbor fd80::2:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any N3 config file --------------------- interface eth2 ! interface eth3 ! interface eth4 ! interface eth5 ! interface eth0 ! interface eth1 ! Thank you, Katherine From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes On 02/04/2020 22:25, Tran (US), Katherine K wrote: I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga. A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command "ip -6 ro sh". Quagga ---------- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium FRR ----- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium I think, the display of 186 it just because you need the `friendly` names specifying You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below. I suspect this is not why routes are being added and withdrawn. $ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp Not sure what else to say. Can you post some more config? if you type do show ipv6 route <a prefix> what comes out? Is the next hop accessible? I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
And do show bgp summary. Do the bgp sessions stay up? Show long timers? Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500? There is an IPv6 funny try sysctl net.ipv6.route.max_size=2147483647 There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115 (sorry, quick reply) Tim On 06/04/2020 22:19, Tran (US), Katherine K wrote:
Hello,
The next hop toggles between being accessible and not accessible. I run the “show ipv6 route” command at two different times and the IPv6 routes toggles between being added and removed.
Vtysh terminal output of the Relay
-----------------------------------------
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
C * fe80::/10 is directly connected, eth3, 00:00:20
C * fe80::/10 is directly connected, eth2, 00:00:20
C * fe80::/10 is directly connected, eth3, 00:00:21
C>* fe80::/10 is directly connected, eth2, 00:00:21
C * fe80::/64 is directly connected, eth1, 00:00:17
C>* fe80::/64 is directly connected, eth0, 00:00:17
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
C * fe80::/10 is directly connected, eth3, 00:02:12
C * fe80::/10 is directly connected, eth2, 00:02:12
C * fe80::/10 is directly connected, eth3, 00:02:13
C>* fe80::/10 is directly connected, eth2, 00:02:13
C * fe80::/64 is directly connected, eth1, 00:02:09
C>* fe80::/64 is directly connected, eth0, 00:02:09
Below are the frr.conf config files for my configuration.
The Relay, Server 1 and Server 2 are all connected to N3.
Relay Config File
--------------------
log file /var/log/frr/frr.log debugging
!
interface eth0
ip address 10.0.4.2/24
!
interface eth1
ipv6 address 2001::1/64
!
interface aero2
ipv6 address fe80::1:1/10
ipv6 address fd80::1:1/112
!
interface aero3
ipv6 address fe80::2:1/10
ipv6 address fd80::2:1/112
!
! Static routes
ip route 0.0.0.0/0 10.0.4.1
!
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
! access-list blackhole deny any
exit-address-family
!
ipv6 access-list blackhole deny any
!
Server1 config file
----------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.2.2/24
!
interface aero0
ipv6 address fe80::1:2/10
ipv6 address fd80::1:2/112
!
! Static routes
!
ip route 0.0.0.0/0 10.0.2.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 2
bgp router-id 10.0.2.2
no bgp default ipv4-unicast
neighbor fd80::1:1 remote-as 1
neighbor fd80::1:1 interface aero0
neighbor fd80::1:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::1:1 activate
neighbor fd80::1:1 next-hop-self
neighbor fd80::1:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
!
Server2 config file
----------------------
interface eth0
ip address 10.0.3.2/24
!
interface aero0
ipv6 address fe80::2:2/10
ipv6 address fd80::2:2/112
!
! Static routes
ip route 0.0.0.0/0 10.0.3.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 3
bgp router-id 10.0.3.2
no bgp default ipv4-unicast
neighbor fd80::2:1 remote-as 1
neighbor fd80::2:1 interface aero0
neighbor fd80::2:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::2:1 activate
neighbor fd80::2:1 next-hop-self
neighbor fd80::2:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
N3 config file
---------------------
interface eth2
!
interface eth3
!
interface eth4
!
interface eth5
!
interface eth0
!
interface eth1
!
Thank you,
Katherine
*From:* Tim Bray [mailto:tim@provu.co.uk] *Sent:* Thursday, April 02, 2020 3:31 PM *To:* frog@lists.frrouting.org *Subject:* Re: [FROG] FRR Adding IPv6 Routes
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying
You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
I suspect this is not why routes are being added and withdrawn.
$ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp
Not sure what else to say. Can you post some more config?
if you type
do show ipv6 route <a prefix>
what comes out?
Is the next hop accessible?
I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue? Doesn't look like the bgp session stay up either.... AERO-Relay# show bgp summary IPv6 Unicast Summary: BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 11120 RIB entries 13, using 2496 bytes of memory Peers 2, using 43 KiB of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd fd80::1:2 4 2 8 5 0 0 0 00:01:32 4 fd80::2:2 4 3 8 5 0 0 0 00:01:32 4 The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it. Thanks, Katherine From: Tim Bray [mailto:tim@provu.co.uk] Sent: Monday, April 06, 2020 2:37 PM To: Tran (US), Katherine K <katherine.k.tran@boeing.com>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes And do show bgp summary. Do the bgp sessions stay up? Show long timers? Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500? There is an IPv6 funny try sysctl net.ipv6.route.max_size=2147483647 There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115 (sorry, quick reply) Tim On 06/04/2020 22:19, Tran (US), Katherine K wrote: Hello, The next hop toggles between being accessible and not accessible. I run the "show ipv6 route" command at two different times and the IPv6 routes toggles between being added and removed. Vtysh terminal output of the Relay ----------------------------------------- Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00 C>* fd80::1:0/112 is directly connected, eth2, 00:00:21 B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00 B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00 C>* fd80::2:0/112 is directly connected, eth3, 00:00:21 B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00 C * fe80::/10 is directly connected, eth3, 00:00:20 C * fe80::/10 is directly connected, eth2, 00:00:20 C * fe80::/10 is directly connected, eth3, 00:00:21 C>* fe80::/10 is directly connected, eth2, 00:00:21 C * fe80::/64 is directly connected, eth1, 00:00:17 C>* fe80::/64 is directly connected, eth0, 00:00:17 Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route C>* fd80::1:0/112 is directly connected, eth2, 00:02:13 C>* fd80::2:0/112 is directly connected, eth3, 00:02:13 C * fe80::/10 is directly connected, eth3, 00:02:12 C * fe80::/10 is directly connected, eth2, 00:02:12 C * fe80::/10 is directly connected, eth3, 00:02:13 C>* fe80::/10 is directly connected, eth2, 00:02:13 C * fe80::/64 is directly connected, eth1, 00:02:09 C>* fe80::/64 is directly connected, eth0, 00:02:09 Below are the frr.conf config files for my configuration. The Relay, Server 1 and Server 2 are all connected to N3. Relay Config File -------------------- log file /var/log/frr/frr.log debugging ! interface eth0 ip address 10.0.4.2/24 ! interface eth1 ipv6 address 2001::1/64 ! interface aero2 ipv6 address fe80::1:1/10 ipv6 address fd80::1:1/112 ! interface aero3 ipv6 address fe80::2:1/10 ipv6 address fd80::2:1/112 ! ! Static routes ip route 0.0.0.0/0 10.0.4.1 ! ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 1 bgp router-id 10.0.4.2 no bgp default ipv4-unicast neighbor fd80::1:2 remote-as 2 neighbor fd80::1:2 interface aero2 neighbor fd80::2:2 remote-as 3 neighbor fd80::2:2 interface aero3 address-family ipv6 neighbor fd80::1:2 activate neighbor fd80::1:2 next-hop-self neighbor fd80::1:2 default-originate neighbor fd80::1:2 distribute-list blackhole out neighbor fd80::2:2 activate neighbor fd80::2:2 next-hop-self neighbor fd80::2:2 default-originate neighbor fd80::2:2 distribute-list blackhole out ! access-list blackhole deny any exit-address-family ! ipv6 access-list blackhole deny any ! Server1 config file ---------------------- log file /var/log/frr/frr.log debugging interface eth0 ip address 10.0.2.2/24 ! interface aero0 ipv6 address fe80::1:2/10 ipv6 address fd80::1:2/112 ! ! Static routes ! ip route 0.0.0.0/0 10.0.2.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 2 bgp router-id 10.0.2.2 no bgp default ipv4-unicast neighbor fd80::1:1 remote-as 1 neighbor fd80::1:1 interface aero0 neighbor fd80::1:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::1:1 activate neighbor fd80::1:1 next-hop-self neighbor fd80::1:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any ! Server2 config file ---------------------- interface eth0 ip address 10.0.3.2/24 ! interface aero0 ipv6 address fe80::2:2/10 ipv6 address fd80::2:2/112 ! ! Static routes ip route 0.0.0.0/0 10.0.3.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 3 bgp router-id 10.0.3.2 no bgp default ipv4-unicast neighbor fd80::2:1 remote-as 1 neighbor fd80::2:1 interface aero0 neighbor fd80::2:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::2:1 activate neighbor fd80::2:1 next-hop-self neighbor fd80::2:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any N3 config file --------------------- interface eth2 ! interface eth3 ! interface eth4 ! interface eth5 ! interface eth0 ! interface eth1 ! Thank you, Katherine From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org<mailto:frog@lists.frrouting.org> Subject: Re: [FROG] FRR Adding IPv6 Routes On 02/04/2020 22:25, Tran (US), Katherine K wrote: I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga. A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command "ip -6 ro sh". Quagga ---------- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium FRR ----- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium I think, the display of 186 it just because you need the `friendly` names specifying You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below. I suspect this is not why routes are being added and withdrawn. $ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp Not sure what else to say. Can you post some more config? if you type do show ipv6 route <a prefix> what comes out? Is the next hop accessible? I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
Hello, So it looks like the routes are added in bgp. But, when I do "show ipv6 route", the routes still toggle back and forth in being removed and added... Is there anything strange or off about the added bgp routes below? AERO-Relay# show bgp BGP table version is 48939, local router ID is 10.0.4.2, vrf id 0 Default local pref 100, local AS 1 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path
2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
fd80::1:0/112 fe80::1:2 0 0 2 ?
fd80::1:2/128 fe80::1:2 1024 0 2 ?
fd80::2:0/112 fe80::2:2 0 0 3 ?
fd80::2:2/128 fe80::2:2 1024 0 3 ?
Displayed 8 routes and 8 total paths AERO-Relay# show bgp summary IPv6 Unicast Summary: BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 49506 RIB entries 11, using 2112 bytes of memory Peers 2, using 43 KiB of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd fd80::1:2 4 2 18 15 0 0 0 00:11:55 4 fd80::2:2 4 3 18 19 0 0 0 00:11:55 4 Total number of neighbors 2 Regards, Katherine From: frog [mailto:frog-bounces@lists.frrouting.org] On Behalf Of Tran (US), Katherine K Sent: Monday, April 06, 2020 2:54 PM To: Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue? Doesn't look like the bgp session stay up either.... AERO-Relay# show bgp summary IPv6 Unicast Summary: BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 11120 RIB entries 13, using 2496 bytes of memory Peers 2, using 43 KiB of memory Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd fd80::1:2 4 2 8 5 0 0 0 00:01:32 4 fd80::2:2 4 3 8 5 0 0 0 00:01:32 4 The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it. Thanks, Katherine From: Tim Bray [mailto:tim@provu.co.uk] Sent: Monday, April 06, 2020 2:37 PM To: Tran (US), Katherine K <katherine.k.tran@boeing.com<mailto:katherine.k.tran@boeing.com>>; frog@lists.frrouting.org<mailto:frog@lists.frrouting.org> Subject: Re: [FROG] FRR Adding IPv6 Routes And do show bgp summary. Do the bgp sessions stay up? Show long timers? Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500? There is an IPv6 funny try sysctl net.ipv6.route.max_size=2147483647 There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115 (sorry, quick reply) Tim On 06/04/2020 22:19, Tran (US), Katherine K wrote: Hello, The next hop toggles between being accessible and not accessible. I run the "show ipv6 route" command at two different times and the IPv6 routes toggles between being added and removed. Vtysh terminal output of the Relay ----------------------------------------- Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00 B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00 B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00 C>* fd80::1:0/112 is directly connected, eth2, 00:00:21 B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00 B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00 C>* fd80::2:0/112 is directly connected, eth3, 00:00:21 B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00 C * fe80::/10 is directly connected, eth3, 00:00:20 C * fe80::/10 is directly connected, eth2, 00:00:20 C * fe80::/10 is directly connected, eth3, 00:00:21 C>* fe80::/10 is directly connected, eth2, 00:00:21 C * fe80::/64 is directly connected, eth1, 00:00:17 C>* fe80::/64 is directly connected, eth0, 00:00:17 Relay# show ipv6 route Codes: K - kernel route, C - connected, S - static, R - RIPng, O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued route, r - rejected route C>* fd80::1:0/112 is directly connected, eth2, 00:02:13 C>* fd80::2:0/112 is directly connected, eth3, 00:02:13 C * fe80::/10 is directly connected, eth3, 00:02:12 C * fe80::/10 is directly connected, eth2, 00:02:12 C * fe80::/10 is directly connected, eth3, 00:02:13 C>* fe80::/10 is directly connected, eth2, 00:02:13 C * fe80::/64 is directly connected, eth1, 00:02:09 C>* fe80::/64 is directly connected, eth0, 00:02:09 Below are the frr.conf config files for my configuration. The Relay, Server 1 and Server 2 are all connected to N3. Relay Config File -------------------- log file /var/log/frr/frr.log debugging ! interface eth0 ip address 10.0.4.2/24 ! interface eth1 ipv6 address 2001::1/64 ! interface aero2 ipv6 address fe80::1:1/10 ipv6 address fd80::1:1/112 ! interface aero3 ipv6 address fe80::2:1/10 ipv6 address fd80::2:1/112 ! ! Static routes ip route 0.0.0.0/0 10.0.4.1 ! ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 1 bgp router-id 10.0.4.2 no bgp default ipv4-unicast neighbor fd80::1:2 remote-as 2 neighbor fd80::1:2 interface aero2 neighbor fd80::2:2 remote-as 3 neighbor fd80::2:2 interface aero3 address-family ipv6 neighbor fd80::1:2 activate neighbor fd80::1:2 next-hop-self neighbor fd80::1:2 default-originate neighbor fd80::1:2 distribute-list blackhole out neighbor fd80::2:2 activate neighbor fd80::2:2 next-hop-self neighbor fd80::2:2 default-originate neighbor fd80::2:2 distribute-list blackhole out ! access-list blackhole deny any exit-address-family ! ipv6 access-list blackhole deny any ! Server1 config file ---------------------- log file /var/log/frr/frr.log debugging interface eth0 ip address 10.0.2.2/24 ! interface aero0 ipv6 address fe80::1:2/10 ipv6 address fd80::1:2/112 ! ! Static routes ! ip route 0.0.0.0/0 10.0.2.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 2 bgp router-id 10.0.2.2 no bgp default ipv4-unicast neighbor fd80::1:1 remote-as 1 neighbor fd80::1:1 interface aero0 neighbor fd80::1:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::1:1 activate neighbor fd80::1:1 next-hop-self neighbor fd80::1:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any ! Server2 config file ---------------------- interface eth0 ip address 10.0.3.2/24 ! interface aero0 ipv6 address fe80::2:2/10 ipv6 address fd80::2:2/112 ! ! Static routes ip route 0.0.0.0/0 10.0.3.1 ! ! ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 3 bgp router-id 10.0.3.2 no bgp default ipv4-unicast neighbor fd80::2:1 remote-as 1 neighbor fd80::2:1 interface aero0 neighbor fd80::2:1 advertisement-interval 0 address-family ipv6 redistribute connected redistribute static redistribute kernel neighbor fd80::2:1 activate neighbor fd80::2:1 next-hop-self neighbor fd80::2:1 distribute-list 101 out exit-address-family ! ipv6 access-list 101 permit 2001::/16 ipv6 access-list 101 permit fd80::/10 ipv6 access-list 101 deny any N3 config file --------------------- interface eth2 ! interface eth3 ! interface eth4 ! interface eth5 ! interface eth0 ! interface eth1 ! Thank you, Katherine From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org<mailto:frog@lists.frrouting.org> Subject: Re: [FROG] FRR Adding IPv6 Routes On 02/04/2020 22:25, Tran (US), Katherine K wrote: I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga. A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command "ip -6 ro sh". Quagga ---------- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium FRR ----- 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium I think, the display of 186 it just because you need the `friendly` names specifying You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below. I suspect this is not why routes are being added and withdrawn. $ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp Not sure what else to say. Can you post some more config? if you type do show ipv6 route <a prefix> what comes out? Is the next hop accessible? I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
What version of FRR again? Longshot here, but your zebra deamon wouldn't happen to be crashing/restarting ? On 4/8/20 2:02 PM, Tran (US), Katherine K wrote:
Hello,
So it looks like the routes are added in bgp. But, when I do "show ipv6 route", the routes still toggle back and forth in being removed and added...
Is there anything strange or off about the added bgp routes below?
AERO-Relay# show bgp
BGP table version is 48939, local router ID is 10.0.4.2, vrf id 0 Default local pref 100, local AS 1 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
fd80::1:0/112 fe80::1:2 0 0 2 ?
fd80::1:2/128 fe80::1:2 1024 0 2 ?
fd80::2:0/112 fe80::2:2 0 0 3 ?
fd80::2:2/128 fe80::2:2 1024 0 3 ?
Displayed 8 routes and 8 total paths
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 49506 RIB entries 11, using 2112 bytes of memory Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 18 15 0 0 0 00:11:55 4
fd80::2:2 4 3 18 19 0 0 0 00:11:55 4
Total number of neighbors 2
Regards,
Katherine
*From:* frog [mailto:frog-bounces@lists.frrouting.org] *On Behalf Of *Tran (US), Katherine K *Sent:* Monday, April 06, 2020 2:54 PM *To:* Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org *Subject:* Re: [FROG] FRR Adding IPv6 Routes
I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue?
Doesn’t look like the bgp session stay up either….
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0
BGP table version 11120
RIB entries 13, using 2496 bytes of memory
Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 8 5 0 0 0 00:01:32 4
fd80::2:2 4 3 8 5 0 0 0 00:01:32 4
The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it.
Thanks,
Katherine
*From:* Tim Bray [mailto:tim@provu.co.uk] *Sent:* Monday, April 06, 2020 2:37 PM *To:* Tran (US), Katherine K <katherine.k.tran@boeing.com <mailto:katherine.k.tran@boeing.com>>; frog@lists.frrouting.org <mailto:frog@lists.frrouting.org> *Subject:* Re: [FROG] FRR Adding IPv6 Routes
And
do show bgp summary.
Do the bgp sessions stay up? Show long timers?
Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500?
There is an IPv6 funny
try
sysctl net.ipv6.route.max_size=2147483647
There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115
(sorry, quick reply)
Tim
On 06/04/2020 22:19, Tran (US), Katherine K wrote:
Hello,
The next hop toggles between being accessible and not accessible. I run the “show ipv6 route” command at two different times and the IPv6 routes toggles between being added and removed.
Vtysh terminal output of the Relay
-----------------------------------------
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
C * fe80::/10 is directly connected, eth3, 00:00:20
C * fe80::/10 is directly connected, eth2, 00:00:20
C * fe80::/10 is directly connected, eth3, 00:00:21
C>* fe80::/10 is directly connected, eth2, 00:00:21
C * fe80::/64 is directly connected, eth1, 00:00:17
C>* fe80::/64 is directly connected, eth0, 00:00:17
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
C * fe80::/10 is directly connected, eth3, 00:02:12
C * fe80::/10 is directly connected, eth2, 00:02:12
C * fe80::/10 is directly connected, eth3, 00:02:13
C>* fe80::/10 is directly connected, eth2, 00:02:13
C * fe80::/64 is directly connected, eth1, 00:02:09
C>* fe80::/64 is directly connected, eth0, 00:02:09
Below are the frr.conf config files for my configuration.
The Relay, Server 1 and Server 2 are all connected to N3.
Relay Config File
--------------------
log file /var/log/frr/frr.log debugging
!
interface eth0
ip address 10.0.4.2/24
!
interface eth1
ipv6 address 2001::1/64
!
interface aero2
ipv6 address fe80::1:1/10
ipv6 address fd80::1:1/112
!
interface aero3
ipv6 address fe80::2:1/10
ipv6 address fd80::2:1/112
!
! Static routes
ip route 0.0.0.0/0 10.0.4.1
!
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
! access-list blackhole deny any
exit-address-family
!
ipv6 access-list blackhole deny any
!
Server1 config file
----------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.2.2/24
!
interface aero0
ipv6 address fe80::1:2/10
ipv6 address fd80::1:2/112
!
! Static routes
!
ip route 0.0.0.0/0 10.0.2.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 2
bgp router-id 10.0.2.2
no bgp default ipv4-unicast
neighbor fd80::1:1 remote-as 1
neighbor fd80::1:1 interface aero0
neighbor fd80::1:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::1:1 activate
neighbor fd80::1:1 next-hop-self
neighbor fd80::1:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
!
Server2 config file
----------------------
interface eth0
ip address 10.0.3.2/24
!
interface aero0
ipv6 address fe80::2:2/10
ipv6 address fd80::2:2/112
!
! Static routes
ip route 0.0.0.0/0 10.0.3.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 3
bgp router-id 10.0.3.2
no bgp default ipv4-unicast
neighbor fd80::2:1 remote-as 1
neighbor fd80::2:1 interface aero0
neighbor fd80::2:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::2:1 activate
neighbor fd80::2:1 next-hop-self
neighbor fd80::2:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
N3 config file
---------------------
interface eth2
!
interface eth3
!
interface eth4
!
interface eth5
!
interface eth0
!
interface eth1
!
Thank you,
Katherine
*From:* Tim Bray [mailto:tim@provu.co.uk] *Sent:* Thursday, April 02, 2020 3:31 PM *To:* frog@lists.frrouting.org <mailto:frog@lists.frrouting.org> *Subject:* Re: [FROG] FRR Adding IPv6 Routes
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying
You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
I suspect this is not why routes are being added and withdrawn.
$ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp
Not sure what else to say. Can you post some more config?
if you type
do show ipv6 route <a prefix>
what comes out?
Is the next hop accessible?
I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
I compiled from the FRR source code recently within the last few weeks. The zebra daemon doesn't seem to be crashing and restarting, but I am not sure if the zebra daemon is running properly either. What could I do if zebra were the issue? -----Original Message----- From: frog [mailto:frog-bounces@lists.frrouting.org] On Behalf Of Patrick Boutilier Sent: Wednesday, April 08, 2020 10:24 AM To: frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes What version of FRR again? Longshot here, but your zebra deamon wouldn't happen to be crashing/restarting ? On 4/8/20 2:02 PM, Tran (US), Katherine K wrote:
Hello,
So it looks like the routes are added in bgp. But, when I do "show ipv6 route", the routes still toggle back and forth in being removed and added...
Is there anything strange or off about the added bgp routes below?
AERO-Relay# show bgp
BGP table version is 48939, local router ID is 10.0.4.2, vrf id 0 Default local pref 100, local AS 1 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
fd80::1:0/112 fe80::1:2 0 0 2 ?
fd80::1:2/128 fe80::1:2 1024 0 2 ?
fd80::2:0/112 fe80::2:2 0 0 3 ?
fd80::2:2/128 fe80::2:2 1024 0 3 ?
Displayed 8 routes and 8 total paths
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 49506 RIB entries 11, using 2112 bytes of memory Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 18 15 0 0 0 00:11:55 4
fd80::2:2 4 3 18 19 0 0 0 00:11:55 4
Total number of neighbors 2
Regards,
Katherine
*From:* frog [mailto:frog-bounces@lists.frrouting.org] *On Behalf Of *Tran (US), Katherine K *Sent:* Monday, April 06, 2020 2:54 PM *To:* Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org *Subject:* Re: [FROG] FRR Adding IPv6 Routes
I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue?
Doesn't look like the bgp session stay up either..
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0
BGP table version 11120
RIB entries 13, using 2496 bytes of memory
Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 8 5 0 0 0 00:01:32 4
fd80::2:2 4 3 8 5 0 0 0 00:01:32 4
The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it.
Thanks,
Katherine
*From:* Tim Bray [mailto:tim@provu.co.uk] *Sent:* Monday, April 06, 2020 2:37 PM *To:* Tran (US), Katherine K <katherine.k.tran@boeing.com <mailto:katherine.k.tran@boeing.com>>; frog@lists.frrouting.org <mailto:frog@lists.frrouting.org> *Subject:* Re: [FROG] FRR Adding IPv6 Routes
And
do show bgp summary.
Do the bgp sessions stay up? Show long timers?
Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500?
There is an IPv6 funny
try
sysctl net.ipv6.route.max_size=2147483647
There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115
(sorry, quick reply)
Tim
On 06/04/2020 22:19, Tran (US), Katherine K wrote:
Hello,
The next hop toggles between being accessible and not accessible. I run the "show ipv6 route" command at two different times and the IPv6 routes toggles between being added and removed.
Vtysh terminal output of the Relay
-----------------------------------------
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
C * fe80::/10 is directly connected, eth3, 00:00:20
C * fe80::/10 is directly connected, eth2, 00:00:20
C * fe80::/10 is directly connected, eth3, 00:00:21
C>* fe80::/10 is directly connected, eth2, 00:00:21
C * fe80::/64 is directly connected, eth1, 00:00:17
C>* fe80::/64 is directly connected, eth0, 00:00:17
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
C * fe80::/10 is directly connected, eth3, 00:02:12
C * fe80::/10 is directly connected, eth2, 00:02:12
C * fe80::/10 is directly connected, eth3, 00:02:13
C>* fe80::/10 is directly connected, eth2, 00:02:13
C * fe80::/64 is directly connected, eth1, 00:02:09
C>* fe80::/64 is directly connected, eth0, 00:02:09
Below are the frr.conf config files for my configuration.
The Relay, Server 1 and Server 2 are all connected to N3.
Relay Config File
--------------------
log file /var/log/frr/frr.log debugging
!
interface eth0
ip address 10.0.4.2/24
!
interface eth1
ipv6 address 2001::1/64
!
interface aero2
ipv6 address fe80::1:1/10
ipv6 address fd80::1:1/112
!
interface aero3
ipv6 address fe80::2:1/10
ipv6 address fd80::2:1/112
!
! Static routes
ip route 0.0.0.0/0 10.0.4.1
!
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
! access-list blackhole deny any
exit-address-family
!
ipv6 access-list blackhole deny any
!
Server1 config file
----------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.2.2/24
!
interface aero0
ipv6 address fe80::1:2/10
ipv6 address fd80::1:2/112
!
! Static routes
!
ip route 0.0.0.0/0 10.0.2.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 2
bgp router-id 10.0.2.2
no bgp default ipv4-unicast
neighbor fd80::1:1 remote-as 1
neighbor fd80::1:1 interface aero0
neighbor fd80::1:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::1:1 activate
neighbor fd80::1:1 next-hop-self
neighbor fd80::1:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
!
Server2 config file
----------------------
interface eth0
ip address 10.0.3.2/24
!
interface aero0
ipv6 address fe80::2:2/10
ipv6 address fd80::2:2/112
!
! Static routes
ip route 0.0.0.0/0 10.0.3.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 3
bgp router-id 10.0.3.2
no bgp default ipv4-unicast
neighbor fd80::2:1 remote-as 1
neighbor fd80::2:1 interface aero0
neighbor fd80::2:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::2:1 activate
neighbor fd80::2:1 next-hop-self
neighbor fd80::2:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
N3 config file
---------------------
interface eth2
!
interface eth3
!
interface eth4
!
interface eth5
!
interface eth0
!
interface eth1
!
Thank you,
Katherine
*From:* Tim Bray [mailto:tim@provu.co.uk] *Sent:* Thursday, April 02, 2020 3:31 PM *To:* frog@lists.frrouting.org <mailto:frog@lists.frrouting.org> *Subject:* Re: [FROG] FRR Adding IPv6 Routes
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command "ip -6 ro sh".
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying
You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
I suspect this is not why routes are being added and withdrawn.
$ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp
Not sure what else to say. Can you post some more config?
if you type
do show ipv6 route <a prefix>
what comes out?
Is the next hop accessible?
I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
Katherine - Typically when I see bgp installing routes and then re-installing the routes on a different path it means that you have a setup where bgp has a recursive path through itself that changes when a more specific route is added in. This typically is a missconfiguration of some sort. Without seeing your setup it's hard to say exactly where, but typically you will have a redistribute connected of the same prefix in multiple places on the network. donald On Wed, Apr 8, 2020 at 1:03 PM Tran (US), Katherine K <katherine.k.tran@boeing.com> wrote:
Hello,
So it looks like the routes are added in bgp. But, when I do "show ipv6 route", the routes still toggle back and forth in being removed and added...
Is there anything strange or off about the added bgp routes below?
AERO-Relay# show bgp
BGP table version is 48939, local router ID is 10.0.4.2, vrf id 0 Default local pref 100, local AS 1 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
fd80::1:0/112 fe80::1:2 0 0 2 ?
fd80::1:2/128 fe80::1:2 1024 0 2 ?
fd80::2:0/112 fe80::2:2 0 0 3 ?
fd80::2:2/128 fe80::2:2 1024 0 3 ?
Displayed 8 routes and 8 total paths
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 49506 RIB entries 11, using 2112 bytes of memory Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 18 15 0 0 0 00:11:55 4
fd80::2:2 4 3 18 19 0 0 0 00:11:55 4
Total number of neighbors 2
Regards,
Katherine
From: frog [mailto:frog-bounces@lists.frrouting.org] On Behalf Of Tran (US), Katherine K Sent: Monday, April 06, 2020 2:54 PM To: Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue?
Doesn’t look like the bgp session stay up either….
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0
BGP table version 11120
RIB entries 13, using 2496 bytes of memory
Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 8 5 0 0 0 00:01:32 4
fd80::2:2 4 3 8 5 0 0 0 00:01:32 4
The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it.
Thanks,
Katherine
From: Tim Bray [mailto:tim@provu.co.uk] Sent: Monday, April 06, 2020 2:37 PM To: Tran (US), Katherine K <katherine.k.tran@boeing.com>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
And
do show bgp summary.
Do the bgp sessions stay up? Show long timers?
Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500?
There is an IPv6 funny
try
sysctl net.ipv6.route.max_size=2147483647
There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115
(sorry, quick reply)
Tim
On 06/04/2020 22:19, Tran (US), Katherine K wrote:
Hello,
The next hop toggles between being accessible and not accessible. I run the “show ipv6 route” command at two different times and the IPv6 routes toggles between being added and removed.
Vtysh terminal output of the Relay
-----------------------------------------
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
C * fe80::/10 is directly connected, eth3, 00:00:20
C * fe80::/10 is directly connected, eth2, 00:00:20
C * fe80::/10 is directly connected, eth3, 00:00:21
C>* fe80::/10 is directly connected, eth2, 00:00:21
C * fe80::/64 is directly connected, eth1, 00:00:17
C>* fe80::/64 is directly connected, eth0, 00:00:17
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
C * fe80::/10 is directly connected, eth3, 00:02:12
C * fe80::/10 is directly connected, eth2, 00:02:12
C * fe80::/10 is directly connected, eth3, 00:02:13
C>* fe80::/10 is directly connected, eth2, 00:02:13
C * fe80::/64 is directly connected, eth1, 00:02:09
C>* fe80::/64 is directly connected, eth0, 00:02:09
Below are the frr.conf config files for my configuration.
The Relay, Server 1 and Server 2 are all connected to N3.
Relay Config File
--------------------
log file /var/log/frr/frr.log debugging
!
interface eth0
ip address 10.0.4.2/24
!
interface eth1
ipv6 address 2001::1/64
!
interface aero2
ipv6 address fe80::1:1/10
ipv6 address fd80::1:1/112
!
interface aero3
ipv6 address fe80::2:1/10
ipv6 address fd80::2:1/112
!
! Static routes
ip route 0.0.0.0/0 10.0.4.1
!
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
! access-list blackhole deny any
exit-address-family
!
ipv6 access-list blackhole deny any
!
Server1 config file
----------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.2.2/24
!
interface aero0
ipv6 address fe80::1:2/10
ipv6 address fd80::1:2/112
!
! Static routes
!
ip route 0.0.0.0/0 10.0.2.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 2
bgp router-id 10.0.2.2
no bgp default ipv4-unicast
neighbor fd80::1:1 remote-as 1
neighbor fd80::1:1 interface aero0
neighbor fd80::1:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::1:1 activate
neighbor fd80::1:1 next-hop-self
neighbor fd80::1:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
!
Server2 config file
----------------------
interface eth0
ip address 10.0.3.2/24
!
interface aero0
ipv6 address fe80::2:2/10
ipv6 address fd80::2:2/112
!
! Static routes
ip route 0.0.0.0/0 10.0.3.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 3
bgp router-id 10.0.3.2
no bgp default ipv4-unicast
neighbor fd80::2:1 remote-as 1
neighbor fd80::2:1 interface aero0
neighbor fd80::2:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::2:1 activate
neighbor fd80::2:1 next-hop-self
neighbor fd80::2:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
N3 config file
---------------------
interface eth2
!
interface eth3
!
interface eth4
!
interface eth5
!
interface eth0
!
interface eth1
!
Thank you,
Katherine
From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying
You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
I suspect this is not why routes are being added and withdrawn.
$ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp
Not sure what else to say. Can you post some more config?
if you type
do show ipv6 route <a prefix>
what comes out?
Is the next hop accessible?
I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
Yeah, the strange thing is that this configuration worked in Quagga. I am trying to get it working in FRR... I added the static routes to the frr.conf. So, I am wondering if certain config commands in Quagga mean something different in FRR... Thanks -----Original Message----- From: Donald Sharp [mailto:sharpd@cumulusnetworks.com] Sent: Wednesday, April 08, 2020 10:27 AM To: Tran (US), Katherine K <katherine.k.tran@boeing.com> Cc: Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes Katherine - Typically when I see bgp installing routes and then re-installing the routes on a different path it means that you have a setup where bgp has a recursive path through itself that changes when a more specific route is added in. This typically is a missconfiguration of some sort. Without seeing your setup it's hard to say exactly where, but typically you will have a redistribute connected of the same prefix in multiple places on the network. donald On Wed, Apr 8, 2020 at 1:03 PM Tran (US), Katherine K <katherine.k.tran@boeing.com> wrote:
Hello,
So it looks like the routes are added in bgp. But, when I do "show ipv6 route", the routes still toggle back and forth in being removed and added...
Is there anything strange or off about the added bgp routes below?
AERO-Relay# show bgp
BGP table version is 48939, local router ID is 10.0.4.2, vrf id 0 Default local pref 100, local AS 1 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
fd80::1:0/112 fe80::1:2 0 0 2 ?
fd80::1:2/128 fe80::1:2 1024 0 2 ?
fd80::2:0/112 fe80::2:2 0 0 3 ?
fd80::2:2/128 fe80::2:2 1024 0 3 ?
Displayed 8 routes and 8 total paths
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 49506 RIB entries 11, using 2112 bytes of memory Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 18 15 0 0 0 00:11:55 4
fd80::2:2 4 3 18 19 0 0 0 00:11:55 4
Total number of neighbors 2
Regards,
Katherine
From: frog [mailto:frog-bounces@lists.frrouting.org] On Behalf Of Tran (US), Katherine K Sent: Monday, April 06, 2020 2:54 PM To: Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue?
Doesn’t look like the bgp session stay up either….
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0
BGP table version 11120
RIB entries 13, using 2496 bytes of memory
Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 8 5 0 0 0 00:01:32 4
fd80::2:2 4 3 8 5 0 0 0 00:01:32 4
The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it.
Thanks,
Katherine
From: Tim Bray [mailto:tim@provu.co.uk] Sent: Monday, April 06, 2020 2:37 PM To: Tran (US), Katherine K <katherine.k.tran@boeing.com>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
And
do show bgp summary.
Do the bgp sessions stay up? Show long timers?
Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500?
There is an IPv6 funny
try
sysctl net.ipv6.route.max_size=2147483647
There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115
(sorry, quick reply)
Tim
On 06/04/2020 22:19, Tran (US), Katherine K wrote:
Hello,
The next hop toggles between being accessible and not accessible. I run the “show ipv6 route” command at two different times and the IPv6 routes toggles between being added and removed.
Vtysh terminal output of the Relay
-----------------------------------------
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
C * fe80::/10 is directly connected, eth3, 00:00:20
C * fe80::/10 is directly connected, eth2, 00:00:20
C * fe80::/10 is directly connected, eth3, 00:00:21
C>* fe80::/10 is directly connected, eth2, 00:00:21
C * fe80::/64 is directly connected, eth1, 00:00:17
C>* fe80::/64 is directly connected, eth0, 00:00:17
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
C * fe80::/10 is directly connected, eth3, 00:02:12
C * fe80::/10 is directly connected, eth2, 00:02:12
C * fe80::/10 is directly connected, eth3, 00:02:13
C>* fe80::/10 is directly connected, eth2, 00:02:13
C * fe80::/64 is directly connected, eth1, 00:02:09
C>* fe80::/64 is directly connected, eth0, 00:02:09
Below are the frr.conf config files for my configuration.
The Relay, Server 1 and Server 2 are all connected to N3.
Relay Config File
--------------------
log file /var/log/frr/frr.log debugging
!
interface eth0
ip address 10.0.4.2/24
!
interface eth1
ipv6 address 2001::1/64
!
interface aero2
ipv6 address fe80::1:1/10
ipv6 address fd80::1:1/112
!
interface aero3
ipv6 address fe80::2:1/10
ipv6 address fd80::2:1/112
!
! Static routes
ip route 0.0.0.0/0 10.0.4.1
!
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
! access-list blackhole deny any
exit-address-family
!
ipv6 access-list blackhole deny any
!
Server1 config file
----------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.2.2/24
!
interface aero0
ipv6 address fe80::1:2/10
ipv6 address fd80::1:2/112
!
! Static routes
!
ip route 0.0.0.0/0 10.0.2.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 2
bgp router-id 10.0.2.2
no bgp default ipv4-unicast
neighbor fd80::1:1 remote-as 1
neighbor fd80::1:1 interface aero0
neighbor fd80::1:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::1:1 activate
neighbor fd80::1:1 next-hop-self
neighbor fd80::1:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
!
Server2 config file
----------------------
interface eth0
ip address 10.0.3.2/24
!
interface aero0
ipv6 address fe80::2:2/10
ipv6 address fd80::2:2/112
!
! Static routes
ip route 0.0.0.0/0 10.0.3.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 3
bgp router-id 10.0.3.2
no bgp default ipv4-unicast
neighbor fd80::2:1 remote-as 1
neighbor fd80::2:1 interface aero0
neighbor fd80::2:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::2:1 activate
neighbor fd80::2:1 next-hop-self
neighbor fd80::2:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
N3 config file
---------------------
interface eth2
!
interface eth3
!
interface eth4
!
interface eth5
!
interface eth0
!
interface eth1
!
Thank you,
Katherine
From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying
You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
I suspect this is not why routes are being added and withdrawn.
$ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp
Not sure what else to say. Can you post some more config?
if you type
do show ipv6 route <a prefix>
what comes out?
Is the next hop accessible?
I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
Hello, For some reason, the FRR configuration seems to map the "fd80" next hop addresses to "fe80" addresses whereas Quagga does not. Since the next hop address appears as a "fe80" address, it is a link-local address and, therefore, cannot be routed. This is possibly why the routes keep toggling between being added and removed. Is this a known issue? The "fd80" address are used as Unique Local IPv6 Unicast Addresses (IETF RFC 4193), which can be routed. How would I achieve this in FRR? Below is the Quagga vtysh terminal output, FRR vtysh terminal output, and frr.conf. *Note: The Quagga.conf file is the same as the frr.conf except the static route line is removed. Instead, the static routes are added by the kernel using "ip route add". -------------------------------------------------------------------------------------------- Quagga vtysh terminal -------------------------------------------------------------------------------------------- AERO-Relay# show bgp BGP table version is 0, local router ID is 10.0.4.2 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, R Removed Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 2001:db8:1:1::/64 fd80::1:2 1024 0 2 ? *> 2001:db8:1:2::/64 fd80::2:2 1024 0 3 ? *> 2001:db8:1:3::/64 fd80::1:2 1024 0 2 ? *> 2001:db8:1:4::/64 fd80::2:2 1024 0 3 ? *> fd80::1:0/112 fd80::1:2 1 0 2 ? *> fd80::1:2/128 fd80::1:2 1024 0 2 ? *> fd80::2:0/112 fd80::2:2 1 0 3 ? *> fd80::2:2/128 fd80::2:2 1024 0 3 ? Total number of prefixes 8 -------------------------------------------------------------------------------------------- FRR vtysh terminal -------------------------------------------------------------------------------------------- AERO-Relay# show bgp BGP table version is 10896, local router ID is 10.0.4.2, vrf id 0 Default local pref 100, local AS 1 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 2001:db8:1:1::/64 fe80::1:2 1024 0 2 ? *> 2001:db8:1:2::/64 fe80::2:2 1024 0 3 ? *> 2001:db8:1:3::/64 fe80::1:2 1024 0 2 ? *> 2001:db8:1:4::/64 fe80::2:2 1024 0 3 ? *> fd80::1:0/112 fe80::1:2 0 0 2 ? *> fd80::1:2/128 fe80::1:2 1024 0 2 ? *> fd80::2:0/112 fe80::2:2 0 0 3 ? *> fd80::2:2/128 fe80::2:2 1024 0 3 ? Displayed 8 routes and 8 total paths -------------------------------------------------------------------------------------------- frr.conf -------------------------------------------------------------------------------------------- log file /var/log/frr/frr.log debugging interface eth0 ip address 10.0.4.2/24 interface eth1 ipv6 address 2001::1/64 interface aero2 ipv6 address fd80::1:1/112 interface aero3 ipv6 address fd80::2:1/112 ! Static routes ip route 0.0.0.0/0 10.0.4.1 ! BGP configuration ! ! You should configure the AS number below, ! along with this router's peers. ! router bgp 1 bgp router-id 10.0.4.2 no bgp default ipv4-unicast neighbor fd80::1:2 remote-as 2 neighbor fd80::1:2 interface aero2 neighbor fd80::2:2 remote-as 3 neighbor fd80::2:2 interface aero3 address-family ipv6 neighbor fd80::1:2 activate neighbor fd80::1:2 next-hop-self neighbor fd80::1:2 default-originate neighbor fd80::1:2 distribute-list blackhole out neighbor fd80::2:2 activate neighbor fd80::2:2 next-hop-self neighbor fd80::2:2 default-originate neighbor fd80::2:2 distribute-list blackhole out exit-address-family ipv6 access-list blackhole deny any Thank you, Katherine -----Original Message----- From: Tran (US), Katherine K Sent: Wednesday, April 08, 2020 1:09 PM To: 'Donald Sharp' <sharpd@cumulusnetworks.com> Cc: Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org Subject: RE: [FROG] FRR Adding IPv6 Routes Yeah, the strange thing is that this configuration worked in Quagga. I am trying to get it working in FRR... I added the static routes to the frr.conf. So, I am wondering if certain config commands in Quagga mean something different in FRR... Thanks -----Original Message----- From: Donald Sharp [mailto:sharpd@cumulusnetworks.com] Sent: Wednesday, April 08, 2020 10:27 AM To: Tran (US), Katherine K <katherine.k.tran@boeing.com> Cc: Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes Katherine - Typically when I see bgp installing routes and then re-installing the routes on a different path it means that you have a setup where bgp has a recursive path through itself that changes when a more specific route is added in. This typically is a missconfiguration of some sort. Without seeing your setup it's hard to say exactly where, but typically you will have a redistribute connected of the same prefix in multiple places on the network. donald On Wed, Apr 8, 2020 at 1:03 PM Tran (US), Katherine K <katherine.k.tran@boeing.com> wrote:
Hello,
So it looks like the routes are added in bgp. But, when I do "show ipv6 route", the routes still toggle back and forth in being removed and added...
Is there anything strange or off about the added bgp routes below?
AERO-Relay# show bgp
BGP table version is 48939, local router ID is 10.0.4.2, vrf id 0 Default local pref 100, local AS 1 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
fd80::1:0/112 fe80::1:2 0 0 2 ?
fd80::1:2/128 fe80::1:2 1024 0 2 ?
fd80::2:0/112 fe80::2:2 0 0 3 ?
fd80::2:2/128 fe80::2:2 1024 0 3 ?
Displayed 8 routes and 8 total paths
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table version 49506 RIB entries 11, using 2112 bytes of memory Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 18 15 0 0 0 00:11:55 4
fd80::2:2 4 3 18 19 0 0 0 00:11:55 4
Total number of neighbors 2
Regards,
Katherine
From: frog [mailto:frog-bounces@lists.frrouting.org] On Behalf Of Tran (US), Katherine K Sent: Monday, April 06, 2020 2:54 PM To: Tim Bray <tim@provu.co.uk>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue?
Doesn’t look like the bgp session stay up either….
AERO-Relay# show bgp summary
IPv6 Unicast Summary:
BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0
BGP table version 11120
RIB entries 13, using 2496 bytes of memory
Peers 2, using 43 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
fd80::1:2 4 2 8 5 0 0 0 00:01:32 4
fd80::2:2 4 3 8 5 0 0 0 00:01:32 4
The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it.
Thanks,
Katherine
From: Tim Bray [mailto:tim@provu.co.uk] Sent: Monday, April 06, 2020 2:37 PM To: Tran (US), Katherine K <katherine.k.tran@boeing.com>; frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
And
do show bgp summary.
Do the bgp sessions stay up? Show long timers?
Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500?
There is an IPv6 funny
try
sysctl net.ipv6.route.max_size=2147483647
There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115
(sorry, quick reply)
Tim
On 06/04/2020 22:19, Tran (US), Katherine K wrote:
Hello,
The next hop toggles between being accessible and not accessible. I run the “show ipv6 route” command at two different times and the IPv6 routes toggles between being added and removed.
Vtysh terminal output of the Relay
-----------------------------------------
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
C * fe80::/10 is directly connected, eth3, 00:00:20
C * fe80::/10 is directly connected, eth2, 00:00:20
C * fe80::/10 is directly connected, eth3, 00:00:21
C>* fe80::/10 is directly connected, eth2, 00:00:21
C * fe80::/64 is directly connected, eth1, 00:00:17
C>* fe80::/64 is directly connected, eth0, 00:00:17
Relay# show ipv6 route
Codes: K - kernel route, C - connected, S - static, R - RIPng,
O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
C * fe80::/10 is directly connected, eth3, 00:02:12
C * fe80::/10 is directly connected, eth2, 00:02:12
C * fe80::/10 is directly connected, eth3, 00:02:13
C>* fe80::/10 is directly connected, eth2, 00:02:13
C * fe80::/64 is directly connected, eth1, 00:02:09
C>* fe80::/64 is directly connected, eth0, 00:02:09
Below are the frr.conf config files for my configuration.
The Relay, Server 1 and Server 2 are all connected to N3.
Relay Config File
--------------------
log file /var/log/frr/frr.log debugging
!
interface eth0
ip address 10.0.4.2/24
!
interface eth1
ipv6 address 2001::1/64
!
interface aero2
ipv6 address fe80::1:1/10
ipv6 address fd80::1:1/112
!
interface aero3
ipv6 address fe80::2:1/10
ipv6 address fd80::2:1/112
!
! Static routes
ip route 0.0.0.0/0 10.0.4.1
!
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
! access-list blackhole deny any
exit-address-family
!
ipv6 access-list blackhole deny any
!
Server1 config file
----------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.2.2/24
!
interface aero0
ipv6 address fe80::1:2/10
ipv6 address fd80::1:2/112
!
! Static routes
!
ip route 0.0.0.0/0 10.0.2.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 2
bgp router-id 10.0.2.2
no bgp default ipv4-unicast
neighbor fd80::1:1 remote-as 1
neighbor fd80::1:1 interface aero0
neighbor fd80::1:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::1:1 activate
neighbor fd80::1:1 next-hop-self
neighbor fd80::1:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
!
Server2 config file
----------------------
interface eth0
ip address 10.0.3.2/24
!
interface aero0
ipv6 address fe80::2:2/10
ipv6 address fd80::2:2/112
!
! Static routes
ip route 0.0.0.0/0 10.0.3.1
!
!
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 3
bgp router-id 10.0.3.2
no bgp default ipv4-unicast
neighbor fd80::2:1 remote-as 1
neighbor fd80::2:1 interface aero0
neighbor fd80::2:1 advertisement-interval 0
address-family ipv6
redistribute connected
redistribute static
redistribute kernel
neighbor fd80::2:1 activate
neighbor fd80::2:1 next-hop-self
neighbor fd80::2:1 distribute-list 101 out
exit-address-family
!
ipv6 access-list 101 permit 2001::/16
ipv6 access-list 101 permit fd80::/10
ipv6 access-list 101 deny any
N3 config file
---------------------
interface eth2
!
interface eth3
!
interface eth4
!
interface eth5
!
interface eth0
!
interface eth1
!
Thank you,
Katherine
From: Tim Bray [mailto:tim@provu.co.uk] Sent: Thursday, April 02, 2020 3:31 PM To: frog@lists.frrouting.org Subject: Re: [FROG] FRR Adding IPv6 Routes
On 02/04/2020 22:25, Tran (US), Katherine K wrote:
I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
Quagga
----------
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref medium
FRR
-----
2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref medium
I think, the display of 186 it just because you need the `friendly` names specifying
You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
I suspect this is not why routes are being added and withdrawn.
$ more /etc/iproute2/rt_protos # # Reserved protocols. # 0 unspec 1 redirect 2 kernel 3 boot 4 static 8 gated 9 ra 10 mrt 11 zebra 12 bird 13 dnrouted 14 xorp 15 ntk 16 dhcp 42 babel 186 bgp 187 isis 188 ospf 189 rip 192 eigrp
Not sure what else to say. Can you post some more config?
if you type
do show ipv6 route <a prefix>
what comes out?
Is the next hop accessible?
I'd keep trying, because FRR fixes so many IPv6 problems you don't know were problems until you switched to FRR :)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
participants (5)
-
Chriztoffer Hansen -
Donald Sharp -
Patrick Boutilier -
Tim Bray -
Tran (US), Katherine K