[FROG] FRR Adding IPv6 Routes
Tran (US), Katherine K
katherine.k.tran at boeing.com
Thu Apr 9 18:04:05 EDT 2020
Hello,
For some reason, the FRR configuration seems to map the "fd80" next hop addresses to "fe80" addresses whereas Quagga does not. Since the next hop address appears as a "fe80" address, it is a link-local address and, therefore, cannot be routed. This is possibly why the routes keep toggling between being added and removed. Is this a known issue?
The "fd80" address are used as Unique Local IPv6 Unicast Addresses (IETF RFC 4193), which can be routed. How would I achieve this in FRR?
Below is the Quagga vtysh terminal output, FRR vtysh terminal output, and frr.conf.
*Note: The Quagga.conf file is the same as the frr.conf except the static route line is removed. Instead, the static routes are added by the kernel using "ip route add".
--------------------------------------------------------------------------------------------
Quagga vtysh terminal
--------------------------------------------------------------------------------------------
AERO-Relay# show bgp
BGP table version is 0, local router ID is 10.0.4.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 2001:db8:1:1::/64 fd80::1:2 1024 0 2 ?
*> 2001:db8:1:2::/64 fd80::2:2 1024 0 3 ?
*> 2001:db8:1:3::/64 fd80::1:2 1024 0 2 ?
*> 2001:db8:1:4::/64 fd80::2:2 1024 0 3 ?
*> fd80::1:0/112 fd80::1:2 1 0 2 ?
*> fd80::1:2/128 fd80::1:2 1024 0 2 ?
*> fd80::2:0/112 fd80::2:2 1 0 3 ?
*> fd80::2:2/128 fd80::2:2 1024 0 3 ?
Total number of prefixes 8
--------------------------------------------------------------------------------------------
FRR vtysh terminal
--------------------------------------------------------------------------------------------
AERO-Relay# show bgp
BGP table version is 10896, local router ID is 10.0.4.2, vrf id 0
Default local pref 100, local AS 1
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
*> 2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
*> 2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
*> 2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
*> fd80::1:0/112 fe80::1:2 0 0 2 ?
*> fd80::1:2/128 fe80::1:2 1024 0 2 ?
*> fd80::2:0/112 fe80::2:2 0 0 3 ?
*> fd80::2:2/128 fe80::2:2 1024 0 3 ?
Displayed 8 routes and 8 total paths
--------------------------------------------------------------------------------------------
frr.conf
--------------------------------------------------------------------------------------------
log file /var/log/frr/frr.log debugging
interface eth0
ip address 10.0.4.2/24
interface eth1
ipv6 address 2001::1/64
interface aero2
ipv6 address fd80::1:1/112
interface aero3
ipv6 address fd80::2:1/112
! Static routes
ip route 0.0.0.0/0 10.0.4.1
! BGP configuration
!
! You should configure the AS number below,
! along with this router's peers.
!
router bgp 1
bgp router-id 10.0.4.2
no bgp default ipv4-unicast
neighbor fd80::1:2 remote-as 2
neighbor fd80::1:2 interface aero2
neighbor fd80::2:2 remote-as 3
neighbor fd80::2:2 interface aero3
address-family ipv6
neighbor fd80::1:2 activate
neighbor fd80::1:2 next-hop-self
neighbor fd80::1:2 default-originate
neighbor fd80::1:2 distribute-list blackhole out
neighbor fd80::2:2 activate
neighbor fd80::2:2 next-hop-self
neighbor fd80::2:2 default-originate
neighbor fd80::2:2 distribute-list blackhole out
exit-address-family
ipv6 access-list blackhole deny any
Thank you,
Katherine
-----Original Message-----
From: Tran (US), Katherine K
Sent: Wednesday, April 08, 2020 1:09 PM
To: 'Donald Sharp' <sharpd at cumulusnetworks.com>
Cc: Tim Bray <tim at provu.co.uk>; frog at lists.frrouting.org
Subject: RE: [FROG] FRR Adding IPv6 Routes
Yeah, the strange thing is that this configuration worked in Quagga. I am trying to get it working in FRR...
I added the static routes to the frr.conf. So, I am wondering if certain config commands in Quagga mean something different in FRR...
Thanks
-----Original Message-----
From: Donald Sharp [mailto:sharpd at cumulusnetworks.com]
Sent: Wednesday, April 08, 2020 10:27 AM
To: Tran (US), Katherine K <katherine.k.tran at boeing.com>
Cc: Tim Bray <tim at provu.co.uk>; frog at lists.frrouting.org
Subject: Re: [FROG] FRR Adding IPv6 Routes
Katherine -
Typically when I see bgp installing routes and then re-installing the routes on a different path it means that you have a setup where bgp has a recursive path through itself that changes when a more specific
route is added in. This typically is a missconfiguration of some
sort. Without seeing your setup it's hard to say exactly where, but
typically you will have a redistribute connected of the same prefix in multiple places on the network.
donald
On Wed, Apr 8, 2020 at 1:03 PM Tran (US), Katherine K <katherine.k.tran at boeing.com> wrote:
>
> Hello,
>
>
>
> So it looks like the routes are added in bgp. But, when I do "show ipv6 route", the routes still toggle back and forth in being removed and added...
>
>
>
> Is there anything strange or off about the added bgp routes below?
>
>
>
> AERO-Relay# show bgp
>
> BGP table version is 48939, local router ID is 10.0.4.2, vrf id 0
> Default local pref 100, local AS 1 Status codes: s suppressed, d
> damped, h history, * valid, > best, = multipath,
>
> i internal, r RIB-failure, S Stale, R Removed Nexthop
> codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i -
> IGP, e - EGP, ? - incomplete
>
>
>
> Network Next Hop Metric LocPrf Weight Path
>
> > 2001:db8:1:1::/64 fe80::1:2 1024 0 2 ?
>
> > 2001:db8:1:2::/64 fe80::2:2 1024 0 3 ?
>
> > 2001:db8:1:3::/64 fe80::1:2 1024 0 2 ?
>
> > 2001:db8:1:4::/64 fe80::2:2 1024 0 3 ?
>
> > fd80::1:0/112 fe80::1:2 0 0 2 ?
>
> > fd80::1:2/128 fe80::1:2 1024 0 2 ?
>
> > fd80::2:0/112 fe80::2:2 0 0 3 ?
>
> > fd80::2:2/128 fe80::2:2 1024 0 3 ?
>
>
>
> Displayed 8 routes and 8 total paths
>
> AERO-Relay# show bgp summary
>
>
>
> IPv6 Unicast Summary:
>
> BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0 BGP table
> version 49506 RIB entries 11, using 2112 bytes of memory Peers 2,
> using 43 KiB of memory
>
>
>
> Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
>
> fd80::1:2 4 2 18 15 0 0 0 00:11:55 4
>
> fd80::2:2 4 3 18 19 0 0 0 00:11:55 4
>
>
>
> Total number of neighbors 2
>
>
>
> Regards,
>
> Katherine
>
>
>
>
>
> From: frog [mailto:frog-bounces at lists.frrouting.org] On Behalf Of Tran
> (US), Katherine K
> Sent: Monday, April 06, 2020 2:54 PM
> To: Tim Bray <tim at provu.co.uk>; frog at lists.frrouting.org
> Subject: Re: [FROG] FRR Adding IPv6 Routes
>
>
>
> I am running in Ubuntu 18.04 with Linux kernel version 4.15.0-91-generic. Looks like the MTU is 1400 for each interface aero2 and aero3. Would that be causing the issue?
>
>
>
> Doesn’t look like the bgp session stay up either….
>
>
>
> AERO-Relay# show bgp summary
>
>
>
> IPv6 Unicast Summary:
>
> BGP router identifier 10.0.4.2, local AS number 1 vrf-id 0
>
> BGP table version 11120
>
> RIB entries 13, using 2496 bytes of memory
>
> Peers 2, using 43 KiB of memory
>
>
>
> Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
>
> fd80::1:2 4 2 8 5 0 0 0 00:01:32 4
>
> fd80::2:2 4 3 8 5 0 0 0 00:01:32 4
>
>
>
> The net.ipv6.route.max_size is currently set to the default value of 4096. I will see if I can change it.
>
>
>
> Thanks,
>
> Katherine
>
>
>
> From: Tim Bray [mailto:tim at provu.co.uk]
> Sent: Monday, April 06, 2020 2:37 PM
> To: Tran (US), Katherine K <katherine.k.tran at boeing.com>;
> frog at lists.frrouting.org
> Subject: Re: [FROG] FRR Adding IPv6 Routes
>
>
>
>
>
> And
>
>
>
> do show bgp summary.
>
>
>
> Do the bgp sessions stay up? Show long timers?
>
>
>
>
>
> Another very long shot question - which linux kernel version? And do you have any interfaces with MTU different to 1500?
>
> There is an IPv6 funny
>
> try
>
> sysctl net.ipv6.route.max_size=2147483647
>
> There isn't really a route cache for IPv6 in modern linux. But there is a bug which causes problems if you have different MTU. some details here.
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=861115
>
> (sorry, quick reply)
>
>
>
> Tim
>
>
>
>
>
>
>
> On 06/04/2020 22:19, Tran (US), Katherine K wrote:
>
> Hello,
>
>
>
> The next hop toggles between being accessible and not accessible. I run the “show ipv6 route” command at two different times and the IPv6 routes toggles between being added and removed.
>
>
>
> Vtysh terminal output of the Relay
>
> -----------------------------------------
>
> Relay# show ipv6 route
>
> Codes: K - kernel route, C - connected, S - static, R - RIPng,
>
> O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
>
> v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
>
> f - OpenFabric,
>
> > - selected route, * - FIB route, q - queued route, r -
> rejected route
>
>
>
> B>* 2001:db8:1:1::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
>
> B>* 2001:db8:1:2::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
>
> B>* 2001:db8:1:3::/64 [20/1024] via fe80::1:2, eth2, 00:00:00
>
> B>* 2001:db8:1:4::/64 [20/1024] via fe80::2:2, eth3, 00:00:00
>
> B fd80::1:0/112 [20/0] via fe80::1:2, eth2, 00:00:00
>
> C>* fd80::1:0/112 is directly connected, eth2, 00:00:21
>
> B>* fd80::1:2/128 [20/1024] via fe80::1:2, eth2, 00:00:00
>
> B fd80::2:0/112 [20/0] via fe80::2:2, eth3, 00:00:00
>
> C>* fd80::2:0/112 is directly connected, eth3, 00:00:21
>
> B>* fd80::2:2/128 [20/1024] via fe80::2:2, eth3, 00:00:00
>
> C * fe80::/10 is directly connected, eth3, 00:00:20
>
> C * fe80::/10 is directly connected, eth2, 00:00:20
>
> C * fe80::/10 is directly connected, eth3, 00:00:21
>
> C>* fe80::/10 is directly connected, eth2, 00:00:21
>
> C * fe80::/64 is directly connected, eth1, 00:00:17
>
> C>* fe80::/64 is directly connected, eth0, 00:00:17
>
>
>
> Relay# show ipv6 route
>
> Codes: K - kernel route, C - connected, S - static, R - RIPng,
>
> O - OSPFv3, I - IS-IS, B - BGP, N - NHRP, T - Table,
>
> v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR,
>
> f - OpenFabric,
>
> > - selected route, * - FIB route, q - queued route, r -
> rejected route
>
>
>
> C>* fd80::1:0/112 is directly connected, eth2, 00:02:13
>
> C>* fd80::2:0/112 is directly connected, eth3, 00:02:13
>
> C * fe80::/10 is directly connected, eth3, 00:02:12
>
> C * fe80::/10 is directly connected, eth2, 00:02:12
>
> C * fe80::/10 is directly connected, eth3, 00:02:13
>
> C>* fe80::/10 is directly connected, eth2, 00:02:13
>
> C * fe80::/64 is directly connected, eth1, 00:02:09
>
> C>* fe80::/64 is directly connected, eth0, 00:02:09
>
>
>
>
>
> Below are the frr.conf config files for my configuration.
>
> The Relay, Server 1 and Server 2 are all connected to N3.
>
>
>
> Relay Config File
>
> --------------------
>
> log file /var/log/frr/frr.log debugging
>
> !
>
> interface eth0
>
> ip address 10.0.4.2/24
>
>
>
> !
>
> interface eth1
>
>
>
> ipv6 address 2001::1/64
>
> !
>
> interface aero2
>
>
>
> ipv6 address fe80::1:1/10
>
> ipv6 address fd80::1:1/112
>
> !
>
> interface aero3
>
>
>
> ipv6 address fe80::2:1/10
>
> ipv6 address fd80::2:1/112
>
> !
>
> ! Static routes
>
> ip route 0.0.0.0/0 10.0.4.1
>
> !
>
> !
>
> !
>
> ! BGP configuration
>
> !
>
> ! You should configure the AS number below,
>
> ! along with this router's peers.
>
> !
>
> router bgp 1
>
> bgp router-id 10.0.4.2
>
> no bgp default ipv4-unicast
>
> neighbor fd80::1:2 remote-as 2
>
> neighbor fd80::1:2 interface aero2
>
> neighbor fd80::2:2 remote-as 3
>
> neighbor fd80::2:2 interface aero3
>
> address-family ipv6
>
> neighbor fd80::1:2 activate
>
> neighbor fd80::1:2 next-hop-self
>
> neighbor fd80::1:2 default-originate
>
> neighbor fd80::1:2 distribute-list blackhole out
>
> neighbor fd80::2:2 activate
>
> neighbor fd80::2:2 next-hop-self
>
> neighbor fd80::2:2 default-originate
>
> neighbor fd80::2:2 distribute-list blackhole out
>
> ! access-list blackhole deny any
>
> exit-address-family
>
> !
>
> ipv6 access-list blackhole deny any
>
> !
>
>
>
> Server1 config file
>
> ----------------------
>
> log file /var/log/frr/frr.log debugging
>
> interface eth0
>
> ip address 10.0.2.2/24
>
> !
>
> interface aero0
>
> ipv6 address fe80::1:2/10
>
> ipv6 address fd80::1:2/112
>
> !
>
> ! Static routes
>
> !
>
> ip route 0.0.0.0/0 10.0.2.1
>
> !
>
> !
>
> ! BGP configuration
>
> !
>
> ! You should configure the AS number below,
>
> ! along with this router's peers.
>
> !
>
> router bgp 2
>
> bgp router-id 10.0.2.2
>
> no bgp default ipv4-unicast
>
> neighbor fd80::1:1 remote-as 1
>
> neighbor fd80::1:1 interface aero0
>
> neighbor fd80::1:1 advertisement-interval 0
>
> address-family ipv6
>
> redistribute connected
>
> redistribute static
>
> redistribute kernel
>
> neighbor fd80::1:1 activate
>
> neighbor fd80::1:1 next-hop-self
>
> neighbor fd80::1:1 distribute-list 101 out
>
> exit-address-family
>
> !
>
> ipv6 access-list 101 permit 2001::/16
>
> ipv6 access-list 101 permit fd80::/10
>
> ipv6 access-list 101 deny any
>
> !
>
>
>
> Server2 config file
>
> ----------------------
>
> interface eth0
>
> ip address 10.0.3.2/24
>
> !
>
> interface aero0
>
> ipv6 address fe80::2:2/10
>
> ipv6 address fd80::2:2/112
>
> !
>
> ! Static routes
>
> ip route 0.0.0.0/0 10.0.3.1
>
> !
>
> !
>
> ! BGP configuration
>
> !
>
> ! You should configure the AS number below,
>
> ! along with this router's peers.
>
> !
>
> router bgp 3
>
> bgp router-id 10.0.3.2
>
> no bgp default ipv4-unicast
>
> neighbor fd80::2:1 remote-as 1
>
> neighbor fd80::2:1 interface aero0
>
> neighbor fd80::2:1 advertisement-interval 0
>
> address-family ipv6
>
> redistribute connected
>
> redistribute static
>
> redistribute kernel
>
> neighbor fd80::2:1 activate
>
> neighbor fd80::2:1 next-hop-self
>
> neighbor fd80::2:1 distribute-list 101 out
>
> exit-address-family
>
> !
>
> ipv6 access-list 101 permit 2001::/16
>
> ipv6 access-list 101 permit fd80::/10
>
> ipv6 access-list 101 deny any
>
>
>
> N3 config file
>
> ---------------------
>
> interface eth2
>
> !
>
> interface eth3
>
> !
>
> interface eth4
>
> !
>
> interface eth5
>
> !
>
> interface eth0
>
> !
>
> interface eth1
>
> !
>
>
>
> Thank you,
>
> Katherine
>
>
>
>
>
> From: Tim Bray [mailto:tim at provu.co.uk]
> Sent: Thursday, April 02, 2020 3:31 PM
> To: frog at lists.frrouting.org
> Subject: Re: [FROG] FRR Adding IPv6 Routes
>
>
>
> On 02/04/2020 22:25, Tran (US), Katherine K wrote:
>
>
>
> I am trying to switch from Quagga to FRR, but I am having issues adding distributed IPv6 routes. Although the IPv6 routes get added, they keep toggling between being added and removed. This did not occur while using Quagga.
>
>
>
> A difference between the added IPv6 routes of Quagga and FRR was that (1) the Quagga routes are by protocol zebra and (2) the FRR routes are by protocol 186, which I deduced to be bgpd. This can be seen by initiating the command “ip -6 ro sh”.
>
>
>
> Quagga
>
> ----------
>
> 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto zebra metric 1024 pref
> medium
>
>
>
> FRR
>
> -----
>
> 2001:XXXX:X:X::/64 via fe80::1:2 dev eth0 proto 186 metric 20 pref
> medium
>
>
>
>
>
> I think, the display of 186 it just because you need the `friendly`
> names specifying
>
> You could edit etc/iproute2/rt_protos and add the missing numbers and the output would look better. See below.
>
> I suspect this is not why routes are being added and withdrawn.
>
>
>
> $ more /etc/iproute2/rt_protos
> #
> # Reserved protocols.
> #
> 0 unspec
> 1 redirect
> 2 kernel
> 3 boot
> 4 static
> 8 gated
> 9 ra
> 10 mrt
> 11 zebra
> 12 bird
> 13 dnrouted
> 14 xorp
> 15 ntk
> 16 dhcp
> 42 babel
> 186 bgp
> 187 isis
> 188 ospf
> 189 rip
> 192 eigrp
>
>
>
> Not sure what else to say. Can you post some more config?
>
> if you type
>
> do show ipv6 route <a prefix>
>
> what comes out?
>
> Is the next hop accessible?
>
>
>
> I'd keep trying, because FRR fixes so many IPv6 problems you don't
> know were problems until you switched to FRR :)
>
>
>
>
>
>
>
> _______________________________________________
> frog mailing list
> frog at lists.frrouting.org
> https://lists.frrouting.org/listinfo/frog
More information about the frog
mailing list