FRR (5.0) on latest pfSense 2.4.4p1 doesn't insert BGP routes into the kernel
The subject basically says it all..... I have two sites connected over an IPSEC tunnel. Each site has its own internal routes managed by OSPF, and I am trying to exchange them by BGP. * We know the traffic is being passed over the IPSEC tunnel, and we can see that each side does get the other's BGP routes. We did have to install directives to allow multihop and disable-connection-checks.... * We do not see the routes being installed into the local kernel routing tables but for the life us, we can't understand why. We see the routes come across in the BGP debug info, but it never makes it into the FIB. Here's our BGPD.conf albeit for one side -- the other just has the AS numbers changed. # BGP Config router bgp 3000000 bgp router-id 10.0.0.5 redistribute connected redistribute static redistribute kernel redistribute ospf # BGP Neighbors neighbor 192.168.101.39 remote-as 2510000 neighbor 192.168.101.39 description Aaron Martin neighbor 192.168.101.39 update-source 10.0.0.5 address-family ipv4 unicast neighbor 192.168.101.39 activate neighbor 192.168.101.39 disable-connected-checks no neighbor 192.168.101.39 send-community neighbor 192.168.101.39 addpath-tx-bestpath-per-AS neighbor 192.168.101.39 allowas-in exit-address-family And here's what we see BGP table version is 208559, local router ID is 10.0.0.5, vrf id 0 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path 0.0.0.0 192.168.101.39 0 2510000 3000000 ? 192.168.101.39 0 0 2510000 ? *> 50.247.114.30 0 32768 ? 10.0.0.0 192.168.101.39 0 2510000 ? 10.0.0.0/16 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.1.6.0/24 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.10.10.1/32 192.168.101.39 1 0 2510000 ? 10.147.20.0/24 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 50.247.114.16/28 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 50.247.114.18/32 192.168.101.39 0 0 2510000 ? 64.62.134.130/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 68.115.209.232/29 192.168.101.39 1 0 2510000 ? 68.115.209.237/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 72.52.104.74/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 172.16.0.0 192.168.101.39 0 2510000 ? 172.16.184.0/24 192.168.101.39 0 0 2510000 ? 172.16.231.0/24 192.168.101.39 1 0 2510000 ? 172.16.232.0/24 192.168.101.39 0 0 2510000 ? 172.16.238.0/24 192.168.101.39 1 0 2510000 ? 172.17.0.0 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 172.21.0.0 192.168.101.39 0 2510000 ? 192.168.1.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.101.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.106.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.108.0 192.168.101.39 0 2510000 ? 192.168.121.0 192.168.101.39 0 2510000 ? 192.168.128.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.131.0 192.168.101.39 0 2510000 ? 192.168.132.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.148.0 192.168.101.39 1 0 2510000 ? 192.168.150.0 192.168.101.39 0 2510000 ? 192.168.228.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 192.168.229.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 196.101.2.0 192.168.101.39 0 2510000 ? 209.51.161.14/32 192.168.101.39 0 0 2510000 ? Displayed 33 routes and 50 total paths Everything's there, but it never makes it into the kernel. I'm sure we've done something wrong, because I tried a different BGP-based router at the other end, and I see the connection, but again, the routes don't seem to make it into the kernel -- clearly I've broken something basic :-)
What's your ip next hop state? ---------- On December 31, 2018 8:02:47 AM "John Antypas" <jantypas@busygeeks.com> wrote:
The subject basically says it all..... I have two sites connected over an IPSEC tunnel. Each site has its own internal routes managed by OSPF, and I am trying to exchange them by BGP.
* We know the traffic is being passed over the IPSEC tunnel, and we can see that each side does get the other's BGP routes. We did have to install directives to allow multihop and disable-connection-checks.... * We do not see the routes being installed into the local kernel routing tables but for the life us, we can't understand why. We see the routes come across in the BGP debug info, but it never makes it into the FIB.
Here's our BGPD.conf albeit for one side -- the other just has the AS numbers changed.
# BGP Config router bgp 3000000 bgp router-id 10.0.0.5 redistribute connected redistribute static redistribute kernel redistribute ospf
# BGP Neighbors neighbor 192.168.101.39 remote-as 2510000 neighbor 192.168.101.39 description Aaron Martin neighbor 192.168.101.39 update-source 10.0.0.5 address-family ipv4 unicast neighbor 192.168.101.39 activate neighbor 192.168.101.39 disable-connected-checks no neighbor 192.168.101.39 send-community neighbor 192.168.101.39 addpath-tx-bestpath-per-AS neighbor 192.168.101.39 allowas-in exit-address-family
And here's what we see
BGP table version is 208559, local router ID is 10.0.0.5, vrf id 0 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path 0.0.0.0 192.168.101.39 0 2510000 3000000 ? 192.168.101.39 0 0 2510000 ? *> 50.247.114.30 0 32768 ? 10.0.0.0 192.168.101.39 0 2510000 ? 10.0.0.0/16 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.1.6.0/24 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.10.10.1/32 192.168.101.39 1 0 2510000 ? 10.147.20.0/24 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 50.247.114.16/28 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 50.247.114.18/32 192.168.101.39 0 0 2510000 ? 64.62.134.130/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 68.115.209.232/29 192.168.101.39 1 0 2510000 ? 68.115.209.237/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 72.52.104.74/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 172.16.0.0 192.168.101.39 0 2510000 ? 172.16.184.0/24 192.168.101.39 0 0 2510000 ? 172.16.231.0/24 192.168.101.39 1 0 2510000 ? 172.16.232.0/24 192.168.101.39 0 0 2510000 ? 172.16.238.0/24 192.168.101.39 1 0 2510000 ? 172.17.0.0 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 172.21.0.0 192.168.101.39 0 2510000 ? 192.168.1.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.101.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.106.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.108.0 192.168.101.39 0 2510000 ? 192.168.121.0 192.168.101.39 0 2510000 ? 192.168.128.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.131.0 192.168.101.39 0 2510000 ? 192.168.132.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.148.0 192.168.101.39 1 0 2510000 ? 192.168.150.0 192.168.101.39 0 2510000 ? 192.168.228.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 192.168.229.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 196.101.2.0 192.168.101.39 0 2510000 ? 209.51.161.14/32 192.168.101.39 0 0 2510000 ? Displayed 33 routes and 50 total paths Everything's there, but it never makes it into the kernel. I'm sure we've done something wrong, because I tried a different BGP-based router at the other end, and I see the connection, but again, the routes don't seem to make it into the kernel -- clearly I've broken something basic :-)
---------- _______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
Am 31.12.2018 um 14:07 schrieb Lou Berger:
What's your ip next hop state?
------------------------------------------------------------------------
On December 31, 2018 8:02:47 AM "John Antypas" <jantypas@busygeeks.com> wrote:
The subject basically says it all..... I have two sites connected over an IPSEC tunnel. Each site has its own internal routes managed by OSPF, and I am trying to exchange them by BGP.
* We know the traffic is being passed over the IPSEC tunnel, and we can see that each side does get the other's BGP routes. We did have to install directives to allow multihop and disable-connection-checks.... * We do not see the routes being installed into the local kernel routing tables but for the life us, we can't understand why. We see the routes come across in the BGP debug info, but it never makes it into the FIB.
John, just to be sure. You are using route-based IPSEC with pfSense? Otherwise you wont get any routes via IPSEC as it is an emulated enc device. It'd also be possible to run a GRE tunnel inside IPSEC ... Michael
Can we get a `show ip nht` and a `show bgp nexthop`? donald On Mon, Dec 31, 2018 at 8:02 AM John Antypas <jantypas@busygeeks.com> wrote:
The subject basically says it all..... I have two sites connected over an IPSEC tunnel. Each site has its own internal routes managed by OSPF, and I am trying to exchange them by BGP.
We know the traffic is being passed over the IPSEC tunnel, and we can see that each side does get the other's BGP routes. We did have to install directives to allow multihop and disable-connection-checks.... We do not see the routes being installed into the local kernel routing tables but for the life us, we can't understand why. We see the routes come across in the BGP debug info, but it never makes it into the FIB.
Here's our BGPD.conf albeit for one side -- the other just has the AS numbers changed.
# BGP Config router bgp 3000000 bgp router-id 10.0.0.5 redistribute connected redistribute static redistribute kernel redistribute ospf
# BGP Neighbors neighbor 192.168.101.39 remote-as 2510000 neighbor 192.168.101.39 description Aaron Martin neighbor 192.168.101.39 update-source 10.0.0.5 address-family ipv4 unicast neighbor 192.168.101.39 activate neighbor 192.168.101.39 disable-connected-checks no neighbor 192.168.101.39 send-community neighbor 192.168.101.39 addpath-tx-bestpath-per-AS neighbor 192.168.101.39 allowas-in exit-address-family
And here's what we see
BGP table version is 208559, local router ID is 10.0.0.5, vrf id 0 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path 0.0.0.0 192.168.101.39 0 2510000 3000000 ? 192.168.101.39 0 0 2510000 ? *> 50.247.114.30 0 32768 ? 10.0.0.0 192.168.101.39 0 2510000 ? 10.0.0.0/16 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.1.6.0/24 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.10.10.1/32 192.168.101.39 1 0 2510000 ? 10.147.20.0/24 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 50.247.114.16/28 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 50.247.114.18/32 192.168.101.39 0 0 2510000 ? 64.62.134.130/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 68.115.209.232/29 192.168.101.39 1 0 2510000 ? 68.115.209.237/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 72.52.104.74/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 172.16.0.0 192.168.101.39 0 2510000 ? 172.16.184.0/24 192.168.101.39 0 0 2510000 ? 172.16.231.0/24 192.168.101.39 1 0 2510000 ? 172.16.232.0/24 192.168.101.39 0 0 2510000 ? 172.16.238.0/24 192.168.101.39 1 0 2510000 ? 172.17.0.0 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 172.21.0.0 192.168.101.39 0 2510000 ? 192.168.1.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.101.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.106.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.108.0 192.168.101.39 0 2510000 ? 192.168.121.0 192.168.101.39 0 2510000 ? 192.168.128.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.131.0 192.168.101.39 0 2510000 ? 192.168.132.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.148.0 192.168.101.39 1 0 2510000 ? 192.168.150.0 192.168.101.39 0 2510000 ? 192.168.228.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 192.168.229.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 196.101.2.0 192.168.101.39 0 2510000 ? 209.51.161.14/32 192.168.101.39 0 0 2510000 ?
Displayed 33 routes and 50 total paths
Everything's there, but it never makes it into the kernel. I'm sure we've done something wrong, because I tried a different BGP-based router at the other end, and I see the connection, but again, the routes don't seem to make it into the kernel -- clearly I've broken something basic :-)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
Here you are: pfSense.antypas.net> show ip nht 192.168.101.39(Connected) unresolved(Connected) Client list: bgp(fd 17) show bgp nexthop doesn't appear to be valid On 12/31/2018 5:14:52 AM, Donald Sharp <sharpd@cumulusnetworks.com> wrote: Can we get a `show ip nht` and a `show bgp nexthop`? donald On Mon, Dec 31, 2018 at 8:02 AM John Antypas wrote:
The subject basically says it all..... I have two sites connected over an IPSEC tunnel. Each site has its own internal routes managed by OSPF, and I am trying to exchange them by BGP.
We know the traffic is being passed over the IPSEC tunnel, and we can see that each side does get the other's BGP routes. We did have to install directives to allow multihop and disable-connection-checks.... We do not see the routes being installed into the local kernel routing tables but for the life us, we can't understand why. We see the routes come across in the BGP debug info, but it never makes it into the FIB.
Here's our BGPD.conf albeit for one side -- the other just has the AS numbers changed.
# BGP Config router bgp 3000000 bgp router-id 10.0.0.5 redistribute connected redistribute static redistribute kernel redistribute ospf
# BGP Neighbors neighbor 192.168.101.39 remote-as 2510000 neighbor 192.168.101.39 description Aaron Martin neighbor 192.168.101.39 update-source 10.0.0.5 address-family ipv4 unicast neighbor 192.168.101.39 activate neighbor 192.168.101.39 disable-connected-checks no neighbor 192.168.101.39 send-community neighbor 192.168.101.39 addpath-tx-bestpath-per-AS neighbor 192.168.101.39 allowas-in exit-address-family
And here's what we see
BGP table version is 208559, local router ID is 10.0.0.5, vrf id 0 Status codes: s suppressed, d damped, h history, * valid, > best, = multipath, i internal, r RIB-failure, S Stale, R Removed Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self=""> Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path 0.0.0.0 192.168.101.39 0 2510000 3000000 ? 192.168.101.39 0 0 2510000 ? *> 50.247.114.30 0 32768 ? 10.0.0.0 192.168.101.39 0 2510000 ? 10.0.0.0/16 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.1.6.0/24 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 10.10.10.1/32 192.168.101.39 1 0 2510000 ? 10.147.20.0/24 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 50.247.114.16/28 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 50.247.114.18/32 192.168.101.39 0 0 2510000 ? 64.62.134.130/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 68.115.209.232/29 192.168.101.39 1 0 2510000 ? 68.115.209.237/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 72.52.104.74/32 192.168.101.39 0 2510000 3000000 ? *> 50.247.114.30 0 32768 ? 172.16.0.0 192.168.101.39 0 2510000 ? 172.16.184.0/24 192.168.101.39 0 0 2510000 ? 172.16.231.0/24 192.168.101.39 1 0 2510000 ? 172.16.232.0/24 192.168.101.39 0 0 2510000 ? 172.16.238.0/24 192.168.101.39 1 0 2510000 ? 172.17.0.0 192.168.101.39 0 2510000 3000000 ? *> 10.0.1.5 110 32768 ? 172.21.0.0 192.168.101.39 0 2510000 ? 192.168.1.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.101.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.106.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.108.0 192.168.101.39 0 2510000 ? 192.168.121.0 192.168.101.39 0 2510000 ? 192.168.128.0 192.168.101.39 1 0 2510000 ? 192.168.101.39 0 2510000 ? 192.168.131.0 192.168.101.39 0 2510000 ? 192.168.132.0 192.168.101.39 0 2510000 ? 192.168.101.39 0 0 2510000 ? 192.168.148.0 192.168.101.39 1 0 2510000 ? 192.168.150.0 192.168.101.39 0 2510000 ? 192.168.228.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 192.168.229.0 192.168.101.39 0 2510000 3000000 ? *> 0.0.0.0 1 32768 ? 196.101.2.0 192.168.101.39 0 2510000 ? 209.51.161.14/32 192.168.101.39 0 0 2510000 ?
Displayed 33 routes and 50 total paths
Everything's there, but it never makes it into the kernel. I'm sure we've done something wrong, because I tried a different BGP-based router at the other end, and I see the connection, but again, the routes don't seem to make it into the kernel -- clearly I've broken something basic :-)
_______________________________________________ frog mailing list frog@lists.frrouting.org https://lists.frrouting.org/listinfo/frog
participants (4)
-
Donald Sharp -
John Antypas -
Lou Berger -
Muenz, Michael