<div dir="ltr">And here is `ip route` after bringing the link from eth0 up again. It still lines up with the `show ip route` output I sent earlier.<div><br></div><div>Carl<br><div><br></div><div><div> root@lab1r1u05:~/ovs-droplets# ip route</div><div> default via 10.112.2.4 dev eth2</div><div> <a href="http://10.112.2.0/25">10.112.2.0/25</a> dev eth2 proto kernel scope link src 10.112.2.15</div><div> 10.112.97.1 proto 186 metric 20</div><div> nexthop encap mpls 696097 via 10.112.129.1 dev eth1 weight 1</div><div> nexthop encap mpls 46 via 10.112.129.5 dev eth0 weight 1</div><div> 10.112.128.2 proto 186 metric 20</div><div> nexthop encap mpls 702241 via 10.112.129.1 dev eth1 weight 1</div><div> nexthop encap mpls 564 via 10.112.129.5 dev eth0 weight 1</div><div> <a href="http://10.112.129.0/30">10.112.129.0/30</a> dev eth1 proto kernel scope link src 10.112.129.2</div><div> <a href="http://10.112.129.4/30">10.112.129.4/30</a> dev eth0 proto kernel scope link src 10.112.129.6</div><div> <a href="http://10.112.129.8/30">10.112.129.8/30</a> via 10.112.129.1 dev eth1</div><div> 10.224.12.10 dev br0 scope link</div><div> 10.224.12.15 dev br0 scope link</div><div> 10.224.12.20 proto 186 metric 20</div><div> nexthop encap mpls 702161 via 10.112.129.1 dev eth1 weight 1</div><div> nexthop encap mpls 560 via 10.112.129.5 dev eth0 weight 1</div><div> 10.224.12.25 proto 186 metric 20</div><div> nexthop encap mpls 702161 via 10.112.129.1 dev eth1 weight 1</div><div> nexthop encap mpls 560 via 10.112.129.5 dev eth0 weight 1</div><div> 10.224.12.70 dev br0 scope link</div><div> 10.224.12.75 dev br0 scope link</div><div> 10.224.12.80 proto 186 metric 20</div><div> nexthop encap mpls 702161 via 10.112.129.1 dev eth1 weight 1</div><div> nexthop encap mpls 560 via 10.112.129.5 dev eth0 weight 1</div><div> 10.224.12.85 proto 186 metric 20</div><div> nexthop encap mpls 702161 via 10.112.129.1 dev eth1 weight 1</div><div> nexthop encap mpls 560 via 10.112.129.5 dev eth0 weight 1</div><div> <a href="http://100.64.0.0/24">100.64.0.0/24</a> dev br0 proto kernel scope link src 100.64.0.1</div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jun 26, 2018 at 6:11 AM Donald Sharp <<a href="mailto:sharpd@cumulusnetworks.com">sharpd@cumulusnetworks.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Can we get the output of `show ip route`, `show ip route<br>
10.112.129.9`, `show mpls fec`, and `show mpls table`?<br>
<br>
donald<br>
<br>
On Mon, Jun 25, 2018 at 3:45 PM, Carl Baldwin <<a href="mailto:carl@ecbaldwin.net" target="_blank">carl@ecbaldwin.net</a>> wrote:<br>
> Hi,<br>
><br>
> We're experimenting with BGP-LU and frr in our lab. We have two host<br>
> machines running frr. Each is connected to two TOR switches through their<br>
> eth0 and eth1 links (four TORs total). Those are connected through another<br>
> pair of switches. We have configured BGP-LU to distribute MPLS labels over<br>
> ebpg throughout. Connectivity between loopback addresses on the two hosts<br>
> works well at this point.<br>
><br>
> Then, I created some namespaces in the hosts and gave them addresses. We use<br>
> BGP to announce those addresses as /32 routes with the loopback address as<br>
> the next hop. The route between namespaces on the two machines is<br>
> recursively resolved so that they push the same MPLS label as the path to<br>
> the other loopback. This all works well to start off with and we did some<br>
> iperf runs that showed pretty good results. ECMP was working because the<br>
> bandwidth was higher than any single link.<br>
><br>
> After some link state changes, we seemed to lose the connection. However,<br>
> pings between the loopback addresses still worked. After some time, we<br>
> noticed that the MPLS labels in the routes to the namespace addresses (/32s)<br>
> were different than the label in the route to the loopback. Since the former<br>
> routes are resolved recursively using the latter, the labels should always<br>
> be the same. Could this be a bug in FRR? Shouldn't the routes to the<br>
> namespaces be invalidated or updated as soon as the route it was based on<br>
> changed? The traffic between namespaces is getting dropped because the<br>
> switch doesn't know about the label being pushed by the host.<br>
><br>
> Any insight would be very helpful.<br>
><br>
> Thanks!<br>
> Carl Baldwin<br>
><br>
> Here are /32 routes received. 10.112.128.1 is the loopback on the other<br>
> host. The four routes are to four namespaces on the other host.<br>
><br>
> lab1r2u05# show ip bgp neighbor 10.112.97.1 received-routes<br>
> BGP table version is 0, local router ID is 10.112.128.2<br>
> Status codes: s suppressed, d damped, h history, * valid, > best, =<br>
> multipath,<br>
> i internal, r RIB-failure, S Stale, R Removed<br>
> Origin codes: i - IGP, e - EGP, ? - incomplete<br>
><br>
> Network Next Hop Metric LocPrf Weight Path<br>
> *> <a href="http://10.224.12.10/32" rel="noreferrer" target="_blank">10.224.12.10/32</a> 10.112.128.1 0 4206900001<br>
> 4206909998 i<br>
> *> <a href="http://10.224.12.15/32" rel="noreferrer" target="_blank">10.224.12.15/32</a> 10.112.128.1 0 4206900001<br>
> 4206909998 i<br>
> *> <a href="http://10.224.12.70/32" rel="noreferrer" target="_blank">10.224.12.70/32</a> 10.112.128.1 0 4206900001<br>
> 4206909998 i<br>
> *> <a href="http://10.224.12.75/32" rel="noreferrer" target="_blank">10.224.12.75/32</a> 10.112.128.1 0 4206900001<br>
> 4206909998 i<br>
><br>
> Total number of prefixes 4<br>
><br>
> Below is the routing table as it looked when we lost connectivity. Notice<br>
> that the mpls label for the loopback route is 306592 via eth1 but the label<br>
> for the four namespace addresses is 306576.<br>
><br>
> root@lab1r2u05:~/ovs-droplets# ip route<br>
> default via 10.112.2.132 dev eth2<br>
> <a href="http://10.112.2.128/25" rel="noreferrer" target="_blank">10.112.2.128/25</a> dev eth2 proto kernel scope link src 10.112.2.145<br>
> 10.112.128.1 encap mpls 306592 via 10.112.129.9 dev eth1 proto 186<br>
> metric 20<br>
> <a href="http://10.112.129.8/30" rel="noreferrer" target="_blank">10.112.129.8/30</a> dev eth1 proto kernel scope link src 10.112.129.10<br>
> <a href="http://10.112.129.12/30" rel="noreferrer" target="_blank">10.112.129.12/30</a> dev eth0 proto kernel scope link src 10.112.129.14<br>
> 10.224.12.10 encap mpls 306576 via 10.112.129.9 dev eth1 proto 186<br>
> metric 20<br>
> 10.224.12.15 encap mpls 306576 via 10.112.129.9 dev eth1 proto 186<br>
> metric 20<br>
> 10.224.12.70 encap mpls 306576 via 10.112.129.9 dev eth1 proto 186<br>
> metric 20<br>
> 10.224.12.75 encap mpls 306576 via 10.112.129.9 dev eth1 proto 186<br>
> metric 20<br>
> 10.224.12.20 dev br0 scope link<br>
> 10.224.12.25 dev br0 scope link<br>
> 10.224.12.80 dev br0 scope link<br>
> 10.224.12.85 dev br0 scope link<br>
><br>
><br>
> _______________________________________________<br>
> dev mailing list<br>
> <a href="mailto:dev@lists.frrouting.org" target="_blank">dev@lists.frrouting.org</a><br>
> <a href="https://lists.frrouting.org/listinfo/dev" rel="noreferrer" target="_blank">https://lists.frrouting.org/listinfo/dev</a><br>
><br>
</blockquote></div>