It was just pointed out to me that I did not explain what happened in mid-august: In zebra (specifically rt_netlink.c) we fixed the code that handled blackhole/unreachable routes and started actually seeing them in zebra. So pre this fix zebra would receive the default unreachable route ( or really any unreachable/blackhole route ) and drop it on the floor. Post-fix we accepted the route and noted it in zebra's rib. The admin distance of `0' caused us to not attempt to override the route at all. On Wed, Sep 27, 2017 at 9:33 AM, Donald Sharp <sharpd@cumulusnetworks.com> wrote:

All -

This PR:

https://github.com/FRRouting/frr/pull/1237

has received a request that we spend a bit of time disseminating the problem and this proposed fix for more feedback.

The Problem:

Linux kernel VRF table lookups follow this behavior: Lookup up in the table prefix A. If no route is found fall back to the main default table and lookup prefix A again. This behavior is considered 'surprising' to people who expect the behavior of VRF's to stop looking when nothing is found in the corresponding table. The linux kernel coders documented that a default unreachable route with a high metric should be installed into the VRF table to prevent the fallback mechanism from being invoked in the kernel. Prior to mid-August this was not that big of a deal as that default unreachable routes were being ignored by rt_netlink.c. As such when a zebra( or higher ) installed a default route, zebra would not have knowledge of the default unreachable route and so zebra would make the decision to just install the default route. Since zebra installs routes into the kernel with a metric of 20, the new default route would be used.

So After mid August, zebra started seeing the default unreachable kernel route. This default unreachable received an admin distance of 0. As such when FRR attempted to install another default route it would fail the admin distance check and not be installed.

The Proposal:

Since there is no good way to specify admin distance to/from the kernel currently, let's interpret the metric/priority for ZEBRA_ROUTE_KERNEL routes a bit differently admin distance == High order byte of metric passed from kernel metric == Low 3 bytes of metric passed from kernel

What this allows:

The vrf creator scripts( something like iproute2 ) can be modified to use a metric that would correspond to an admin distance of 255. This way when the default unreachable is read we translate the metric to Admin distance = 255 metric = <some value>

This allows zebra to allow static routes and routing protocols to override the default unreachable and it allows people who do not modify the metric values for normal ( ip route .... commands ) for their routes to behave like they think it should (the kernel should win)

This does not affect *bsd because most bsd's have a uint8_t metric and we will not change behavior their.

donald