[FROG] v6 Route Replace semantics

[Donald Sharp]

All -

We recently introduced, into master, a new cli switch for zebra on
linux `--v6-rr-semantics`.  This was allow zebra to take advantage of
the linux kernel getting v6 route replace semantics that is coming
soon in the 4.18 linux kernel.  So if you are on the bleeding edge of
linux kernels and FRR you can start to take advantage of this new
feature.

What are Route Replace semantics you ask?

Suppose we have a v6 route with 2 nexthops:
S>* 4:6:7::/64 [1/0] via 1:2:3::4, enp0s3, 00:00:02
                     via 1:2:8::8, enp0s8, 00:00:02

Now suppose we remove a nexthop:
robot# conf t
robot(config)# no ipv6 route 5:7:9::/78 1:2:8::9
robot(config)# end
robot# show ipv6 route
[snip]
S>* 5:7:9::/78 [1/0] via 1:2:3::4, enp0s3, 00:01:04

Let's look at the netlink notifications for this event:
[ROUTE]5:7:9::/78 proto static metric 20      <----------- From
creation of the 2x ecmp for 5:7:9::/78
nexthop via 1:2:3::4 dev enp0s3 weight 1
nexthop via 1:2:8::9 dev enp0s8 weight 1
[ROUTE]Deleted 5:7:9::/78 proto static metric 20 <---- We delete the route
nexthop via 1:2:3::4 dev enp0s3 weight 1
nexthop via 1:2:8::9 dev enp0s8 weight 1
[ROUTE]5:7:9::/78 via 1:2:3::4 dev enp0s3 proto static metric 20 pref
medium <----- We add it back in.

In order to get the routes right in the kernel we had to delete the
route first and then add it back in.  Exposing a small window of
packets being dropped while the route is fixed in the fib.

Route Replace semantics tells the linux kernel to just replace the
nexthops for the route.  New packets coming in would immediately start
using the new ecmp nexthop group path:
[ROUTE]5:7:9::/78 proto static metric 20
nexthop via 1:2:3::4 dev enp0s3 weight 1
nexthop via 1:2:8::9 dev enp0s8 weight 1
[ROUTE]5:7:9::/78 via 1:2:3::4 dev enp0s3 proto static metric 20 pref medium

thanks!

donald