Re: [dev] Open Network Equipment and FRRouting
On Mon, Jan 21, 2019 at 01:54:48PM +0000, Łukasz Wójcik via dev wrote:
Date: Mon, 21 Jan 2019 14:17:45 +0100 From: Łukasz Wójcik <lukasz.wojcik@zoho.com> To: dev@lists.frrouting.org Subject: Open Network Equipment and FRRouting
Hello there,
In the era of Open Networking equipment, e.g. switching ASICs support forwarding based on L3 information. I was wondering, if FRRouting supports or plans to support any mechanisms that would allow for passing L3 data to the hardware, and therefore offloading some work to the HW ? I would imagine some kind of API or event-driven mechanisms that would require HW manufacturer's
Hi Łukasz There are multiple ways to do this. The simplest is to just use the kernel APIs as they are for adding/removing routes. The kernel already has all the internal plumbing to pass the information down into the switch driver, and offload it to the hardware. FFR just works. The problem is, few of the Open Networking equipment are actually open. They have a big binary blob SDK running in user space, not an open source driver in the kernel. Andrew
Hello Andrew, Indeed, I am aware of how netlink works, I was just hoping that there will be some kind of unified API, same for pretty much every ASIC. In any case, there's also ORC in ONL. As I understand it, it provides it's own virtual interfaces attached to actual switch ports. Any change to this interface (or routes related to these) would be translated to ASIC. This approach seems very nice, as you could simply use these virtual interfaces, as you would with physical interfaces, and therefore you would not need any changes in FRR. The problem I see is that I cannot find any documentation. All info I've found is few mailing list posts + mention on ONL's website. Thank you, -ŁW On 21-Jan-19 19:57, Andrew Lunn wrote:
On Mon, Jan 21, 2019 at 01:54:48PM +0000, Łukasz Wójcik via dev wrote:
Date: Mon, 21 Jan 2019 14:17:45 +0100 From: Łukasz Wójcik <lukasz.wojcik@zoho.com> To: dev@lists.frrouting.org Subject: Open Network Equipment and FRRouting
Hello there,
In the era of Open Networking equipment, e.g. switching ASICs support forwarding based on L3 information. I was wondering, if FRRouting supports or plans to support any mechanisms that would allow for passing L3 data to the hardware, and therefore offloading some work to the HW ? I would imagine some kind of API or event-driven mechanisms that would require HW manufacturer's Hi Łukasz
There are multiple ways to do this. The simplest is to just use the kernel APIs as they are for adding/removing routes. The kernel already has all the internal plumbing to pass the information down into the switch driver, and offload it to the hardware. FFR just works.
The problem is, few of the Open Networking equipment are actually open. They have a big binary blob SDK running in user space, not an open source driver in the kernel.
Andrew
On Mon, Jan 21, 2019 at 08:08:06PM +0100, Łukasz Wójcik wrote:
Hello Andrew,
Indeed, I am aware of how netlink works, I was just hoping that there will be some kind of unified API, same for pretty much every ASIC.
I doubt that is going to happen for a while, at least for L3 devices. L2 is pretty good at the moment. There are probably over 50 switches supported in the Linux kernel, with some vendors supporting their own switches. However the L3 vendors don't yet seem to like true open networking for their ASICs, they want to hide too many secrets in the SDK.
In any case, there's also ORC in ONL. As I understand it, it provides it's own virtual interfaces attached to actual switch ports. Any change to this interface (or routes related to these) would be translated to ASIC. This approach seems very nice, as you could simply use these virtual interfaces, as you would with physical interfaces, and therefore you would not need any changes in FRR. The problem I see is that I cannot find any documentation.
This is basically how the kernel does it. Each port of the switch is a true Linux interface. You add routes to the interface, and the kernel will offload it to the hardware. If you create a bridge and add interfaces to the bridge, the kernel will offload it to the hardware. If you use the team driver to create a LAG, and add interfaces to it, the kernel will offload it to the hardware. All your usual tools, routing daemon, snmp agents, just work. Andrew
On Mon, 2019-01-21 at 20:36 +0100, Andrew Lunn wrote:
On Mon, Jan 21, 2019 at 08:08:06PM +0100, Łukasz Wójcik wrote:
Hello Andrew,
Indeed, I am aware of how netlink works, I was just hoping that there will be some kind of unified API, same for pretty much every ASIC.
I doubt that is going to happen for a while, at least for L3 devices.
I agree and imho the most promising approach seems to be the Stratum Project (https://stratumproject.org/) which is going to be open sourced this year. Hence you would be able to deal with fixed function ASICs and flexible ones. Let's keep fingers crossed ;) /tobi
This is basically how the kernel does it. Each port of the switch is a true Linux interface. You add routes to the interface, and the kernel will offload it to the hardware. If you create a bridge and add interfaces to the bridge, the kernel will offload it to the hardware. If you use the team driver to create a LAG, and add interfaces to it, the kernel will offload it to the hardware. All your usual tools, routing daemon, snmp agents, just work.
Can you give an example of equipment that I can actually buy? All my professional life I use 1U x86 Linux servers for NAT|Routing + L2 Switch for LACP's|VLAN's, and always want something like this setup but in one box. k
On Wed, Jan 23, 2019 at 02:09:09PM +0700, Konstantin Shalygin wrote:
This is basically how the kernel does it. Each port of the switch is a true Linux interface. You add routes to the interface, and the kernel will offload it to the hardware. If you create a bridge and add interfaces to the bridge, the kernel will offload it to the hardware. If you use the team driver to create a LAG, and add interfaces to it, the kernel will offload it to the hardware. All your usual tools, routing daemon, snmp agents, just work.
Can you give an example of equipment that I can actually buy? All my professional life I use 1U x86 Linux servers for NAT|Routing + L2 Switch for LACP's|VLAN's, and always want something like this setup but in one box.
Hi Konstantin If you want a fully open source in kernel solution, you are currently looking at a Mellanox based device. They are only L3 switch vendor contributing to the Linux kernel. If you are happy to do NAT and Routing in software, and don't need too many ports, you can look at some of the ARM machines using Marvell or Broadcom L2 switches. Solidrun Clearfog variants for example. At the moment, it is mostly industrial applications which are driving most of the kernel L2 user cases. So switches/routers for trains, planes, busses and industrial automation. Andrew
participants (4)
-
Andrew Lunn -
Konstantin Shalygin -
Tobias Jungel -
Łukasz Wójcik