Hi Donald, Donald Sharp wrote on 9/11/2022 21:28:
I noticed the bgp crash in rpki myself this yesterday morning as well. https://github.com/FRRouting/frr/pull/12287/commits/31d0363ffc3c387204068d9b... <https://github.com/FRRouting/frr/pull/12287/commits/31d0363ffc3c387204068d9b81c4f281a2116342> That is the commit you want to fix the issue. I'll make sure it gets backported to 8.4
Ah! :-) Ok will look out for the fix in 8.4. Thank you.
For the eth0 issue. What do you have configured on that interface? and Can I see `show int eth0` inside of vtysh?
Looking at the logs and the timing, I think this is a feature of bgpd starting up. The eth0 message only pops up right before the EBGP sessions are re-established after the bgpd crash in 8.4. The IPv6 sessions come up, and the message about v6 LL never reappears after that. The below is from 8.3.1 - but it is the same format as my other 8.4 instance (a testbed). frr.nsrc.org# show interface eth0 Interface eth0 is up, line protocol is up Link ups: 0 last: (never) Link downs: 0 last: (never) vrf: default index 56 metric 0 mtu 1500 speed 10000 flags: <UP,BROADCAST,RUNNING,MULTICAST> Type: Ethernet HWaddr: 00:16:3e:35:10:ab inet 128.xxx.xxx.xxx/25 inet6 2607:xxx:xxx::xxx/64 inet6 fe80::216:3eff:fe35:10ab/64 Interface Type VETH Interface Slave Type None protodown: off Parent ifindex: 57 Looking back through older logs, I see the same in 8.3.1 each time I start up frr after system reboot or "systemctl restart frr". Which is not very often at all so I never noticed before. Perhaps nothing to worry about, tbh. philip --
On Tue, Nov 8, 2022 at 8:10 PM Philip Smith <philip@nsrc.org <mailto:philip@nsrc.org>> wrote:
Hi everyone,
FRR8.4 updated on my FRR8.3.1 system yesterday. 8.3.1 has been running fine for many weeks.
FRR8.4's BGPD crashes every 25 minutes or so with this appearing in syslog:
******************** Nov 8 22:59:29 frr BGP[81651]: Received signal 6 at 1667948369 (si_addr 0x7000013ef3, PC 0x7fe3e2e5ba7c); aborting... Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_backtrace_sigsafe+0x71) [0x7fe3e3184f01] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_signal+0xf5) [0x7fe3e3185105] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0xc3b45) [0x7fe3e31afb45] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fe3e2e07520] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c) [0x7fe3e2e5ba7c] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(raise+0x16) [0x7fe3e2e07476] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3) [0x7fe3e2ded7f3] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(_zlog_assert_failed+0xed) [0x7fe3e31d400d] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(route_node_delete+0x16e) [0x7fe3e31b940e] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/modules/bgpd_rpki.so(+0x7a65) [0x7fe3e2bd0a65] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(thread_call+0x81) [0x7fe3e31c3c01] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(frr_run+0xe8) [0x7fe3e3180508] Nov 8 22:59:30 frr BGP[81651]: /usr/lib/frr/bgpd(main+0x37c) [0x55d34af51dec] Nov 8 22:59:30 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fe3e2deed90] Nov 8 22:59:30 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7fe3e2deee40] Nov 8 22:59:30 frr BGP[81651]: /usr/lib/frr/bgpd(_start+0x25) [0x55d34af52895] Nov 8 22:59:30 frr BGP[81651]: in thread bgpd_sync_callback scheduled from bgpd/bgp_rpki.c:403 bgpd_sync_callback() ... Nov 8 23:24:44 frr BGP[84102]: Received signal 6 at 1667949884 (si_addr 0x7000014886, PC 0x7ff44d243a7c); aborting... Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_backtrace_sigsafe+0x71) [0x7ff44d56cf01] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_signal+0xf5) [0x7ff44d56d105] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0xc3b45) [0x7ff44d597b45] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7ff44d1ef520] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c) [0x7ff44d243a7c] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(raise+0x16) [0x7ff44d1ef476] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3) [0x7ff44d1d57f3] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(_zlog_assert_failed+0xed) [0x7ff44d5bc00d] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(route_node_delete+0x16e) [0x7ff44d5a140e] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/modules/bgpd_rpki.so(+0x7a65) [0x7ff44cfb8a65] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(thread_call+0x81) [0x7ff44d5abc01] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(frr_run+0xe8) [0x7ff44d568508] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/frr/bgpd(main+0x37c) [0x55750e7f8dec] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7ff44d1d6d90] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7ff44d1d6e40] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/frr/bgpd(_start+0x25) [0x55750e7f9895] Nov 8 23:24:44 frr BGP[84102]: in thread bgpd_sync_callback scheduled from bgpd/bgp_rpki.c:403 bgpd_sync_callback() ********************
This is with frr, frr-pythontools, and frr-rpki-rtrlib installed.
Looks like unhappiness in the rpki component.
I've got about 60 BGP feeds into this collector, some full table, but most are 30k routes.
I also noticed this new issue appearing with FRR8.4:
******************** 2022/11/08 23:37:24 BGP: [ZM2F8-MV4BJ][EC 33554509] Interface: eth0 does not have a v6 LL address associated with it, waiting until one is created for it ********************
No issue in FRR8.3.1 before it and eth0 definitely does have a v6 LL address associated with it, as per below:
******************** 56: eth0@if57: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 00:16:3e:35:10:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 128.xxx.xxx.xxx brd 128.xxx.xxx.xxx scope global eth0 valid_lft forever preferred_lft forever inet6 2607:xxxx:xxxx::xxxx/64 scope global valid_lft forever preferred_lft forever inet6 fe80::216:3eff:fe35:10ab/64 scope link valid_lft forever preferred_lft forever ********************
If any other bits are need to help decipher any of this, please let me know.
I've had to revert to 8.3.1 unfortunately, as the instabilities were causing issues for the providers of the feeds I am receiving.
Thanks!
philip --
_______________________________________________ frog mailing list frog@lists.frrouting.org <mailto:frog@lists.frrouting.org> https://lists.frrouting.org/listinfo/frog <https://lists.frrouting.org/listinfo/frog>