Hi everyone, FRR8.4 updated on my FRR8.3.1 system yesterday. 8.3.1 has been running fine for many weeks. FRR8.4's BGPD crashes every 25 minutes or so with this appearing in syslog: ******************** Nov 8 22:59:29 frr BGP[81651]: Received signal 6 at 1667948369 (si_addr 0x7000013ef3, PC 0x7fe3e2e5ba7c); aborting... Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_backtrace_sigsafe+0x71) [0x7fe3e3184f01] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_signal+0xf5) [0x7fe3e3185105] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0xc3b45) [0x7fe3e31afb45] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fe3e2e07520] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c) [0x7fe3e2e5ba7c] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(raise+0x16) [0x7fe3e2e07476] Nov 8 22:59:29 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3) [0x7fe3e2ded7f3] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(_zlog_assert_failed+0xed) [0x7fe3e31d400d] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(route_node_delete+0x16e) [0x7fe3e31b940e] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/modules/bgpd_rpki.so(+0x7a65) [0x7fe3e2bd0a65] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(thread_call+0x81) [0x7fe3e31c3c01] Nov 8 22:59:29 frr BGP[81651]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(frr_run+0xe8) [0x7fe3e3180508] Nov 8 22:59:30 frr BGP[81651]: /usr/lib/frr/bgpd(main+0x37c) [0x55d34af51dec] Nov 8 22:59:30 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fe3e2deed90] Nov 8 22:59:30 frr BGP[81651]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7fe3e2deee40] Nov 8 22:59:30 frr BGP[81651]: /usr/lib/frr/bgpd(_start+0x25) [0x55d34af52895] Nov 8 22:59:30 frr BGP[81651]: in thread bgpd_sync_callback scheduled from bgpd/bgp_rpki.c:403 bgpd_sync_callback() ... Nov 8 23:24:44 frr BGP[84102]: Received signal 6 at 1667949884 (si_addr 0x7000014886, PC 0x7ff44d243a7c); aborting... Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_backtrace_sigsafe+0x71) [0x7ff44d56cf01] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_signal+0xf5) [0x7ff44d56d105] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0xc3b45) [0x7ff44d597b45] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7ff44d1ef520] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c) [0x7ff44d243a7c] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(raise+0x16) [0x7ff44d1ef476] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3) [0x7ff44d1d57f3] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(_zlog_assert_failed+0xed) [0x7ff44d5bc00d] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(route_node_delete+0x16e) [0x7ff44d5a140e] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/modules/bgpd_rpki.so(+0x7a65) [0x7ff44cfb8a65] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(thread_call+0x81) [0x7ff44d5abc01] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(frr_run+0xe8) [0x7ff44d568508] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/frr/bgpd(main+0x37c) [0x55750e7f8dec] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7ff44d1d6d90] Nov 8 23:24:44 frr BGP[84102]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7ff44d1d6e40] Nov 8 23:24:44 frr BGP[84102]: /usr/lib/frr/bgpd(_start+0x25) [0x55750e7f9895] Nov 8 23:24:44 frr BGP[84102]: in thread bgpd_sync_callback scheduled from bgpd/bgp_rpki.c:403 bgpd_sync_callback() ******************** This is with frr, frr-pythontools, and frr-rpki-rtrlib installed. Looks like unhappiness in the rpki component. I've got about 60 BGP feeds into this collector, some full table, but most are 30k routes. I also noticed this new issue appearing with FRR8.4: ******************** 2022/11/08 23:37:24 BGP: [ZM2F8-MV4BJ][EC 33554509] Interface: eth0 does not have a v6 LL address associated with it, waiting until one is created for it ******************** No issue in FRR8.3.1 before it and eth0 definitely does have a v6 LL address associated with it, as per below: ******************** 56: eth0@if57: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 00:16:3e:35:10:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 128.xxx.xxx.xxx brd 128.xxx.xxx.xxx scope global eth0 valid_lft forever preferred_lft forever inet6 2607:xxxx:xxxx::xxxx/64 scope global valid_lft forever preferred_lft forever inet6 fe80::216:3eff:fe35:10ab/64 scope link valid_lft forever preferred_lft forever ******************** If any other bits are need to help decipher any of this, please let me know. I've had to revert to 8.3.1 unfortunately, as the instabilities were causing issues for the providers of the feeds I am receiving. Thanks! philip --