frr 8.4.x and 8.5 trigger assertion in bgpd/bgp_lcommunity.c:236
Hi! This caused bgpd to terminate on several boxes this morning: 2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting... OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ? 7.5.1 is not affected... Any ideas on how to avoid the assert ? -- pi@opsec.eu +49 171 3101372 Now what ?
Hi!
This caused bgpd to terminate on several boxes this morning:
2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting...
OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ?
7.5.1 is not affected...
Any ideas on how to avoid the assert ?
There's the lookup for community aliases added in that function: const char *com2alias = translate_alias ? bgp_community2alias(lcsb) : lcsb; len = strlcat(str_buf, com2alias, str_buf_sz); and it seems to trigger the issue. -- pi@opsec.eu +49 171 3101372 Now what ?
Hi!
This caused bgpd to terminate on several boxes this morning:
2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting...
OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ?
7.5.1 is not affected...
Any ideas on how to avoid the assert ?
There's the lookup for community aliases added in that function:
const char *com2alias = translate_alias ? bgp_community2alias(lcsb) : lcsb;
len = strlcat(str_buf, com2alias, str_buf_sz);
and it seems to trigger the issue.
I created a patch for the FreeBSD port, see: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270910 -- pi@opsec.eu +49 171 3101372 Now what ?
On 4/18/2023 5:28 AM, Kurt Jaeger wrote:
Hi!
This caused bgpd to terminate on several boxes this morning:
2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting...
OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ?
7.5.1 is not affected...
Any ideas on how to avoid the assert ? There's the lookup for community aliases added in that function:
const char *com2alias = translate_alias ? bgp_community2alias(lcsb) : lcsb;
len = strlcat(str_buf, com2alias, str_buf_sz);
and it seems to trigger the issue.
I am seeing this crash as well. Sadly the patch in the freebsd pr does not seem to work around the issue. If its an issue of communities, is there an easy way to strip out communities from peers for now ? ---Mike
On 4/20/2023 9:30 AM, mike tancsa wrote:
On 4/18/2023 5:28 AM, Kurt Jaeger wrote:
Hi!
This caused bgpd to terminate on several boxes this morning:
2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting...
OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ?
7.5.1 is not affected...
Any ideas on how to avoid the assert ? There's the lookup for community aliases added in that function:
const char *com2alias = translate_alias ? bgp_community2alias(lcsb) : lcsb; len = strlcat(str_buf, com2alias, str_buf_sz);
and it seems to trigger the issue.
I am seeing this crash as well. Sadly the patch in the freebsd pr does not seem to work around the issue. If its an issue of communities, is there an easy way to strip out communities from peers for now ?
Adding some debugging. It seems to be a large community that is bigger than 1024 bytes. Looking at the pcap, 09:27:57.240613 IP (tos 0xc0, ttl 1, id 22455, offset 0, flags [DF], proto TCP (6), length 1500) 192.168.158.77.179 > 192.168.158.78.64274: Flags [.], cksum 0x96a5 (correct), seq 4567154:4568614, ack 964, win 31913, length 1460: BGP [|BGP] Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 45758 55808 147027 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 64070 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 110.164.235.0/24 110.164.234.0/24 Update Message (2), length: 1021 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 3491 60068 983 38136 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Large Community (32), length: 936, Flags [OTE]: 4635:0:0, 4635:1:906, 4635:1:2906, 4635:1:4515, 4635:1:4594, 4635:1:4641, 4635:1:4760, 4635:1:7979, 4635:1:9253, 4635:1:9293, 4635:1:9304, 4635:1:9908, 4635:1:13335, 4635:1:16265, 4635:1:17924, 4635:1:18013, 4635:1:20940, 4635:1:22822, 4635:1:24429, 4635:1:24482, 4635:1:32590, 4635:1:32934, 4635:1:36692, 4635:1:38008, 4635:1:38819, 4635:1:41378, 4635:1:45753, 4635:1:46489, 4635:1:49544, 4635:1:51847, 4635:1:54574, 4635:1:54994, 4635:1:55720, 4635:1:56059, 4635:1:57724, 4635:1:65021, 4635:1:134823, 4635:1:136907, 4635:1:146961, 24115:0:24115, 24115:1:906, 24115:1:2906, 24115:1:4515, 24115:1:4594, 24115:1:4641, 24115:1:4760, 24115:1:7979, 24115:1:9253, 24115:1:9293, 24115:1:9304, 24115:1:9908, 24115:1:13335, 24115:1:16265, 24115:1:17924, 24115:1:18013, 24115:1:20940, 24115:1:22822, 24115:1:24429, 24115:1:24482, 24115:1:32590, 24115:1:32934, 24115:1:36692, 24115:1:38008, 24115:1:38819, 24115:1:41378, 24115:1:45753, 24115:1:46489, 24115:1:49544, 24115:1:51847, 24115:1:54574, 24115:1:54994, 24115:1:55720, 24115:1:56059, 24115:1:57724, 24115:1:65021, 24115:1:134823, 24115:1:136907, 24115:1:146961 Updated routes: 103.135.103.0/24 Update Message (2), length: 77 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 262589 14259 16921 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 48050 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 200.108.121.0/24 Update Message (2), length: 73 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 14, Flags [T]: 174 1239 54174 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Updated routes: 208.21.37.0/24 Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 1299 6939 6939 398704 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 1 Community (8), length: 8, Flags [OT]: 174:21000, 174:22003 Updated routes: 205.198.124.0/24 With some added debugging, it seems its exceeding the size of the buffer zlog_warn("set_lcommunity_string: str_buf_sz=%lu [%s] [%s]\n", str_buf_sz, lcsb, str_buf); Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:136907] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:146961] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: bgpd/bgp_lcommunity.c:239: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed
Please try this patch: diff --git a/bgpd/bgp_lcommunity.c b/bgpd/bgp_lcommunity.c index 15bf419868..c61684f21e 100644 --- a/bgpd/bgp_lcommunity.c +++ b/bgpd/bgp_lcommunity.c @@ -197,7 +197,7 @@ static void set_lcommunity_string(struct lcommunity *lcom, bool make_json, } /* 1 space + lcom->size lcom strings + null terminator */ - size_t str_buf_sz = BUFSIZ; + size_t str_buf_sz = BUFSIZ * 10; str_buf = XCALLOC(MTYPE_LCOMMUNITY_STR, str_buf_sz); for (i = 0; i < lcom->size; i++) { we are creating a string that is not big enough for the community. We need to look at this a bit more precisely than my multiplication by 10 On Thu, Apr 20, 2023 at 10:33 AM mike tancsa <mike@sentex.net> wrote:
On 4/20/2023 9:30 AM, mike tancsa wrote:
On 4/18/2023 5:28 AM, Kurt Jaeger wrote:
Hi!
This caused bgpd to terminate on several boxes this morning:
2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting...
OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ?
7.5.1 is not affected...
Any ideas on how to avoid the assert ? There's the lookup for community aliases added in that function:
const char *com2alias = translate_alias ? bgp_community2alias(lcsb) : lcsb; len = strlcat(str_buf, com2alias, str_buf_sz);
and it seems to trigger the issue.
I am seeing this crash as well. Sadly the patch in the freebsd pr does not seem to work around the issue. If its an issue of communities, is there an easy way to strip out communities from peers for now ?
Adding some debugging. It seems to be a large community that is bigger than 1024 bytes. Looking at the pcap,
09:27:57.240613 IP (tos 0xc0, ttl 1, id 22455, offset 0, flags [DF], proto TCP (6), length 1500) 192.168.158.77.179 > 192.168.158.78.64274: Flags [.], cksum 0x96a5 (correct), seq 4567154:4568614, ack 964, win 31913, length 1460: BGP [|BGP] Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 45758 55808 147027 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 64070 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 110.164.235.0/24 110.164.234.0/24 Update Message (2), length: 1021 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 3491 60068 983 38136 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Large Community (32), length: 936, Flags [OTE]: 4635:0:0, 4635:1:906, 4635:1:2906, 4635:1:4515, 4635:1:4594, 4635:1:4641, 4635:1:4760, 4635:1:7979, 4635:1:9253, 4635:1:9293, 4635:1:9304, 4635:1:9908, 4635:1:13335, 4635:1:16265, 4635:1:17924, 4635:1:18013, 4635:1:20940, 4635:1:22822, 4635:1:24429, 4635:1:24482, 4635:1:32590, 4635:1:32934, 4635:1:36692, 4635:1:38008, 4635:1:38819, 4635:1:41378, 4635:1:45753, 4635:1:46489, 4635:1:49544, 4635:1:51847, 4635:1:54574, 4635:1:54994, 4635:1:55720, 4635:1:56059, 4635:1:57724, 4635:1:65021, 4635:1:134823, 4635:1:136907, 4635:1:146961, 24115:0:24115, 24115:1:906, 24115:1:2906, 24115:1:4515, 24115:1:4594, 24115:1:4641, 24115:1:4760, 24115:1:7979, 24115:1:9253, 24115:1:9293, 24115:1:9304, 24115:1:9908, 24115:1:13335, 24115:1:16265, 24115:1:17924, 24115:1:18013, 24115:1:20940, 24115:1:22822, 24115:1:24429, 24115:1:24482, 24115:1:32590, 24115:1:32934, 24115:1:36692, 24115:1:38008, 24115:1:38819, 24115:1:41378, 24115:1:45753, 24115:1:46489, 24115:1:49544, 24115:1:51847, 24115:1:54574, 24115:1:54994, 24115:1:55720, 24115:1:56059, 24115:1:57724, 24115:1:65021, 24115:1:134823, 24115:1:136907, 24115:1:146961 Updated routes: 103.135.103.0/24 Update Message (2), length: 77 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 262589 14259 16921 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 48050 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 200.108.121.0/24 Update Message (2), length: 73 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 14, Flags [T]: 174 1239 54174 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Updated routes: 208.21.37.0/24 Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 1299 6939 6939 398704 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 1 Community (8), length: 8, Flags [OT]: 174:21000, 174:22003 Updated routes: 205.198.124.0/24
With some added debugging, it seems its exceeding the size of the buffer zlog_warn("set_lcommunity_string: str_buf_sz=%lu [%s] [%s]\n", str_buf_sz, lcsb, str_buf);
Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:136907] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:146961] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: bgpd/bgp_lcommunity.c:239: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed
_______________________________________________ dev mailing list dev@lists.frrouting.org https://lists.frrouting.org/listinfo/dev
Thanks. Just to be clear, this will need to be applied to ALL my bgp speakers first, otherwise it will just propagate the issue to all my ibgp peers and then they will crash. Is there a way to drop these large communities to protect things ? ---Mike On 4/20/2023 10:46 AM, Donald Sharp wrote:
Please try this patch:
diff --git a/bgpd/bgp_lcommunity.c b/bgpd/bgp_lcommunity.c index 15bf419868..c61684f21e 100644 --- a/bgpd/bgp_lcommunity.c +++ b/bgpd/bgp_lcommunity.c @@ -197,7 +197,7 @@ static void set_lcommunity_string(struct lcommunity *lcom, bool make_json, }
/* 1 space + lcom->size lcom strings + null terminator */ - size_t str_buf_sz = BUFSIZ; + size_t str_buf_sz = BUFSIZ * 10; str_buf = XCALLOC(MTYPE_LCOMMUNITY_STR, str_buf_sz);
for (i = 0; i < lcom->size; i++) {
we are creating a string that is not big enough for the community. We need to look at this a bit more precisely than my multiplication by 10
On Thu, Apr 20, 2023 at 10:33 AM mike tancsa <mike@sentex.net> wrote:
On 4/20/2023 9:30 AM, mike tancsa wrote: > On 4/18/2023 5:28 AM, Kurt Jaeger wrote: >> Hi! >> >>> This caused bgpd to terminate on several boxes this morning: >>> >>> 2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: >>> set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) >>> failed >>> BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting... >>> >>> OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending >>> strange things... ? >>> >>> 7.5.1 is not affected... >>> >>> Any ideas on how to avoid the assert ? >> There's the lookup for community aliases added in that function: >> >> const char *com2alias = >> translate_alias ? bgp_community2alias(lcsb) >> : lcsb; >> len = strlcat(str_buf, com2alias, str_buf_sz); >> >> and it seems to trigger the issue. > > > I am seeing this crash as well. Sadly the patch in the freebsd pr does > not seem to work around the issue. If its an issue of communities, is > there an easy way to strip out communities from peers for now ? > Adding some debugging. It seems to be a large community that is bigger than 1024 bytes. Looking at the pcap,
09:27:57.240613 IP (tos 0xc0, ttl 1, id 22455, offset 0, flags [DF], proto TCP (6), length 1500) 192.168.158.77.179 > 192.168.158.78.64274: Flags [.], cksum 0x96a5 (correct), seq 4567154:4568614, ack 964, win 31913, length 1460: BGP [|BGP] Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 45758 55808 147027 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 64070 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 110.164.235.0/24 <http://110.164.235.0/24> 110.164.234.0/24 <http://110.164.234.0/24> Update Message (2), length: 1021 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 3491 60068 983 38136 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Large Community (32), length: 936, Flags [OTE]: 4635:0:0, 4635:1:906, 4635:1:2906, 4635:1:4515, 4635:1:4594, 4635:1:4641, 4635:1:4760, 4635:1:7979, 4635:1:9253, 4635:1:9293, 4635:1:9304, 4635:1:9908, 4635:1:13335, 4635:1:16265, 4635:1:17924, 4635:1:18013, 4635:1:20940, 4635:1:22822, 4635:1:24429, 4635:1:24482, 4635:1:32590, 4635:1:32934, 4635:1:36692, 4635:1:38008, 4635:1:38819, 4635:1:41378, 4635:1:45753, 4635:1:46489, 4635:1:49544, 4635:1:51847, 4635:1:54574, 4635:1:54994, 4635:1:55720, 4635:1:56059, 4635:1:57724, 4635:1:65021, 4635:1:134823, 4635:1:136907, 4635:1:146961, 24115:0:24115, 24115:1:906, 24115:1:2906, 24115:1:4515, 24115:1:4594, 24115:1:4641, 24115:1:4760, 24115:1:7979, 24115:1:9253, 24115:1:9293, 24115:1:9304, 24115:1:9908, 24115:1:13335, 24115:1:16265, 24115:1:17924, 24115:1:18013, 24115:1:20940, 24115:1:22822, 24115:1:24429, 24115:1:24482, 24115:1:32590, 24115:1:32934, 24115:1:36692, 24115:1:38008, 24115:1:38819, 24115:1:41378, 24115:1:45753, 24115:1:46489, 24115:1:49544, 24115:1:51847, 24115:1:54574, 24115:1:54994, 24115:1:55720, 24115:1:56059, 24115:1:57724, 24115:1:65021, 24115:1:134823, 24115:1:136907, 24115:1:146961 Updated routes: 103.135.103.0/24 <http://103.135.103.0/24> Update Message (2), length: 77 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 262589 14259 16921 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 48050 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 200.108.121.0/24 <http://200.108.121.0/24> Update Message (2), length: 73 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 14, Flags [T]: 174 1239 54174 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Updated routes: 208.21.37.0/24 <http://208.21.37.0/24> Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 1299 6939 6939 398704 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 1 Community (8), length: 8, Flags [OT]: 174:21000, 174:22003 Updated routes: 205.198.124.0/24 <http://205.198.124.0/24>
With some added debugging, it seems its exceeding the size of the buffer zlog_warn("set_lcommunity_string: str_buf_sz=%lu [%s] [%s]\n", str_buf_sz, lcsb, str_buf);
Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:136907] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:146961] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: bgpd/bgp_lcommunity.c:239: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed
_______________________________________________ dev mailing list dev@lists.frrouting.org https://lists.frrouting.org/listinfo/dev
BTW, to recreate the problem, you can do it on the actual test instance do something like route-map LARGE-OUT permit 10 set large-community 4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:9304 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 24115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 24115:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 24115:1:100000 24115:1:100001 24115:1:100002 exit And it will crash the sender. ---Mike On 4/20/2023 10:52 AM, mike tancsa wrote:
Thanks. Just to be clear, this will need to be applied to ALL my bgp speakers first, otherwise it will just propagate the issue to all my ibgp peers and then they will crash.
Is there a way to drop these large communities to protect things ?
---Mike
On 4/20/2023 10:46 AM, Donald Sharp wrote:
Please try this patch:
diff --git a/bgpd/bgp_lcommunity.c b/bgpd/bgp_lcommunity.c index 15bf419868..c61684f21e 100644 --- a/bgpd/bgp_lcommunity.c +++ b/bgpd/bgp_lcommunity.c @@ -197,7 +197,7 @@ static void set_lcommunity_string(struct lcommunity *lcom, bool make_json, }
/* 1 space + lcom->size lcom strings + null terminator */ - size_t str_buf_sz = BUFSIZ; + size_t str_buf_sz = BUFSIZ * 10; str_buf = XCALLOC(MTYPE_LCOMMUNITY_STR, str_buf_sz);
for (i = 0; i < lcom->size; i++) {
we are creating a string that is not big enough for the community. We need to look at this a bit more precisely than my multiplication by 10
On Thu, Apr 20, 2023 at 10:33 AM mike tancsa <mike@sentex.net> wrote:
On 4/20/2023 9:30 AM, mike tancsa wrote: > On 4/18/2023 5:28 AM, Kurt Jaeger wrote: >> Hi! >> >>> This caused bgpd to terminate on several boxes this morning: >>> >>> 2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: >>> set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) >>> failed >>> BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting... >>> >>> OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending >>> strange things... ? >>> >>> 7.5.1 is not affected... >>> >>> Any ideas on how to avoid the assert ? >> There's the lookup for community aliases added in that function: >> >> const char *com2alias = >> translate_alias ? bgp_community2alias(lcsb) >> : lcsb; >> len = strlcat(str_buf, com2alias, str_buf_sz); >> >> and it seems to trigger the issue. > > > I am seeing this crash as well. Sadly the patch in the freebsd pr does > not seem to work around the issue. If its an issue of communities, is > there an easy way to strip out communities from peers for now ? > Adding some debugging. It seems to be a large community that is bigger than 1024 bytes. Looking at the pcap,
09:27:57.240613 IP (tos 0xc0, ttl 1, id 22455, offset 0, flags [DF], proto TCP (6), length 1500) 192.168.158.77.179 > 192.168.158.78.64274: Flags [.], cksum 0x96a5 (correct), seq 4567154:4568614, ack 964, win 31913, length 1460: BGP [|BGP] Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 45758 55808 147027 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 64070 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 110.164.235.0/24 <http://110.164.235.0/24> 110.164.234.0/24 <http://110.164.234.0/24> Update Message (2), length: 1021 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 3491 60068 983 38136 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Large Community (32), length: 936, Flags [OTE]: 4635:0:0, 4635:1:906, 4635:1:2906, 4635:1:4515, 4635:1:4594, 4635:1:4641, 4635:1:4760, 4635:1:7979, 4635:1:9253, 4635:1:9293, 4635:1:9304, 4635:1:9908, 4635:1:13335, 4635:1:16265, 4635:1:17924, 4635:1:18013, 4635:1:20940, 4635:1:22822, 4635:1:24429, 4635:1:24482, 4635:1:32590, 4635:1:32934, 4635:1:36692, 4635:1:38008, 4635:1:38819, 4635:1:41378, 4635:1:45753, 4635:1:46489, 4635:1:49544, 4635:1:51847, 4635:1:54574, 4635:1:54994, 4635:1:55720, 4635:1:56059, 4635:1:57724, 4635:1:65021, 4635:1:134823, 4635:1:136907, 4635:1:146961, 24115:0:24115, 24115:1:906, 24115:1:2906, 24115:1:4515, 24115:1:4594, 24115:1:4641, 24115:1:4760, 24115:1:7979, 24115:1:9253, 24115:1:9293, 24115:1:9304, 24115:1:9908, 24115:1:13335, 24115:1:16265, 24115:1:17924, 24115:1:18013, 24115:1:20940, 24115:1:22822, 24115:1:24429, 24115:1:24482, 24115:1:32590, 24115:1:32934, 24115:1:36692, 24115:1:38008, 24115:1:38819, 24115:1:41378, 24115:1:45753, 24115:1:46489, 24115:1:49544, 24115:1:51847, 24115:1:54574, 24115:1:54994, 24115:1:55720, 24115:1:56059, 24115:1:57724, 24115:1:65021, 24115:1:134823, 24115:1:136907, 24115:1:146961 Updated routes: 103.135.103.0/24 <http://103.135.103.0/24> Update Message (2), length: 77 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 262589 14259 16921 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 48050 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 200.108.121.0/24 <http://200.108.121.0/24> Update Message (2), length: 73 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 14, Flags [T]: 174 1239 54174 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Updated routes: 208.21.37.0/24 <http://208.21.37.0/24> Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 1299 6939 6939 398704 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 1 Community (8), length: 8, Flags [OT]: 174:21000, 174:22003 Updated routes: 205.198.124.0/24 <http://205.198.124.0/24>
With some added debugging, it seems its exceeding the size of the buffer zlog_warn("set_lcommunity_string: str_buf_sz=%lu [%s] [%s]\n", str_buf_sz, lcsb, str_buf);
Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:136907] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:146961] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: bgpd/bgp_lcommunity.c:239: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed
_______________________________________________ dev mailing list dev@lists.frrouting.org https://lists.frrouting.org/listinfo/dev
_______________________________________________ dev mailing list dev@lists.frrouting.org https://lists.frrouting.org/listinfo/dev
Thanks for the recreate. I have submitted this PR https://github.com/FRRouting/frr/pull/13341 donald On Thu, Apr 20, 2023 at 3:45 PM mike tancsa <mike@sentex.net> wrote:
BTW, to recreate the problem, you can do it on the actual test instance
do something like
route-map LARGE-OUT permit 10 set large-community 4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:9304 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 24115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 24115:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 24115:1:100000 24115:1:100001 24115:1:100002 exit
And it will crash the sender.
---Mike
On 4/20/2023 10:52 AM, mike tancsa wrote:
Thanks. Just to be clear, this will need to be applied to ALL my bgp speakers first, otherwise it will just propagate the issue to all my ibgp peers and then they will crash.
Is there a way to drop these large communities to protect things ?
---Mike
On 4/20/2023 10:46 AM, Donald Sharp wrote:
Please try this patch:
diff --git a/bgpd/bgp_lcommunity.c b/bgpd/bgp_lcommunity.c index 15bf419868..c61684f21e 100644 --- a/bgpd/bgp_lcommunity.c +++ b/bgpd/bgp_lcommunity.c @@ -197,7 +197,7 @@ static void set_lcommunity_string(struct lcommunity *lcom, bool make_json, }
/* 1 space + lcom->size lcom strings + null terminator */ - size_t str_buf_sz = BUFSIZ; + size_t str_buf_sz = BUFSIZ * 10; str_buf = XCALLOC(MTYPE_LCOMMUNITY_STR, str_buf_sz);
for (i = 0; i < lcom->size; i++) {
we are creating a string that is not big enough for the community. We need to look at this a bit more precisely than my multiplication by 10
On Thu, Apr 20, 2023 at 10:33 AM mike tancsa <mike@sentex.net> wrote:
On 4/20/2023 9:30 AM, mike tancsa wrote:
On 4/18/2023 5:28 AM, Kurt Jaeger wrote:
Hi!
This caused bgpd to terminate on several boxes this morning:
2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting...
OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ?
7.5.1 is not affected...
Any ideas on how to avoid the assert ? There's the lookup for community aliases added in that function:
const char *com2alias = translate_alias ? bgp_community2alias(lcsb) : lcsb; len = strlcat(str_buf, com2alias, str_buf_sz);
and it seems to trigger the issue.
I am seeing this crash as well. Sadly the patch in the freebsd pr does not seem to work around the issue. If its an issue of communities, is there an easy way to strip out communities from peers for now ?
Adding some debugging. It seems to be a large community that is bigger than 1024 bytes. Looking at the pcap,
09:27:57.240613 IP (tos 0xc0, ttl 1, id 22455, offset 0, flags [DF], proto TCP (6), length 1500) 192.168.158.77.179 > 192.168.158.78.64274: Flags [.], cksum 0x96a5 (correct), seq 4567154:4568614, ack 964, win 31913, length 1460: BGP [|BGP] Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 45758 55808 147027 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 64070 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 110.164.235.0/24 110.164.234.0/24 Update Message (2), length: 1021 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 3491 60068 983 38136 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Large Community (32), length: 936, Flags [OTE]: 4635:0:0, 4635:1:906, 4635:1:2906, 4635:1:4515, 4635:1:4594, 4635:1:4641, 4635:1:4760, 4635:1:7979, 4635:1:9253, 4635:1:9293, 4635:1:9304, 4635:1:9908, 4635:1:13335, 4635:1:16265, 4635:1:17924, 4635:1:18013, 4635:1:20940, 4635:1:22822, 4635:1:24429, 4635:1:24482, 4635:1:32590, 4635:1:32934, 4635:1:36692, 4635:1:38008, 4635:1:38819, 4635:1:41378, 4635:1:45753, 4635:1:46489, 4635:1:49544, 4635:1:51847, 4635:1:54574, 4635:1:54994, 4635:1:55720, 4635:1:56059, 4635:1:57724, 4635:1:65021, 4635:1:134823, 4635:1:136907, 4635:1:146961, 24115:0:24115, 24115:1:906, 24115:1:2906, 24115:1:4515, 24115:1:4594, 24115:1:4641, 24115:1:4760, 24115:1:7979, 24115:1:9253, 24115:1:9293, 24115:1:9304, 24115:1:9908, 24115:1:13335, 24115:1:16265, 24115:1:17924, 24115:1:18013, 24115:1:20940, 24115:1:22822, 24115:1:24429, 24115:1:24482, 24115:1:32590, 24115:1:32934, 24115:1:36692, 24115:1:38008, 24115:1:38819, 24115:1:41378, 24115:1:45753, 24115:1:46489, 24115:1:49544, 24115:1:51847, 24115:1:54574, 24115:1:54994, 24115:1:55720, 24115:1:56059, 24115:1:57724, 24115:1:65021, 24115:1:134823, 24115:1:136907, 24115:1:146961 Updated routes: 103.135.103.0/24 Update Message (2), length: 77 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 262589 14259 16921 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 48050 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 200.108.121.0/24 Update Message (2), length: 73 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 14, Flags [T]: 174 1239 54174 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Updated routes: 208.21.37.0/24 Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 1299 6939 6939 398704 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 1 Community (8), length: 8, Flags [OT]: 174:21000, 174:22003 Updated routes: 205.198.124.0/24
With some added debugging, it seems its exceeding the size of the buffer zlog_warn("set_lcommunity_string: str_buf_sz=%lu [%s] [%s]\n", str_buf_sz, lcsb, str_buf);
Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:136907] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:146961] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: bgpd/bgp_lcommunity.c:239: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed
_______________________________________________ dev mailing list dev@lists.frrouting.org https://lists.frrouting.org/listinfo/dev
_______________________________________________ dev mailing listdev@lists.frrouting.orghttps://lists.frrouting.org/listinfo/dev
Hi!
Please try this patch:
Building, will roll it out to one of my test boxes in approx. 10min. Any ideas why it did not kill the frr7.5 instances ?
diff --git a/bgpd/bgp_lcommunity.c b/bgpd/bgp_lcommunity.c index 15bf419868..c61684f21e 100644 --- a/bgpd/bgp_lcommunity.c +++ b/bgpd/bgp_lcommunity.c @@ -197,7 +197,7 @@ static void set_lcommunity_string(struct lcommunity *lcom, bool make_json, }
/* 1 space + lcom->size lcom strings + null terminator */ - size_t str_buf_sz = BUFSIZ; + size_t str_buf_sz = BUFSIZ * 10; str_buf = XCALLOC(MTYPE_LCOMMUNITY_STR, str_buf_sz);
for (i = 0; i < lcom->size; i++) {
we are creating a string that is not big enough for the community. We need to look at this a bit more precisely than my multiplication by 10
On Thu, Apr 20, 2023 at 10:33???AM mike tancsa <mike@sentex.net> wrote:
On 4/20/2023 9:30 AM, mike tancsa wrote:
On 4/18/2023 5:28 AM, Kurt Jaeger wrote:
Hi!
This caused bgpd to terminate on several boxes this morning:
2023/04/18 08:43:47 BGP: bgpd/bgp_lcommunity.c:236: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed BGP: Received signal 6 at 1681800227 (si_addr 0x0); aborting...
OS: FreeBSD 13.1p3. frr was 8.4.x and 8.5. So someone is sending strange things... ?
7.5.1 is not affected...
Any ideas on how to avoid the assert ? There's the lookup for community aliases added in that function:
const char *com2alias = translate_alias ? bgp_community2alias(lcsb) : lcsb; len = strlcat(str_buf, com2alias, str_buf_sz);
and it seems to trigger the issue.
I am seeing this crash as well. Sadly the patch in the freebsd pr does not seem to work around the issue. If its an issue of communities, is there an easy way to strip out communities from peers for now ?
Adding some debugging. It seems to be a large community that is bigger than 1024 bytes. Looking at the pcap,
09:27:57.240613 IP (tos 0xc0, ttl 1, id 22455, offset 0, flags [DF], proto TCP (6), length 1500) 192.168.158.77.179 > 192.168.158.78.64274: Flags [.], cksum 0x96a5 (correct), seq 4567154:4568614, ack 964, win 31913, length 1460: BGP [|BGP] Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 45758 55808 147027 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 64070 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 110.164.235.0/24 110.164.234.0/24 Update Message (2), length: 1021 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 3491 60068 983 38136 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Large Community (32), length: 936, Flags [OTE]: 4635:0:0, 4635:1:906, 4635:1:2906, 4635:1:4515, 4635:1:4594, 4635:1:4641, 4635:1:4760, 4635:1:7979, 4635:1:9253, 4635:1:9293, 4635:1:9304, 4635:1:9908, 4635:1:13335, 4635:1:16265, 4635:1:17924, 4635:1:18013, 4635:1:20940, 4635:1:22822, 4635:1:24429, 4635:1:24482, 4635:1:32590, 4635:1:32934, 4635:1:36692, 4635:1:38008, 4635:1:38819, 4635:1:41378, 4635:1:45753, 4635:1:46489, 4635:1:49544, 4635:1:51847, 4635:1:54574, 4635:1:54994, 4635:1:55720, 4635:1:56059, 4635:1:57724, 4635:1:65021, 4635:1:134823, 4635:1:136907, 4635:1:146961, 24115:0:24115, 24115:1:906, 24115:1:2906, 24115:1:4515, 24115:1:4594, 24115:1:4641, 24115:1:4760, 24115:1:7979, 24115:1:9253, 24115:1:9293, 24115:1:9304, 24115:1:9908, 24115:1:13335, 24115:1:16265, 24115:1:17924, 24115:1:18013, 24115:1:20940, 24115:1:22822, 24115:1:24429, 24115:1:24482, 24115:1:32590, 24115:1:32934, 24115:1:36692, 24115:1:38008, 24115:1:38819, 24115:1:41378, 24115:1:45753, 24115:1:46489, 24115:1:49544, 24115:1:51847, 24115:1:54574, 24115:1:54994, 24115:1:55720, 24115:1:56059, 24115:1:57724, 24115:1:65021, 24115:1:134823, 24115:1:136907, 24115:1:146961 Updated routes: 103.135.103.0/24 Update Message (2), length: 77 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 18, Flags [T]: 174 262589 14259 16921 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 48050 Community (8), length: 8, Flags [OT]: 174:21001, 174:22013 Updated routes: 200.108.121.0/24 Update Message (2), length: 73 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 14, Flags [T]: 174 1239 54174 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 15030 Community (8), length: 8, Flags [OT]: 174:21000, 174:22013 Updated routes: 208.21.37.0/24 Update Message (2), length: 81 Origin (1), length: 1, Flags [T]: IGP AS Path (2), length: 22, Flags [T]: 174 1299 6939 6939 398704 Next Hop (3), length: 4, Flags [T]: 192.168.158.77 Multi Exit Discriminator (4), length: 4, Flags [O]: 1 Community (8), length: 8, Flags [OT]: 174:21000, 174:22003 Updated routes: 205.198.124.0/24
With some added debugging, it seems its exceeding the size of the buffer zlog_warn("set_lcommunity_string: str_buf_sz=%lu [%s] [%s]\n", str_buf_sz, lcsb, str_buf);
Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:136907] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: [SH1YD-SM74V] set_lcommunity_string: str_buf_sz=1024 [24115:1:146961] [4635:0:0 4635:1:906 4635:1:2906 4635:1:4515 4635:1:4594 4635:1:4641 4635:1:4760 4635:1:7979 4635:1:9253 4635:1:9293 4635:1:93 04 4635:1:9908 4635:1:13335 4635:1:16265 4635:1:17924 4635:1:18013 4635:1:20940 4635:1:22822 4635:1:24429 4635:1:24482 4635:1:32590 4635:1:32934 4635:1:36692 4635:1:38008 4635:1:38819 4635:1:41378 4635:1:45753 4635:1:46489 4635:1:49544 4 635:1:51847 4635:1:54574 4635:1:54994 4635:1:55720 4635:1:56059 4635:1:57724 4635:1:65021 4635:1:134823 4635:1:136907 4635:1:146961 24115:0:24115 24115:1:906 24115:1:2906 24115:1:4515 24115:1:4594 24115:1:4641 24115:1:4760 24115:1:7979 2 4115:1:9253 24115:1:9293 24115:1:9304 24115:1:9908 24115:1:13335 24115:1:16265 24115:1:17924 24115:1:18013 24115:1:20940 24115:1:22822 24115:1:24429 24115:1:24482 24115:1:32590 24115:1:32934 24115:1:36692 24115:1:38008 24115:1:38819 2411 5:1:41378 24115:1:45753 24115:1:46489 24115:1:49544 24115:1:51847 24115:1:54574 24115:1:54994 24115:1:55720 24115:1:56059 24115:1:57724 24115:1:65021 24115:1:134823 24115:1:136907 ] Apr 20 10:02:25 cogent-2023 bgpd[43237]: bgpd/bgp_lcommunity.c:239: set_lcommunity_string(): assertion ((unsigned int)len < str_buf_sz) failed
_______________________________________________ dev mailing list dev@lists.frrouting.org https://lists.frrouting.org/listinfo/dev
-- pi@opsec.eu +49 171 3101372 Now what ?
participants (3)
-
Donald Sharp -
Kurt Jaeger -
mike tancsa