Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak seen in bgpd on FRR 8.5.1 after stress test of announcing/withdrawing routes #15459

Closed
2 tasks done
saiarcot895 opened this issue Feb 29, 2024 · 4 comments
Closed
2 tasks done
Milestone

Comments

@saiarcot895
Copy link

Description

On FRR 8.5.1, after repeatedly running a stress test where 6400 routes are repeatedly announced and withdrawn, there is a memory leak that builds up in a short amount of time. Specifically, about 11MB is taken in bgpd after each loop. This results in the device running out of memory eventually.

It appears that the memory increase appears to be coming from the number of community objects increasing. The number of community objects increased by 50k-100k after every loop.

This may be partially related to #14828, which saw a memory leak in the large-community objects.

Version

str2-7260cx3-acs-9# show version
FRRouting 8.5.1 (str2-7260cx3-acs-9) on Linux(5.10.0-23-2-amd64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--libexecdir=${prefix}/lib/x86_64-linux-gnu' '--disable-maintainer-mode' '--localstatedir=/var/run/frr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--with-vtysh-pager=/usr/bin/pager' '--libdir=/usr/lib/x86_64-linux-gnu/frr' '--with-moduledir=/usr/lib/x86_64-linux-gnu/frr/modules' '--disable-dependency-tracking' '--disable-rpki' '--disable-scripting' '--enable-pim6d' '--with-libpam' '--enable-doc' '--enable-doc-html' '--enable-snmp' '--enable-fpm' '--disable-protobuf' '--disable-zeromq' '--enable-ospfapi' '--enable-bgp-vnc' '--enable-multipath=256' '--enable-user=frr' '--enable-group=frr' '--enable-vty-group=frrvty' '--enable-configfile-mask=0640' '--enable-logfile-mask=0640' 'build_alias=x86_64-linux-gnu' 'PYTHON=python3'

How to reproduce

Set up a topology with 4 upstream peers in a different ASN. Have all 4 upstream peers advertise the same 6400 prefixes to this device.

Then, from each of the peers, withdraw and re-announce all of the routes. This process of withdraw and announce is considered one loop. Repeat for 10-100 times.

Expected behavior

There is no memory increase from this.

Actual behavior

There is a steady 10-11MB increase after each loop.

Additional context

No response

Checklist

  • I have searched the open issues for this bug.
  • I have not included sensitive information in this report.
@saiarcot895 saiarcot895 added the triage Needs further investigation label Feb 29, 2024
@ton31337
Copy link
Member

ton31337 commented Mar 1, 2024

Closing as this is sort of duplicate to #14828.

@ton31337 ton31337 closed this as completed Mar 1, 2024
@donaldsharp
Copy link
Member

I'm going to reopen this issue. It is not clear to me yet what the problem is and I would like to get a bit more data before fully marking this as a duplicate( although I would tend to agree at a very first glance, just want more data ). @saiarcot895 We will need actual configuration and before and after show memory outputs for us to debug this further.

@donaldsharp donaldsharp reopened this Mar 1, 2024
@saiarcot895
Copy link
Author

Attached a tarball containing the simplified config, show memory output after each bgp shut/no shut, and some information about the leaks that LSAN detected (I suspect this may have been cut off at some point, since the leak summary isn't here). The config does contain some referenced, but undefined route maps, but I'm thinking since those are never defined, they won't cause an issue for the purposes of finding the memory leak here.

frrDebug.tar.gz

@IvayloJ
Copy link

IvayloJ commented Mar 2, 2024

Is it possible to remove the clause additive from the route maps (let it overwrite community strings) and repeat the tests ?

Also in case you did not notice what I wrote you in #14828 , please take a look.

donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 2, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 3, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 4, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
@ton31337 ton31337 added bgp bug and removed triage Needs further investigation labels Mar 5, 2024
@ton31337 ton31337 added this to the 10.0 milestone Mar 5, 2024
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 5, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 6, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 7, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 7, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 8, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
StormLiangMS pushed a commit to sonic-net/sonic-buildimage that referenced this issue Mar 8, 2024
…some-cases (#18246)

Why I did it
FRR issue
FRRouting/frr#15459
Add patch for FRR PR for 202305
FRRouting/frr#15466

Work item tracking
Microsoft ADO (number only):
26876083
How I did it
Add patch for FRR PR
FRRouting/frr#15466

How to verify it
Run the original case locally
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 8, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 13, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 13, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
donaldsharp added a commit to donaldsharp/frr that referenced this issue Mar 13, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
mergify bot pushed a commit that referenced this issue Mar 14, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: #15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
(cherry picked from commit addff17)
mergify bot pushed a commit that referenced this issue Mar 14, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: #15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
(cherry picked from commit addff17)
mergify bot pushed a commit that referenced this issue Mar 14, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: #15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
(cherry picked from commit addff17)

# Conflicts:
#	bgpd/bgp_route.c
mergify bot pushed a commit that referenced this issue Mar 14, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: #15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
(cherry picked from commit addff17)

# Conflicts:
#	bgpd/bgp_route.c
mergify bot pushed a commit that referenced this issue Mar 14, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: #15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
(cherry picked from commit addff17)

# Conflicts:
#	bgpd/bgp_route.c
ton31337 pushed a commit to opensourcerouting/frr that referenced this issue Mar 18, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
ton31337 pushed a commit to opensourcerouting/frr that referenced this issue Mar 18, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
ton31337 pushed a commit to opensourcerouting/frr that referenced this issue Mar 18, 2024
Customer has this valgrind trace:

Direct leak of 2829120 byte(s) in 70728 object(s) allocated from:
  0 in community_new ../bgpd/bgp_community.c:39
  1 in community_uniq_sort ../bgpd/bgp_community.c:170
  2 in route_set_community ../bgpd/bgp_routemap.c:2342
  3 in route_map_apply_ext ../lib/routemap.c:2673
  4 in subgroup_announce_check ../bgpd/bgp_route.c:2367
  5 in subgroup_process_announce_selected ../bgpd/bgp_route.c:2914
  6 in group_announce_route_walkcb ../bgpd/bgp_updgrp_adv.c:199
  7 in hash_walk ../lib/hash.c:285
  8 in update_group_af_walk ../bgpd/bgp_updgrp.c:2061
  9 in group_announce_route ../bgpd/bgp_updgrp_adv.c:1059
 10 in bgp_process_main_one ../bgpd/bgp_route.c:3221
 11 in bgp_process_wq ../bgpd/bgp_route.c:3221
 12 in work_queue_run ../lib/workqueue.c:282

The above leak detected by valgrind was from a screenshot so I copied it
by hand.  Any mistakes in line numbers are purely from my transcription.
Additionally this is against a slightly modified 8.5.1 version of FRR.
Code inspection of 8.5.1 -vs- latest master shows the same problem
exists.  Code should be able to be followed from there to here.

What is happening:

There is a route-map being applied that modifes the outgoing community
to a peer.  This is saved in the attr copy created in
subgroup_process_announce_selected.  This community pointer is not
interned.  So the community->refcount is still 0.  Normally when
a prefix is announced, the attr and the prefix are placed on a
adjency out structure where the attribute is interned.  This will
cause the community to be saved in the community hash list as well.
In a non-normal operation when the decision to send is aborted after
the route-map application, the attribute is just dropped and the
pointer to the community is just dropped too, leading to situations
where the memory is leaked.  The usage of bgp suppress-fib would
would be a case where the community is caused to be leaked.
Additionally the previous commit where an unsuppress-map is used
to modify the outgoing attribute but since unsuppress-map was
not considered part of outgoing policy the attribute would be dropped as
well.  This pointer drop also extends to any dynamically allocated
memory saved by the attribute pointer that was not interned yet as well.

So let's modify the return case where the decision is made to
not send the prefix to the peer to always just flush the attribute
to ensure memory is not leaked.

Fixes: FRRouting#15459
Signed-off-by: Donald Sharp <sharpd@nvidia.com>
StormLiangMS pushed a commit to sonic-net/sonic-buildimage that referenced this issue Apr 15, 2024
Why I did it
Porting fix for FRRouting/frr#15459

Adding patch for PR FRRouting/frr#15466

Work item tracking
Microsoft ADO (number only):
How I did it
Ported the fix FRRouting/frr#15466 to 8.5.1

How to verify it
Running the test_stress_route and ensure no memory leak
StormLiangMS pushed a commit to sonic-net/sonic-buildimage that referenced this issue Apr 18, 2024
Porting PR #18272 to 202311.

Why I did it
Porting fix for FRRouting/frr#15459

Adding patch for PR FRRouting/frr#15466

Work item tracking
Microsoft ADO (number only):
How I did it
Ported the fix FRRouting/frr#15466 to 8.5.1

How to verify it
Running the test_stress_route and ensure no memory leak
StormLiangMS pushed a commit to sonic-net/sonic-buildimage that referenced this issue Apr 29, 2024
Why I did it
Upgrading FRR 8.5.4 to include latest fixes.

Work item tracking
Microsoft ADO (number only):
How I did it
New patches that were added:

Patch	FRR Pull request	Issue fixed
0024-lib-use-snmp-s-large-fd-sets-for-agentx.patch	FRRouting/frr#13396	FRRouting/frr#14143
0025-bgp-community-memory-leak-fix.patch	FRRouting/frr#15466	FRRouting/frr#15459
0026-bgp-fib-suppress-announce-fix.patch	FRRouting/frr#15634	FRRouting/frr#15626
0027-lib-Do-not-convert-EVPN-prefixes-into-IPv4-IPv6-if-n.patch	FRRouting/frr#15418	FRRouting/frr#14419
Removed patches:

Patch	Upstream FRR commit that is present in 8.5.4
0019-zebra-Abstract-dplane_ctx_route_init-to-init-route-w.patch	FRRouting/frr@3f01977
0020-zebra-Fix-crash-when-dplane_fpm_nl-fails-to-process-.patch	FRRouting/frr@fe5f624
0022-bgpd-Don-t-read-the-first-byte-of-ORF-header-if-we-a.patch	FRRouting/frr@3515178
0023-bgpd-Make-sure-we-have-enough-data-to-read-two-bytes.patch	FRRouting/frr@460ee93
0024-bgpd-Do-not-process-NLRIs-if-the-attribute-length-is.patch	FRRouting/frr@f291f1e
0025-bgpd-Use-treat-as-withdraw-for-tunnel-encapsulation-.patch	FRRouting/frr@8a4a88c
0026-zebra-Add-encap-type-when-building-packet-for-FPM.patch	FRRouting/frr@f0f7b28
0028-bgpd-Check-mandatory-attributes-more-carefully-for-U.patch	FRRouting/frr@21418d6
0029-bgpd-Handle-MP_REACH_NLRI-malformed-packets-with-ses.patch	FRRouting/frr@30b5c2a
0030-bgpd-Treat-EOR-as-withdrawn-to-avoid-unwanted-handli.patch	FRRouting/frr@01f232c
0031-bgpd-Ignore-handling-NLRIs-if-we-received-MP_UNREACH.patch	FRRouting/frr@a0c4ec2
0032-zebra-Fix-fpm-multipath-encap-addition.patch	FRRouting/frr@10a9a5f
Realigned patches:

Old Patch	New patch
0005-Add-support-of-bgp-l3vni-evpn.patch	0005-Add-support-of-bgp-l3vni-evpn.patch
0021-zebra-remove-duplicated-nexthops-when-sending-fpm-msg.patch	0019-zebra-remove-duplicated-nexthops-when-sending-fpm-msg.patch
0027-zebra-Fix-non-notification-of-better-admin-won.patch	0020-zebra-Fix-non-notification-of-better-admin-won.patch
Disable-ipv6-src-address-test-in-pceplib.patch	0021-Disable-ipv6-src-address-test-in-pceplib.patch
cross-compile-changes.patch	0022-cross-compile-changes.patch
0033-zebra-The-dplane_fpm_nl-return-path-leaks-memory.patch	0023-zebra-The-dplane_fpm_nl-return-path-leaks-memory.patch
How to verify it
Running sonic-mgmt test suite.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants