https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/NETCONF-88 netconf <p>When a device is mounted on NETCONF through RESTCONF, a bunch of memory is leaked. Initially it is about 0.9 MB but with each "mount" request the leak increases by about 8 KB. The "initial" leak caused by a first device mount depends on total mount operations executed and is not affected by device unmounts.</p> <p>Additionally, device unmounts also leak memory but they don't add anything to the amount leaked by each mount. The amount of memory leaked by a device unmount depends on the count of devices currently mounted: When only one device is present and it is deleted, the leaked memory is roughly 300 KB. For each 100 devices mounted at the time of the deletion unmount, the leak increases by roughly 800 KB.</p> <p>To reproduce the basic form of the leak use the attached script. It mounts and then unmounts a device and reports memory leaked so far and memory leaked by the last mount/unmount pair and whether any errors encountered. You can specify a repeat count on the commandline which defaults to 1.</p> <p>This bug is present in both, Lithium and Beryllium.</p> <p>Operating System: All<br/> Platform: All</p> NETCONF-88

Memory leak when mounting/unmounting devices

Bug Resolved Done Robert Varga Jozef Behran Thu, 22 Oct 2015 09:49:17 +0000 Fri, 15 Mar 2019 22:22:16 +0000 Wed, 13 Jan 2016 09:51:25 +0000 netconf 0 4 <p>Attachment leakwatch.sh has been added with description: Tool to bombard ODL Netconf with device mount/unmount requests via Restconf</p> <p>Additional information is that the leak occurs even when the device is not actually there (connecting fails). No testtool is thus necessary and it does not matter what port is configured for the device.</p> <p>Also this bug occurs with the default configuration of ODL. Just install odl-restconf and odl-netconf-connector-all.</p> <p>Use this utility to mount a bunch of devices and then unmount them. The default size of the bunch is 100, if you want a different size, state it on the commandline.</p> <p>Attachment multimount.sh has been added with description: Utility to mount and then unmount a bunch of devices</p> <p>The change <a href="https://git.opendaylight.org/gerrit/#/c/28709/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/28709/</a> was giving some hope of improving matters but it turns out nothing was changed by it.</p> <p>UPDATE: It turned out the behavior of the bug is different than originally reported. The leak actually roughly depends on the count of devices currently mounted but the dependency is kind of "lagging" and also timing dependent, giving the illusion of a behavior described in the original report, especially when the mount-unmount operations are run in a quick succession. Additionally, it seems to me to be also time dependent (running mount-unmount multiple times on a quick succession tends to leak more memory than letting ODL to "cool down" after each mount-unmount pair for a few tens of seconds). Additionally, occassionally (very rarely) the leak is much bigger than what the count of mounted devices suggests (2-3 times larger) and occassionally (very rarely though more commonly than the previous case) there is no memory leak at all (but all I can confirm is that the memory was not growing at that time but it might be that the leak did not happen and it also might be that memory did leak but something unrelated was garbage collected, hiding the leak).</p> <p>Seems to only happen when configuring via restconf with config-subsystem loopback, with initial configs from testtool I can get 5k devices without any issues.<br/> Most likely this is related to the way restconf works(read contents of entire list to check if configuration is present, and then insert config)</p> <p>&gt; Most likely this is related to the way restconf works<br/> &gt; (read contents of entire list to check if configuration<br/> &gt; is present, and then insert config)</p> <p>And stick a reference to that "entire list" somewhere where it stays reachable even after restconf is done with that "entire list" so that the garbage collector cannot collect the "entire list" and they keep accumulating.</p> <p>Maybe memory profiling will give more answers to this.</p> <p>Be: <a href="https://git.opendaylight.org/gerrit/30487" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/30487</a></p> <p>This affects Lithium, too, so should be backported.</p> <p>Further fix: <a href="https://git.opendaylight.org/gerrit/#/c/31075/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/31075/</a></p> <p>Tested and confirmed to be gone.</p> <p>These patches need to be merged to kill the bug (in the indicated order):</p> <p><a href="https://git.opendaylight.org/gerrit/31808" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/31808</a><br/> <a href="https://git.opendaylight.org/gerrit/31858" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/31858</a></p> <p>Li: <a href="https://git.opendaylight.org/gerrit/32277" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/32277</a></p> Development External issue ID 4514 External issue URL Issue Type ODL SR Target Milestone Rank 0|i01wdr: