https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/OVSDB-367 ovsdb <p>Functest and yardstick are opnfv testing tools. They creating a lot of neutron objects and deleting them again. Especially floating ips, which are not yet implemented by the vpnservice.</p> <p>When running this test environments it happens that ovsdb crashes cause of GC overhaed limit exceeded. Check the karaf log for more information.</p> <p>Operating System: Linux<br/> Platform: Other</p> OVSDB-367

java.lang.OutOfMemoryError: GC overhead limit exceeded in org.opendaylight.ovsdb

Bug Resolved Cannot Reproduce Unassigned Nikolas Hermanns Tue, 23 Aug 2016 07:15:33 +0000 Thu, 19 Oct 2017 21:26:55 +0000 Thu, 2 Feb 2017 20:04:30 +0000 unspecified Other 0 6 <p>Attachment karaf.log has been added with description: karaf logs</p> <p>reproduced an OOM (assuming similar to this) with recent boron distro. Loaded<br/> odl-netvirt-openstack and ran the following shell script:</p> <p>for i in </p> {1..1000} <p>;do ovs-vsctl set-manager tcp:127.0.0.1:6640; sleep 0.1; ovs-vsctl add-br brjamo$i; sleep 0.1; ovs-vsctl set-controller brjamo$i tcp:127.0.0.1:6633; sleep 0.1; ovs-vsctl del-br brjamo$i; ovs-vsctl del-manager; sleep 0.1; done</p> <p>VPNSERVICE has dropped out of Boron. Retargeting this bug for Carbon instead:</p> <p><a href="https://lists.opendaylight.org/pipermail/vpnservice-dev/2016-August/000302.html" class="external-link" target="_blank" rel="nofollow noopener">https://lists.opendaylight.org/pipermail/vpnservice-dev/2016-August/000302.html</a></p> <p>Hey,</p> <p>I think that is not right. We confirmed on beryllium and on boron.</p> <p>Beryllium -&gt; vpnservice<br/> Boron -&gt; odl-netvirt-openstack.</p> <p>So it has nothing to do with vpnservice as such or? So we should leave it as a blocker to Boron.</p> <p>(In reply to Nikolas Hermanns from comment #3)<br/> &gt; Hey,<br/> &gt; <br/> &gt; I think that is not right. We confirmed on beryllium and on boron.<br/> &gt; <br/> &gt; Beryllium -&gt; vpnservice<br/> &gt; Boron -&gt; odl-netvirt-openstack.<br/> &gt; <br/> &gt; So it has nothing to do with vpnservice as such or? So we should leave it as<br/> &gt; a blocker to Boron.</p> <p>agreed</p> <p>Okay, changing the target back to Boron as per the discussion.</p> <p>Who is handling this?</p> <p>Is there an ETA for this bug and someone assigned to fix?</p> <p>Reassigning to OVSDB as per this discussion: <a href="https://lists.opendaylight.org/pipermail/netvirt-dev/2016-September/001426.html" class="external-link" target="_blank" rel="nofollow noopener">https://lists.opendaylight.org/pipermail/netvirt-dev/2016-September/001426.html</a></p> <p>This bug is being reassigned to critical since testing with new GC is finished and it seems to resolve the issue for the testing as per this email thread:</p> <p><a href="https://lists.opendaylight.org/pipermail/release/2016-September/008119.html" class="external-link" target="_blank" rel="nofollow noopener">https://lists.opendaylight.org/pipermail/release/2016-September/008119.html</a></p> <p>I've run a 5 hour test with the commands in comment #1, and with the max memory set to 8G. There is no OOM crash. It appears that we do end up consuming<br/> aprox 3.5-4.5G of heap while this is happening, so I am thinking that the "workaround" for this issue is to ensure the max mem is set to 5G or larger.</p> <p>Hey,</p> <p>thanks for the investigation. However I cannot really believe what takes 5GB here. it is a lot of ram space. Are we sure that after 10 hours we do not have 6GB of mem consumption? ODL is meant to run for ages <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Do we state somewhere that together with our feature we recommend to set it to ???GB or ram? We should change it then to min 5GB I guess.</p> <p>(In reply to Nikolas Hermanns from comment #11)<br/> &gt; Hey,<br/> &gt; <br/> &gt; thanks for the investigation. However I cannot really believe what takes 5GB<br/> &gt; here. it is a lot of ram space. Are we sure that after 10 hours we do not<br/> &gt; have 6GB of mem consumption? ODL is meant to run for ages <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>I was running a debugger and visually checking the heap usage and it did <br/> appear to level off around the 3.5-4.5 G area. But, a longer lived test<br/> might be in order, like you suggest.</p> <p>&gt; Do we state somewhere that together with our feature we recommend to set it<br/> &gt; to ???GB or ram? We should change it then to min 5GB I guess.</p> <p>I don't think this is officially documented anywhere in upstream ODL. I think<br/> some vendors may have their own recommended specs though.</p> <p>Jamo/Nikolas,</p> <p>I believe the conclusion was to increase the heap space and that should resolve the issue. Regarding the heap usage i think this is something consumer project need to check by digging in the heap dumps to see what's eating so much of memory. </p> <p>I am closing this issue as of now, but if there is anything specific that needs to be addressed through this bug, please feel free to open it back.</p> Development External issue ID 6508 External issue URL Issue Type ODL SR Target Milestone Priority Rank 0|i022an: