Operating System: Linux
Platform: Other

Attachment karaf.log has been added with description: karaf logs

reproduced an OOM (assuming similar to this) with recent boron distro. Loaded
odl-netvirt-openstack and ran the following shell script:

for i in

;do ovs-vsctl set-manager tcp:127.0.0.1:6640; sleep 0.1; ovs-vsctl add-br brjamo$i; sleep 0.1; ovs-vsctl set-controller brjamo$i tcp:127.0.0.1:6633; sleep 0.1; ovs-vsctl del-br brjamo$i; ovs-vsctl del-manager; sleep 0.1; done

VPNSERVICE has dropped out of Boron. Retargeting this bug for Carbon instead:

https://lists.opendaylight.org/pipermail/vpnservice-dev/2016-August/000302.html

Hey,

I think that is not right. We confirmed on beryllium and on boron.

Beryllium -> vpnservice
Boron -> odl-netvirt-openstack.

So it has nothing to do with vpnservice as such or? So we should leave it as a blocker to Boron.

(In reply to Nikolas Hermanns from comment #3)
> Hey,
>
> I think that is not right. We confirmed on beryllium and on boron.
>
> Beryllium -> vpnservice
> Boron -> odl-netvirt-openstack.
>
> So it has nothing to do with vpnservice as such or? So we should leave it as
> a blocker to Boron.

agreed

Okay, changing the target back to Boron as per the discussion.

Who is handling this?

Is there an ETA for this bug and someone assigned to fix?

Reassigning to OVSDB as per this discussion: https://lists.opendaylight.org/pipermail/netvirt-dev/2016-September/001426.html

This bug is being reassigned to critical since testing with new GC is finished and it seems to resolve the issue for the testing as per this email thread:

https://lists.opendaylight.org/pipermail/release/2016-September/008119.html

I've run a 5 hour test with the commands in comment #1, and with the max memory set to 8G. There is no OOM crash. It appears that we do end up consuming
aprox 3.5-4.5G of heap while this is happening, so I am thinking that the "workaround" for this issue is to ensure the max mem is set to 5G or larger.

Hey,

thanks for the investigation. However I cannot really believe what takes 5GB here. it is a lot of ram space. Are we sure that after 10 hours we do not have 6GB of mem consumption? ODL is meant to run for ages

Do we state somewhere that together with our feature we recommend to set it to ???GB or ram? We should change it then to min 5GB I guess.

(In reply to Nikolas Hermanns from comment #11)
> Hey,
>
> thanks for the investigation. However I cannot really believe what takes 5GB
> here. it is a lot of ram space. Are we sure that after 10 hours we do not
> have 6GB of mem consumption? ODL is meant to run for ages

I was running a debugger and visually checking the heap usage and it did
appear to level off around the 3.5-4.5 G area. But, a longer lived test
might be in order, like you suggest.

> Do we state somewhere that together with our feature we recommend to set it
> to ???GB or ram? We should change it then to min 5GB I guess.

I don't think this is officially documented anywhere in upstream ODL. I think
some vendors may have their own recommended specs though.

Jamo/Nikolas,

I believe the conclusion was to increase the heap space and that should resolve the issue. Regarding the heap usage i think this is something consumer project need to check by digging in the heap dumps to see what's eating so much of memory.

I am closing this issue as of now, but if there is anything specific that needs to be addressed through this bug, please feel free to open it back.