[OPNFLWPLUG-507] OFHandshake thread leak leads to OOM

Mon, 27 Sep 2021 09:01:35 +0000

this is similar to the following fixed bugs:
https://bugs.opendaylight.org/show_bug.cgi?id=2394
https://bugs.opendaylight.org/show_bug.cgi?id=2429

The 2429 issue has CI that is passing, so this is another flavor. The quickest way to reproduce this would be to install odl-openflowplugin-southbound and then execute this linux tool from another system:

for i in

{1..40000}

; do nc -w 0 <controller> 6633; done;

that will very quickly (to make this a fast reproduction) open and close
40k TCP connections on port 6633 to the controller.

you can get a thread count with this command:

ps --no-headers -o nlwp <controller PID>

after some time, you can see that the TCP connections are not maintained on the controller (which is what we expect):

netstat -na | grep 6633

after aprox 4-5m you should see an OutOfMemoryError.

with a debugger, you can see that OFHandshake threads are hanging around.

if you want to be nicer with the test, you can use a non-zero value for the 'nc' command above. Using 1, you can get one TCP connection open/closed every second and watch the thread count increase.

This was done on the 6/17 RC1 build for Lithium.

OpenDaylight JIRA

[OPNFLWPLUG-507] OFHandshake thread leak leads to OOM