[OPNFLWPLUG-539] memory leak with single switch connected having a lot of flows (50k) Created: 25/Aug/15  Updated: 27/Sep/21  Resolved: 12/Dec/17

Status: Resolved
Project: OpenFlowPlugin
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Jamo Luhrsen Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


External issue ID: 4186

 Description   

This is occurring when using the karaf feature "odl-openflowplugin-flow-services-ui", but NOT coming when using the re-designed plugin "old-openflowplugin-flow-services-ui-li".

Using the Lithium release bits.
https://nexus.opendaylight.org/content/groups/public/org/opendaylight/integration/distribution-karaf/0.3.0-Lithium/distribution-karaf-0.3.0-Lithium.zip

Steps to recreate:

  • start karaf and install odl-openflowplugin-flow-services-ui
  • connect one OVS switch (ovs version 2.0.2 was used when creating this bug)
  • configure 50k flows via NB REST API.
    I used our robot test [0] to do this, which in turn calls a python script [1], but
    it's possible that any method to configure 50k flows will reproduce this. If you
    want to try the robot method, edit the robot script to have "FATAL ERROR" after
    line 44 which configures the flows. That forces robot to quit after installing the
    flows. The pybot command line that I used is here [2]
  • wait.

after some time (maybe many hours) you will see messages like this on the karaf console:

java.lang.OutOfMemoryError: Java heap space
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.growArray(ForkJoinPool.java:1090)
Exception in thread "Timer-0"
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Timer-0"
Exception in thread "qtp1473670433-83" Exception in thread "Karaf Console Local for user karaf" Exception in thread "qtp1473670433-80" at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1978)
Exception in thread "pool-38-thread-1" Exception in thread "qtp1473670433-1550" Exception in thread "qtp1473670433-85" Exception in thread "RMI TCP Connection(idle)" Exception in thread "sshd-SshServer[27195dd9]-timer-thread-1" Exception in thread "pool-34-thread-1" Exception in thread "RMI TCP Connection(idle)" Exception in thread "qtp1473670433-81" at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
Exception in thread "Timer-1" Exception in thread "RMI TCP Connection(idle)" Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space

I neglected to collect the karaf.log, but can do so if it's needed.

[0] https://github.com/opendaylight/integration/blob/master/test/csit/suites/openflowplugin/Performance_Flow_Config/070_Stats_Collection_100k_WithMininetRestart.robot
[1] https://github.com/opendaylight/integration/blob/master/test/csit/libraries/ScaleClient.py
[2] pybot -d /tmp/RobotResults/ -c critical -e exclude -v BUNDLEFOLDER:distribution-karaf-0.4.0-SNAPSHOT -v WORKSPACE:/tmp -v NEXUSURL_PREFIX:http://nexus01.dfw.opendaylight.org:8081/nexus -v CONTROLLER:<your controller ip> -v CONTROLLER_USER:fedora -v MININET:<your mininet vm ip> -v MININET1: -v MININET2: -v MININET_USER:ubuntu -v DEFAULT_LINUX_PROMPT:\$ -v USER_HOME:/home/jluhrsen -L TRACE Performance_Flow_Config/070_Stats_Collection_100k_WithMininetRestart.robot



 Comments   
Comment by Anil Vishnoi [ 12/Dec/17 ]

No activity on the bug and lithium branch is not supported now. Please re-open the bug if you hit the issue with the latest carbon/nitrgeon/oxygen branch.

Generated at Wed Feb 07 20:32:45 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.