[OPNFLWPLUG-391] Controller went out of memory after connecting 40 openflow switches in fully mesh topology Created: 31/Mar/15  Updated: 27/Sep/21  Resolved: 16/Mar/16

Status: Resolved
Project: OpenFlowPlugin
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: SANDEEP GANGADHARAN Assignee: SANDEEP GANGADHARAN
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: File FullyMesh40.py     Zip Archive karaf.zip     Zip Archive log.zip    
External issue ID: 2930

 Description   

1) Run karaf
2) Enable the below features
feature:install odl-openflowplugin-flow-services
feature:install odl-dlux-all
feature:install odl-l2switch-all
feature:install odl-l2switch-switch-ui
feature:install odl-l2switch-switch-rest

3) Using mininet connect 40 switches in fully mesh topology
4) Controller console is unresponsive for few minutes.
5) Then it throws the below exception.
Exception in thread "odl-stat-rpc-oper-thread-0" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2367)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
at java.lang.StringBuilder.append(StringBuilder.java:132)
at java.lang.Throwable.toString(Throwable.java:481)
at org.apache.log4j.OsgiThrowableRenderer.doRender(OsgiThrowableRenderer.java:97)
at org.apache.log4j.OsgiThrowableRenderer.doRender(OsgiThrowableRenderer.java:118)

6) I see all 40 openflow sessions.
netstat -ant | grep 6633 | grep ESTABLISHED | wc -l
40

7) But rest call shows me only 20 nodes.
http://10.11.200.254:8181/restconf/operational/opendaylight-inventory:nodes

 Comments   
Comment by SANDEEP GANGADHARAN [ 31/Mar/15 ]

Attachment log.zip has been added with description: attaching karaf logs

Comment by Kamal Rameshan [ 07/May/15 ]

I hope you are running against Lithium, since there were a couple of memory optimizations done at the Datastore clustering end.

I have run 255 switches, single mininet with a full mesh against 3 controllers and it ran fine.

How much memory is given to the controller. Can you connect jconsole to your nodes and check?

thanks
Kamal

Comment by SANDEEP GANGADHARAN [ 07/May/15 ]

(In reply to Kamal Rameshan from comment #1)
> I hope you are running against Lithium, since there were a couple of memory
> optimizations done at the Datastore clustering end.
>
> I have run 255 switches, single mininet with a full mesh against 3
> controllers and it ran fine.
>
> How much memory is given to the controller. Can you connect jconsole to your
> nodes and check?
>
> thanks
> Kamal

I reported this issue on 31st March. I have not tried it on latest builds. Which build are you running this?

Regards
Sandeep

Comment by Abhijit Kumbhare [ 01/Jun/15 ]

Should not be an issue any longer - needs to be retested.

Comment by SANDEEP GANGADHARAN [ 02/Jun/15 ]

Attachment karaf.zip has been added with description: Karaf logs on build 2175

Comment by Abhijit Kumbhare [ 10/Nov/15 ]

Sandeep,

Can you share the script to create a fully mesh 40 switch topology?

Thanks,
Abhijit

Comment by Sania Zara [ 07/Dec/15 ]

I tried to reproduce this bug using Lithium SR2 and the controller dint crash. I monitored the memory consumption using JConsole and I dint notice any increase in the heap size. I'm attaching the script that I used to create a 40 switch fully mesh topology.

Comment by Sania Zara [ 07/Dec/15 ]

Attachment FullyMesh40.py has been added with description: Script to create a fully mesh topology of 40 switches.

Comment by Jozef Bacigal [ 16/Mar/16 ]

Long time no response, if persist open a new bug please.

Generated at Wed Feb 07 20:32:20 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.