[CONTROLLER-283] Cluster has memory issues with jgroups 3.3.1, need to upgrade infinispan to 6.0.2 Final which uses jgroup 3.4.1 Created: 08/Apr/14  Updated: 25/Jul/23  Resolved: 02/Jun/15

Status: Resolved
Project: controller
Component/s: clustering
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: RaviKumar Chiguruvada Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Linux
Platform: PC

Attachments: Text File c1-opendaylight.log    
External issue ID: 682

 Description   

While testing the ODL commands from OSGI console in clustered environment, its found that the data replication is not consistent and the commands are failing most of the times, with an error "Address Not Found", reason being routing table cache is not replicated in the remote node when the command is failed.

While debugging the ClusterManager.java file which uses Infinispan 5.3.0 its observed that infinispan 5.3.0 uses jgroups 3.3.1 version which has issues with memory leaks. In the below form related to similar issue with jgroups in infinispan cluster, searched through some of the jboss forums and found that some of forums suggested to go and upgrade the infinispan to get the latest stable version of jgroups that will fix the issue.

Forum : https://community.jboss.org/thread/229121
jgroups 3.3.1 issues: https://issues.jboss.org/browse/JGRP/fixforversion/12321978

With the above information its better to upgraded the infinispan from 5.3.0 to 6.0.2 which uses jgroups 3.4.1 Final that didn't have major issues with the cluster.

 Comments   
Comment by Giovanni Meo [ 08/Apr/14 ]

RaviKumar, can you explain a bit the test scenario? Would help to understand the issue.

Comment by Prasanna Huddar [ 08/Apr/14 ]

Ravi has upgraded the Infinispan and Jgroups to latest stable release and will submit a patch

Comment by RaviKumar Chiguruvada [ 08/Apr/14 ]

(In reply to Giovanni Meo from comment #1)
> RaviKumar, can you explain a bit the test scenario? Would help to understand
> the issue.

The test scenario is in clusted environment when we try to add/modify a flow for a switch which is connected to the other controller it fails in most of the scenarios and we will see an error message like "Address Not Found", when we checked the replicated cache at that time on the other node its not consistent, it could happen because of the failures in the data replication.

Comment by Giovanni Meo [ 08/Apr/14 ]

(In reply to Prasanna Huddar from comment #2)
> Ravi has upgraded the Infinispan and Jgroups to latest stable release and
> will submit a patch

Prasanna, great to hear you guys have already verified.(In reply to RaviKumar Chiguruvada from comment #3)

> (In reply to Giovanni Meo from comment #1)
> > RaviKumar, can you explain a bit the test scenario? Would help to understand
> > the issue.
>
> The test scenario is in clusted environment when we try to add/modify a flow
> for a switch which is connected to the other controller it fails in most of
> the scenarios and we will see an error message like "Address Not Found",
> when we checked the replicated cache at that time on the other node its not
> consistent, it could happen because of the failures in the data replication.

RaviKumar, ok just for the benefit of the bug maybe a test scenario with the logs would be good to have. I assume anyway you have them.
Thanks,
Giovanni

Comment by RaviKumar Chiguruvada [ 08/Apr/14 ]

Attachment c1-opendaylight.log has been added with description: opendaylight.log with the error

Comment by Colin Dixon [ 02/Jun/15 ]

Fixed: https://git.opendaylight.org/gerrit/#/c/5957/

Generated at Wed Feb 07 19:52:38 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.