[OVSDB-352] potential memory leak in JsonRpcEndpoint Created: 05/Jul/16  Updated: 24/Jul/16  Resolved: 24/Jul/16

Status: Resolved
Project: ovsdb
Component/s: Library
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Ashwin P Assignee: Anil Vishnoi
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: Text File Bug6157_exception.txt     Text File Bug6157_reproducer.txt    
External issue ID: 6157
Priority: High

 Description   

Summary:
If the same OVSDB client repeatedly makes requests to read the OVSDB configuration, the methodContext hashmap keeps growing.

Steps to reproduce:

  • add multiple ports/interfaces (>30) to OVSDB (using ovs-vsctl) so that the increase in memory consumption is easy to detect.
  • using a client, make requests to read the configuration in OVSDB repeatedly.
  • observe the memory growth of this test program using a tool like jstack.

Some questions:

  • Is there a recommendation that the same OVSDB client should not be used for multiple requests?
  • Is there a way to reinitialize JsonRpcEndpoint for the same OVSDB client?


 Comments   
Comment by Anil Vishnoi [ 06/Jul/16 ]

Hi Ashwin,

Can you please give some more details about "- using a client, make requests to read the configuration in OVSDB repeatedly.". How are you making the request? are you using RESTCONF to fetch the data from data store? Or are you using explicit library calls for that ?

Comment by Ashwin P [ 07/Jul/16 ]

@Anil:
> How are you making the request? are you using RESTCONF to fetch the data from data store? Or are you using explicit library calls for that ?

Library calls [an example -- repeated ovsdb.lib.operations.TransactionBuilder.execute calls that execute a select operation on the Port table (containing > 30 ports) using an OvsdbClient]. I believe the problem is that there are only "put" invocations on the methodContext hash-map in JsonRpcEndpoint. There is no piece of code that removes elements from the hash-map after the request is processed.

Comment by Ashwin P [ 10/Jul/16 ]

Attachment #1: Bug6157 reproducer JUnit test

Comment by Ashwin P [ 10/Jul/16 ]

Attachment Bug6157_reproducer.txt has been added with description: Bug6157 reproducer

Comment by Ashwin P [ 10/Jul/16 ]

Attachment #2: Bug6157 OutOfMemory exception after running the reproducer.

Comment by Ashwin P [ 10/Jul/16 ]

Attachment Bug6157_exception.txt has been added with description: Bug6157 OutOfMemory exception after running the reproducer

Comment by Ashwin P [ 10/Jul/16 ]

@Anil:

I have attached two files. One of them is a reproducer. It simply creates a select OVSDB transaction using ODL library calls on the OVSDB Port table, and executes it 300 times.

If you run the JUnit test after adding a large number of ports (~ 300) using "ovs-vsctl add-port <BRIDGE_NAME> <PORT_NAME>" to OVSDB, I believe you should be able to observe the OutOfMemoryError exception (second attachment). `ps aux` output shows continuous increase in the memory consumption when the test is run. I used the Beryllium-1.2.3-SR2 stable released version of ODL for running these tests.

I'd really appreciate if the fix for this Bug can be included in the forthcoming SR3 release.

Comment by Ashwin P [ 15/Jul/16 ]

The following change may be a potential workaround/fix:
-----a/library/impl/src/main/java/org/opendaylight/ovsdb/lib/jsonrpc/JsonRpcEndpoint.java
+++ b/library/impl/src/main/java/org/opendaylight/ovsdb/lib/jsonrpc/JsonRpcEndpoint.java
@@ -128,6 +128,8 @@ public class JsonRpcEndpoint

{ return; }

+ methodContext.remove(response.get("id").asText());
+
if (ListenableFuture.class == returnCtxt.getMethod().getReturnType()) {
TypeToken<?> retType = TypeToken.of(
returnCtxt.getMethod().getGenericReturnType())


@Anil: If you could please provide an ETA for the fix for this Bug, I'd really appreciate it. Will the fix make it to the forthcoming SR3 release?

Comment by Anil Vishnoi [ 24/Jul/16 ]

Master : https://git.opendaylight.org/gerrit/42211
Stable/beryllium : https://git.opendaylight.org/gerrit/42113

Generated at Wed Feb 07 20:36:10 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.