[OVSDB-435] New OOM due to more TX leaks seen in the nova-boot scenario Created: 30/Nov/17  Updated: 06/Dec/17  Resolved: 30/Nov/17

Status: Resolved
Project: ovsdb
Component/s: None
Affects Version/s: Carbon-SR3
Fix Version/s: Carbon-SR3, Nitrogen-SR2, Oxygen

Type: Bug Priority: Highest
Reporter: Michael Vorburger Assignee: Stephen Kitt
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File controller-0_open-transactions.txt     Text File controller-1_open-transactions.txt     Text File controller-2_open-transactions.txt    
Issue Links:
Blocks
blocks NETVIRT-878 CSIT should help to detect possible m... In Progress
is blocked by NETVIRT-985 java.lang.OutOfMemoryError: Java heap... Resolved
Cloners
is cloned by GENIUS-102 New OOM due to more TX leaks seen in ... Resolved
is cloned by OPNFLWPLUG-961 New OOM due to more TX leaks seen in ... Resolved

 Description   

Following the (many!) fixes that we did in NETVIRT-985, a new round of testing on the latest stable/carbon using the "nova-boot scenario" has hit an OOM again. A new "trace:transaction" output reveals more TX leaks which we need to plug; see attached.

The biggest one with 273x open newReadOnlyTransaction likely causing the new OOM this time appears in (1) ovsdb; https://git.opendaylight.org/gerrit/#/c/65567/ seems to not have helped for that one - curious... or we made a mess in downstream porting?!

Filing this in JIRA project ovsdb due to above; but the attached shows a number of other leaks in (2) openflowplugin and (3) genius.lockmanager & (4) genius.interfacemanager (x90!) which we need to plug as part of this JIRA as well.

 Comments   
Comment by Stephen Kitt [ 30/Nov/17 ]

https://git.opendaylight.org/gerrit/66092 fixes the main leak in Oxygen (backports incoming...).

Comment by Stephen Kitt [ 30/Nov/17 ]

The other leaks are:

  • 34× org.opendaylight.ovsdb.southbound.reconciliation.configuration.BridgeConfigReconciliationTask.reconcileConfiguration(BridgeConfigReconciliationTask.java:73) (fixed by https://git.opendaylight.org/gerrit/65569)
  • 2× org.opendaylight.ovsdb.hwvtepsouthbound.HwvtepSouthboundProvider.init(HwvtepSouthboundProvider.java:78) (related to long-running transactions in the transaction invoker)
  • 2× org.opendaylight.ovsdb.southbound.SouthboundProvider.init(SouthboundProvider.java:79) (likewise)
  • 1× org.opendaylight.ovsdb.southbound.reconciliation.ReconciliationTask.run(ReconciliationTask.java:93) (likewise)
Comment by Michael Vorburger [ 30/Nov/17 ]

https://git.opendaylight.org/gerrit/#/q/topic:OVSDB-435 will have all changes re. this new OOM accross ovsdb, genius and openflowplugin.

Generated at Wed Feb 07 20:36:23 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.