[OPNFLWPLUG-394] Read failures in StatAbstractListenCommit

Mon, 27 Sep 2021 09:01:27 +0000

With CDS, I noticed several "transaction is closed" ReadFailedExceptions in the log with corresponding messages re: ReadData messages going to dead letters. I tracked the issue to the StatAbstractListenCommit class. This is easily reproduced via the steps in https://bugs.opendaylight.org/show_bug.cgi?id=2588.

StatAbstractListenCommit contains a volatile currentReadTx which is created lazily and used to perform reads in the readLatestConfiguration method. It also has an onDataChanged callback method whose only purpose is to close the currentReadTx. It looks like this is done because the DOM read-only Tx captures a snapshot of the data when first created so when data changes it assumes the RO Tx snapshot is now stale so it closes the Tx. However the synchronization between the 2 methods isn’t correct, i.e. the currentReadTx instance is published before the read call is performed during which time onDataChanged could be called to close it. While I observed this scenario with CDS, it could also occur with IMDS.

I prototyped changes to add a currentReadTxStale flag and setting it in onDataChanged instead of blindly closing the Tx. In readLatestConfiguration, if currentReadTxStale id set, it closes the previous Tx and creates a new one. This prevents onDataChanged from affecting in-flight reads. I ran the repro steps several times with no errors.

However, the same issue could occur if readLatestConfiguration could be called concurrently. I'm not familiar with the rest of the code so unclear about the calling patterns. I'm guessing readLatestConfiguration can't be called concurrently as the code that I checked in the call chain doesn't appear to be thread-safe.

OpenDaylight JIRA

[OPNFLWPLUG-394] Read failures in StatAbstractListenCommit