[CONTROLLER-1745] produce-transactions can get stuck when closing itemProducer

Tue, 25 Jul 2023 08:24:45 +0000

This is a fork of ~~CONTROLLER-1687~~ as new investigation suggests the cause of this Bug is different than the previously fixed ~~CONTROLLER-1687~~ cause.

Filling against Controller/clustering, even though the guilty project might be Mdsal instead.

This affects one CSIT test case [34], but the probability of failure is less than 10%.

The test calls write-transactions on a shard follower, then isolates that member, then waits some time (115s) expecting no failure, and then some time (20 s) expecting a failure (from RequestTimeoutException). If this Bug appears, write-transactions does not return anything in withing reasonable time.

Copied from ~~CONTROLLER-1687~~ comments:

> I will try to reproduce with more logging.

Reproduced. This time the karaf.log [39] shows that the progress stops when closing itemProducer. Will try to repeat with even more logs [40].

2017-07-26 17:28:10,129 | TRACE | pool-31-thread-2 | ProduceTransactionsHandler | 257 - org.opendaylight.controller.samples.clustering-it-provider - 1.5.2.SNAPSHOT | org.opendaylight.controller.clustering.it.provider.impl.ProduceTransactionsHandler
@30b30b0 Entering runFailed.

[34] https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/794/log.html.gz#s1-s34-t7-k2-k16-k1-k1
[39] https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-ci-only-carbon/18/odl1_karaf.log.gz
[40] https://git.opendaylight.org/gerrit/#/c/60783/1

OpenDaylight JIRA

[CONTROLLER-1745] produce-transactions can get stuck when closing itemProducer