https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/GENIUS-86 genius <p>There are chances that the lockManager gets into a state of deadlock when an AskTimeException occurs.</p> <p>Operating System: All<br/> Platform: All</p> GENIUS-86

LockManager being deadlocked

Bug Resolved Done Kency Kurian Kency Kurian Fri, 11 Aug 2017 05:46:02 +0000 Tue, 31 Jul 2018 15:36:48 +0000 Tue, 31 Jul 2018 15:36:48 +0000 Nitrogen General 0 4 <p><a href="https://git.opendaylight.org/gerrit/61977" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/61977</a></p> <p>The patch is in merge conflict. Can we incorporate Michael's review comments or discuss on what is missing?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=kencykurian%40gmail.com" class="user-hover" rel="kencykurian@gmail.com">kencykurian@gmail.com</a> is this reproducible? Do you have logs showing those AskTimeException?</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/65233/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/65233/</a></p> <p>Hi Michael,</p> <p>I am not able to find the logs for the same. But I had send a mail to controller dev with just the exception message, pasting it here:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka.tcp://opendaylight-cluster-data@192.168.123.3:2550/), Path(/user/shardmanager-operational/192.168.123.3-shard-default-operational/shard-192.168.123.4:datastore-operational@1:790#1851512584)]] after [30000 ms]. Sender[null] sent message of type "org.opendaylight.controller.cluster.datastore.messages.BatchedModifications".]]}</pre> </div></div> <p>But eventually the transaction got successful, but because of retries, on reading the DS, lockmanager thinks it got locked by some other thread but it was actually the same thread which eventually got successful. In order to avoid these situations we have introduced the owner field.</p> <p>IMHO the TransactionCommitFailedException caused by AskTimeoutException after 30s of cluster issues should not be handled by application (genius lockmanager) code, and not lead to retries but fails - contrary to the OptimisticLockFailedException. In c/65233 it will do just that (whereas in c/61526 you retry for any failure). </p> <p>IMHO If we need to handle cluster issues longer than 30s than perhaps we need to bump some timeout somewhere there, not retry from application code.</p> <p>The lockmanager should still never deadlock though of course, so if your change fixes that, then I'm all for that part!</p> Development External issue ID 8975 External issue URL Rank 0|i01orj: