https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/CONTROLLER-1761 controller <p>On the same environment as the one described in <a href="https://jira.opendaylight.org/browse/CONTROLLER-1756" title="OOM due to huge Map in ShardDataTree" class="issue-link" data-issue-key="CONTROLLER-1756"><del>CONTROLLER-1756</del></a> but the problem described here is been seen on that environment but without any OOM I'm told - thus filing this as a separate new issue:</p> <p>We're seeing ODL "commit suicide" (shut itself down, by System.exit) with the last line in the log being:</p> <p>2017-08-24 10:03:04,487 | ERROR | -dispatcher-5689 | ActorSystemImpl | 186 - com.typesafe.akka.slf4j - 2.4.18 | Uncaught error from thread <span class="error">&#91;opendaylight-cluster-data-shard-dispatcher-88&#93;</span> shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled</p> <p>Before that there are numerous errors from (...)rd-dispatcher-88 like this:</p> <p>2017-08-24 10:03:03,446 | ERROR | rd-dispatcher-88 | Shard | 204 - org.opendaylight.controller.sal-clustering-commons - 1.5.2.Carbon | member-1-shard-default-operational, An exception occurred while committing transaction member-2-datastore-operational-fe-1-txn-10344-0<br/> java.lang.IllegalStateException: Store tree org.opendaylight.yangtools.yang.data.api.schema.tree.spi.LazyContainerNode@3cde34f3 and candidate base org.opendaylight.yangtools.yang.data.api.schema.tree.spi.LazyContainerNode@129ff94a differ.</p> <p>Full log is attached.</p> <p>Operating System: All<br/> Platform: All</p> CONTROLLER-1761

Uncaught error from thread ... shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled

Bug Resolved Duplicate Unassigned Michael Vorburger Mon, 28 Aug 2017 10:42:36 +0000 Thu, 19 Oct 2017 21:25:30 +0000 Mon, 4 Sep 2017 08:28:48 +0000 Carbon mdsal 0 1 <p>&gt; Full log is attached.</p> <p>2.8M was too big, so its on <a href="https://drive.google.com/open?id=0B7gTXYDlI5sLR3ZhcmNZZkw1OEU" class="external-link" target="_blank" rel="nofollow noopener">https://drive.google.com/open?id=0B7gTXYDlI5sLR3ZhcmNZZkw1OEU</a></p> <p>&gt; Full log is attached.</p> <p>There are (a lot, hard to be sure that's relevant and what not..) other errors in that log as well, including this kind (from different components but both having the same root cause AFAIK) - are these related to the problem here:</p> <p>2017-08-24 10:02:42,772 | ERROR | Pool-1-worker-81 | DataStoreJobCoordinator | 319 - org.opendaylight.genius.mdsalutil-api - 0.2.2.Carbon | Exception when executing jobEntry: JobEntry</p> {key='VPNINTERFACE-7b4c0454-6452-4e5f-9856-aae03a9b223e', mainWorker=org.opendaylight.netvirt.vpnmanager.InterfaceStateChangeListener$$Lambda$919/186376061@7bb23fe4, rollbackWorker=null, retryCount=0, futures=null} <p>java.lang.RuntimeException: TransactionCommitFailedException</p> {message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-operational/member-1-shard-default-operational#856921107)]] after [30000 ms]. Sender[null] sent message of type "org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction".]]} <p>2017-08-24 10:02:42,832 | ERROR | CommitFutures-11 | TransactionChainManager | 307 - org.opendaylight.openflowplugin.impl - 0.4.2.Carbon | Transaction commit failed. <br/> TransactionCommitFailedException</p> {message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-operational/member-1-shard-default-operational#856921107)]] after [30000 ms]. Sender[null] sent message of type "org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction".]]} <p>Have you checked with latest Carbon?</p> <p>&gt; Have you checked with latest Carbon?</p> <p>No, this was seen with the Carbon build mentioned in <a href="https://jira.opendaylight.org/browse/CONTROLLER-1756" title="OOM due to huge Map in ShardDataTree" class="issue-link" data-issue-key="CONTROLLER-1756"><del>CONTROLLER-1756</del></a> (opendaylight-6.2.0-0.1.20170817rel1931.el7.noarch.rpm) - but can we deduce from your Q that you have hopes that recent fixes e.g. <a href="https://jira.opendaylight.org/browse/CONTROLLER-1755" title="RaftActor lastApplied index moves backwards" class="issue-link" data-issue-key="CONTROLLER-1755"><del>CONTROLLER-1755</del></a> or <a href="https://jira.opendaylight.org/browse/CONTROLLER-1746" title="OOM with large number of closed transactions" class="issue-link" data-issue-key="CONTROLLER-1746"><del>CONTROLLER-1746</del></a> <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/help_16.png" height="16" width="16" align="absmiddle" alt="" border="0"/> may solve this? If so, just to learn, why?</p> <p>But we'll redo this scale test ASAP - now waiting for <a href="https://jira.opendaylight.org/browse/OPNFLWPLUG-933" title="IllegalStateException: Attempted to close chain with outstanding transaction PingPongTransaction at org.opendaylight.openflowplugin.impl.device.TransactionChainManager.createTxChain" class="issue-link" data-issue-key="OPNFLWPLUG-933"><del>OPNFLWPLUG-933</del></a> c/62358 merge...</p> <p>Well, for one, the ISE mentioned in description is a <a href="https://jira.opendaylight.org/browse/CONTROLLER-1755" title="RaftActor lastApplied index moves backwards" class="issue-link" data-issue-key="CONTROLLER-1755"><del>CONTROLLER-1755</del></a> duplicate. Fixing that may mask whoever is not catching their exception again...</p> <p>As far as I can tell, this is fixed with 9028. Please reopen with updated codebase/logs if it persists.</p> Duplicate CONTROLLER-1755 Development External issue ID 9061 External issue URL Rank 0|i02skf: