[CONTROLLER-997] Clustering : Occasional failure to create transactions during single node integration test

Blocks — Tue, 25 Jul 2023 08:24:18 +0000

2014-10-31 11:29:49,804 | WARN | lt-dispatcher-44 | MConcurrentDataCommitCoordinator | 137 - org.opendaylight.controller.sal-broker-impl - 1.2.0.SNAPSHOT | Tx: DOM-CHAIN-1-4 Error during phase CAN_COMMIT, starting Abort
org.opendaylight.controller.cluster.datastore.exceptions.TimeoutException: Sending message class org.opendaylight.controller.protobuff.messages.transaction.ShardTransactionMessages$CreateTransaction to actor ActorSelection[Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-operational/member-1-shard-inventory-operational)]
failed. Try again later.
at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeOperation(ActorContext.java:231)[262:org.opendaylight.controller.sal-distributed-datast
ore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.TransactionProxy.createTransactionIfMissing(TransactionProxy.java:387)[262:org.opendaylight.controller.sal-distri
buted-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.TransactionProxy.merge(TransactionProxy.java:289)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0
.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.DOMForwardedWriteTransaction.merge(DOMForwardedWriteTransaction.java:102)[137:org.opendaylight.controller.sa
l-broker-impl:1.2.0.SNAPSHOT] at org.opendaylight.controller.md.sal.binding.impl.BindingDataWriteTransactionImpl.ensureParentsByMerge(BindingDataWriteTransactionImpl.java:67)[140:org.opendayli
ght.controller.sal-binding-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.binding.impl.AbstractWriteTransaction.merge(AbstractWriteTransaction.java:58)[140:org.opendaylight.controller.sal-binding-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.inventory.manager.NodeChangeCommiter$5.applyOperation(NodeChangeCommiter.java:142)[152:org.opendaylight.controller.md.inventory-
manager:1.2.0.SNAPSHOT] at org.opendaylight.controller.md.inventory.manager.FlowCapableInventoryProvider.createCustomizedTransaction(FlowCapableInventoryProvider.java:167)[152:org.openda
ylight.controller.md.inventory-manager:1.2.0.SNAPSHOT] at org.opendaylight.controller.md.inventory.manager.FlowCapableInventoryProvider.submitOperations(FlowCapableInventoryProvider.java:108)[152:org.opendaylight.controller.md.inventory-manager:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.inventory.manager.FlowCapableInventoryProvider.run(FlowCapableInventoryProvider.java:82)[152:org.opendaylight.controller.md.inventory-manager:1.2.0.SNAPSHOT]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_65]
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [5 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.Await$.result(package.scala:107)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.Await.result(package.scala)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeOperation(ActorContext.java:229)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
... 10 more

2014-10-31 11:29:49,813 | ERROR | CommitFutures-0 | FlowCapableInventoryProvider | 152 - org.opendaylight.controller.md.inventory-manager - 1.2.0.SNAPSHOT | Failed to export Flow Capable Inventory, Transaction DOM-CHAIN-1-4 failed.
TransactionCommitFailedException

{message=preCommit encountered an unexpected failure, errorList=[RpcError [message=preCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.datastore.exceptions.TimeoutException: Sending message class org.opendaylight.controller.protobuff.messages.transaction.ShardTransactionMessages$CreateTransaction to actor ActorSelection[Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-operational/member-1-shard-inventory-operational)] failed. Try again later.]]}

at org.opendaylight.controller.md.sal.dom.broker.impl.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37)[137:org.opendaylight.controller.sal-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18)[137:org.opendaylight.controller.sal-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:96)[55:org.opendaylight.yangtools.util:0.7.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.DOMConcurrentDataCommitCoordinator.handleException(DOMConcurrentDataCommitCoordinator.java:194)[137:org.opendaylight.controller.sal-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.DOMConcurrentDataCommitCoordinator.access$100(DOMConcurrentDataCommitCoordinator.java:38)[137:org.opendaylight.controller.sal-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.DOMConcurrentDataCommitCoordinator$1.onFailure(DOMConcurrentDataCommitCoordinator.java:110)[137:org.opendaylight.controller.sal-broker-impl:1.2.0.SNAPSHOT]
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1140)[52:com.google.guava:14.0.1]
at org.opendaylight.controller.md.sal.dom.broker.impl.DOMConcurrentDataCommitCoordinator$SimpleSameThreadExecutor.execute(DOMConcurrentDataCommitCoordinator.java:317)[137:org.opendaylight.controller.sal-broker-impl:1.2.0.SNAPSHOT]
at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:150)[52:com.google.guava:14.0.1]
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:135)[52:com.google.guava:14.0.1]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:203)[52:com.google.guava:14.0.1]
at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:68)[52:com.google.guava:14.0.1]
at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$2.onComplete(ThreePhaseCommitCohortProxy.java:93)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$2.onComplete(ThreePhaseCommitCohortProxy.java:86)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at akka.dispatch.OnComplete.internal(Future.scala:246)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.OnComplete.internal(Future.scala:244)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:174)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:171)[247:com.typesafe.akka.actor:2.3.4]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)[247:com.typesafe.akka.actor:2.3.4]
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)[247:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)[247:com.typesafe.akka.actor:2.3.4]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]

Caused by: org.opendaylight.controller.cluster.datastore.exceptions.TimeoutException: Sending message class org.opendaylight.controller.protobuff.messages.transaction.ShardTransactionMessages$CreateTransaction to actor ActorSelection[Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-operational/member-1-shard-inventory-operational)] failed. Try again later.
at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeOperation(ActorContext.java:231)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.TransactionProxy.createTransactionIfMissing(TransactionProxy.java:387)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.TransactionProxy.merge(TransactionProxy.java:289)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.DOMForwardedWriteTransaction.merge(DOMForwardedWriteTransaction.java:102)[137:org.opendaylight.controller.sal-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.binding.impl.BindingDataWriteTransactionImpl.ensureParentsByMerge(BindingDataWriteTransactionImpl.java:67)[140:org.opendaylight.controller.sal-binding-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.binding.impl.AbstractWriteTransaction.merge(AbstractWriteTransaction.java:58)[140:org.opendaylight.controller.sal-binding-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.inventory.manager.NodeChangeCommiter$5.applyOperation(NodeChangeCommiter.java:142)[152:org.opendaylight.controller.md.inventory-manager:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.inventory.manager.FlowCapableInventoryProvider.createCustomizedTransaction(FlowCapableInventoryProvider.java:167)[152:org.opendaylight.controller.md.inventory-manager:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.inventory.manager.FlowCapableInventoryProvider.submitOperations(FlowCapableInventoryProvider.java:108)[152:org.opendaylight.controller.md.inventory-manager:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.inventory.manager.FlowCapableInventoryProvider.run(FlowCapableInventoryProvider.java:82)[152:org.opendaylight.controller.md.inventory-manager:1.2.0.SNAPSHOT]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_65]
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [5 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.Await$.result(package.scala:107)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.Await.result(package.scala)[244:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeOperation(ActorContext.java:229)[262:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
... 10 more

OpenDaylight JIRA

[CONTROLLER-997] Clustering : Occasional failure to create transactions during single node integration test