[GENIUS-196] Optimistic lock failed for path /(urn:opendaylight:genius:itm:config?revision=2016-04-06)tunnel-monitor-params/enabled Created: 31/Jul/18  Updated: 02/Aug/18

Status: Open
Project: genius
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Medium
Reporter: Michael Vorburger Assignee: Dimple J
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
blocks GENIUS-200 Make Genius CSIT fail in case any exc... Open
is blocked by CONTROLLER-1855 Controller should not error log any O... Resolved

 Description   

https://lists.opendaylight.org/pipermail/genius-dev/2018-July/003210.html

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/genius-csit-verify-1node-verify/246/odl_1/odl1_karaf.log.gz

2018-07-31T07:25:28,231 | WARN  | opendaylight-cluster-data-shard-dispatcher-39 | ShardDataTree                    | 228 - org.opendaylight.controller.sal-distributed-datastore - 1.7.3.SNAPSHOT | member-1-shard-default-operational: Store Tx member-1-datastore-operational-fe-0-txn-847-0: Conflicting modification for path /(urn:opendaylight:genius:itm:config?revision=2016-04-06)tunnel-monitor-params/enabled.
2018-07-31T07:25:28,232 | ERROR | opendaylight-cluster-data-akka.actor.default-dispatcher-31 | LocalThreePhaseCommitCohort      | 228 - org.opendaylight.controller.sal-distributed-datastore - 1.7.3.SNAPSHOT | Failed to prepare transaction member-1-datastore-operational-fe-0-txn-847-0 on backend
org.opendaylight.controller.md.sal.common.api.data.OptimisticLockFailedException: Optimistic lock failed for path /(urn:opendaylight:genius:itm:config?revision=2016-04-06)tunnel-monitor-params/enabled
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:749) ~[228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:787) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:734) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:888) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:99) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:239) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:201) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:739) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:336) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) [213:org.opendaylight.controller.sal-akka-raft:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44) [221:org.opendaylight.controller.sal-clustering-commons:1.7.3.SNAPSHOT]
	at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:275) [47:com.typesafe.akka.persistence:2.5.11]
	at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104) [221:org.opendaylight.controller.sal-clustering-commons:1.7.3.SNAPSHOT]
	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:608) [44:com.typesafe.akka.actor:2.5.11]
	at akka.actor.Actor.aroundReceive(Actor.scala:517) [44:com.typesafe.akka.actor:2.5.11]
	at akka.actor.Actor.aroundReceive$(Actor.scala:515) [44:com.typesafe.akka.actor:2.5.11]
	at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:273) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:691) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:192) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:191) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:273) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:590) [44:com.typesafe.akka.actor:2.5.11]
	at akka.actor.ActorCell.invoke(ActorCell.scala:559) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.Mailbox.run(Mailbox.scala:224) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [44:com.typesafe.akka.actor:2.5.11]
Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:80) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkNotConflicting(SchemaAwareApplyOperation.java:111) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkMergeApplicable(SchemaAwareApplyOperation.java:160) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:137) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:302) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkMergeApplicable(AbstractNodeContainerModificationStrategy.java:313) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:137) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:100) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:302) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:284) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:71) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:38) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:741) ~[228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	... 30 more
2018-07-31T07:25:28,233 | ERROR | CommitFutures-3  | ItmUtils                         | 255 - org.opendaylight.genius.itm-impl - 0.4.3.SNAPSHOT | Error in Datastore write operation
org.opendaylight.controller.md.sal.common.api.data.OptimisticLockFailedException: Optimistic lock failed for path /(urn:opendaylight:genius:itm:config?revision=2016-04-06)tunnel-monitor-params/enabled
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:749) ~[228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:787) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:734) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:888) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:99) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:239) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:201) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:739) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:336) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) [213:org.opendaylight.controller.sal-akka-raft:1.7.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44) [221:org.opendaylight.controller.sal-clustering-commons:1.7.3.SNAPSHOT]
	at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:275) [47:com.typesafe.akka.persistence:2.5.11]
	at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104) [221:org.opendaylight.controller.sal-clustering-commons:1.7.3.SNAPSHOT]
	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:608) [44:com.typesafe.akka.actor:2.5.11]
	at akka.actor.Actor.aroundReceive(Actor.scala:517) [44:com.typesafe.akka.actor:2.5.11]
	at akka.actor.Actor.aroundReceive$(Actor.scala:515) [44:com.typesafe.akka.actor:2.5.11]
	at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:273) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:691) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:192) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:191) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:273) [47:com.typesafe.akka.persistence:2.5.11]
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:590) [44:com.typesafe.akka.actor:2.5.11]
	at akka.actor.ActorCell.invoke(ActorCell.scala:559) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.Mailbox.run(Mailbox.scala:224) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [44:com.typesafe.akka.actor:2.5.11]
	at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [44:com.typesafe.akka.actor:2.5.11]
Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:80) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkNotConflicting(SchemaAwareApplyOperation.java:111) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkMergeApplicable(SchemaAwareApplyOperation.java:160) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:137) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:302) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkMergeApplicable(AbstractNodeContainerModificationStrategy.java:313) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:137) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:100) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:302) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:284) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:71) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:38) ~[386:org.opendaylight.yangtools.yang-data-impl:2.0.5]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:741) ~[228:org.opendaylight.controller.sal-distributed-datastore:1.7.3.SNAPSHOT]
	... 30 more

 Comments   
Comment by Michael Vorburger [ 31/Jul/18 ]

This could probably be fix by using the RetryingManagedTransactionRunner ...

Comment by Michael Vorburger [ 31/Jul/18 ]

The real problem is all those datastore helper methods in ItmUtils ... we should use this issue to replace all of those, which are actually already deprecated but of course still present, with ManagedNewTransactionRunner usages instead - and remove them from ItmUtils - incl. that DEFAULT_CALLBACK (which is where the log above comes from). The ManagedNewTransactionRunner, combined with infrautils' error logging utility for Futures (see usages elsewhere in genius and netvirt), can replace that.

Comment by Michael Vorburger [ 31/Jul/18 ]

CONTROLLER-1855 would take care of the log from controller, but that alone is not sufficient; as the logging from genius (above) also needs to be handled.

Generated at Wed Feb 07 20:00:09 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.