https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/NETVIRT-1023 netvirt <p><a href="https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-oxygen/406/odl_1/odl1_exception.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-oxygen/406/odl_1/odl1_exception.log.gz</a></p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2017-11-22 07:50:27,620 | WARN | rd-dispatcher-38 | ShardDataTree | 217 - org.opendaylight.controller.sal-distributed-datastore - 1.7.0.SNAPSHOT | member-1-shard-default-config: Store Tx member-1-datastore-config-fe-0-txn-62009-0: Conflicting modification for path /(urn:opendaylight:genius:itm:op?revision=2016-04-06)dpn-endpoints/DPN-TEPs-info/DPN-TEPs-info[{(urn:opendaylight:genius:itm:op?revision=2016-04-06)DPN-ID=220398017525900}]/tunnel-end-points/tunnel-end-points[{(urn:opendaylight:genius:itm:op?revision=2016-04-06)portname=tunnel_port, (urn:opendaylight:genius:itm:op?revision=2016-04-06)VLAN-ID=0, (urn:opendaylight:genius:itm:op?revision=2016-04-06)ip-address=10.29.13.180, (urn:opendaylight:genius:itm:op?revision=2016-04-06)tunnel-type=(urn:opendaylight:genius:interfacemanager?revision=2016-04-06)tunnel-type-vxlan}]. 2017-11-22 07:50:27,621 | ERROR | ult-dispatcher-6 | LocalThreePhaseCommitCohort | 217 - org.opendaylight.controller.sal-distributed-datastore - 1.7.0.SNAPSHOT | Failed to prepare transaction member-1-datastore-config-fe-0-txn-62009-0 on backend OptimisticLockFailedException{message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:731)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:731)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[202:org.opendaylight.controller.sal-akka-raft:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[210:org.opendaylight.controller.sal-clustering-commons:1.7.0.SNAPSHOT] at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18] at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[210:org.opendaylight.controller.sal-clustering-commons:1.7.0.SNAPSHOT] at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18] at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18] at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18] at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction. at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:78)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkNotConflicting(SchemaAwareApplyOperation.java:106)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:170)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:128)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkMergeApplicable(AbstractNodeContainerModificationStrategy.java:313)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:723)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] ... 28 more 2017-11-22 07:50:27,621 | WARN | ult-dispatcher-4 | ConcurrentDOMDataBroker | 217 - org.opendaylight.controller.sal-distributed-datastore - 1.7.0.SNAPSHOT | Tx: DOM-103880 Error during phase CAN_COMMIT, starting Abort OptimisticLockFailedException{message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:731)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:731)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[202:org.opendaylight.controller.sal-akka-raft:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[210:org.opendaylight.controller.sal-clustering-commons:1.7.0.SNAPSHOT] at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18] at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[210:org.opendaylight.controller.sal-clustering-commons:1.7.0.SNAPSHOT] at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18] at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18] at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18] at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction. at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:78)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkNotConflicting(SchemaAwareApplyOperation.java:106)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:170)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:128)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkMergeApplicable(AbstractNodeContainerModificationStrategy.java:313)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:723)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] ... 28 more 2017-11-22 07:50:27,624 | WARN | CommitFutures-9 | JobCoordinatorImpl | 261 - org.opendaylight.infrautils.jobcoordinator-impl - 1.3.0.SNAPSHOT | Job: JobEntry{key='cbe61d1c-a622-45c9-ad46-019863fa6f6a', mainWorker=ItmTepRemoveWorker { Delete Dpn List : [DPNTEPsInfo [_dPNID=220398017525900, _key=DPNTEPsInfoKey [_dPNID=220398017525900], _tunnelEndPoints=[TunnelEndPoints [_gwIpAddress=IpAddress [_ipv4Address=Ipv4Address [_value=0.0.0.0]], _interfaceName=220398017525900:tunnel_port:0, _ipAddress=IpAddress [_ipv4Address=Ipv4Address [_value=10.29.13.180]], _key=TunnelEndPointsKey [_portname=tunnel_port, _vLANID=0, _ipAddress=IpAddress [_ipv4Address=Ipv4Address [_value=10.29.13.180]], _tunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan], _optionTunnelTos=0, _portname=tunnel_port, _subnetMask=IpPrefix [_ipv4Prefix=Ipv4Prefix [_value=0.0.0.0/0]], _tunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan, _tzMembership=[TzMembership [_key=TzMembershipKey [_zoneName=cbe61d1c-a622-45c9-ad46-019863fa6f6a], _zoneName=cbe61d1c-a622-45c9-ad46-019863fa6f6a, augmentation=[]]], _vLANID=0, _optionOfTunnel=false, augmentation=[]]], augmentation=[]]] }, rollbackWorker=null, retryCount=0, futures=[org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$AsyncNotifyingSettableFuture@4b953b81, com.google.common.util.concurrent.ImmediateFuture$ImmediateSuccessfulCheckedFuture@4faab62b]} failed OptimisticLockFailedException{message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:731)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:731)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[202:org.opendaylight.controller.sal-akka-raft:1.7.0.SNAPSHOT] at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[210:org.opendaylight.controller.sal-clustering-commons:1.7.0.SNAPSHOT] at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18] at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[210:org.opendaylight.controller.sal-clustering-commons:1.7.0.SNAPSHOT] at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18] at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18] at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18] at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18] at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18] at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[418:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc] Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction. at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:78)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkNotConflicting(SchemaAwareApplyOperation.java:106)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:170)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:128)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkMergeApplicable(AbstractNodeContainerModificationStrategy.java:313)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[399:org.opendaylight.yangtools.yang-data-impl:1.2.0] at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:723)[217:org.opendaylight.controller.sal-distributed-datastore:1.7.0.SNAPSHOT] ... 28 more</pre> </div></div> NETVIRT-1023

OptimisticLockFailedException: infrautils.jobcoordinator-impl... ItmTepRemoveWorker... ConflictingModificationAppliedException: Node was replaced by other transaction

Bug Medium Resolved Cannot Reproduce Victor Pickard Sam Hague csit:exception Wed, 22 Nov 2017 12:59:59 +0000 Fri, 6 Apr 2018 12:43:02 +0000 Fri, 6 Apr 2018 12:43:02 +0000 Carbon General 0 3 <p>Still seen in carbon: <a href="https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-carbon/205/odl_1/odl1_karaf.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-carbon/205/odl_1/odl1_karaf.log.gz</a></p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2017-11-27 22:43:29,449 | INFO | eChangeHandler-0 | ElanMacEntryListener | 341 - org.opendaylight.netvirt.elanmanager-impl - 0.4.3.SNAPSHOT | ElanMacEntryListener : remove macEntry KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.forwarding.entries.MacEntry, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.ElanForwardingTables, org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.elan.forwarding.tables.MacTable[key=MacTableKey [_elanInstanceName=c123b1f0-88e2-49f4-a90c-4c285d0999c1]], org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.elan.rev150602.forwarding.entries.MacEntry[key=MacEntryKey [_macAddress=PhysAddress [_value=fa:16:3e:88:44:09]]]]} 2017-11-27 22:43:29,450 | WARN | d-dispatcher-673 | ShardDataTree | 184 - org.opendaylight.controller.sal-distributed-datastore - 1.5.3.SNAPSHOT | member-1-shard-default-config: Store Tx member-1-datastore-config-fe-0-txn-97132-0: Conflicting modification for path /(urn:opendaylight:genius:itm:op?revision=2016-04-06)dpn-endpoints/DPN-TEPs-info/DPN-TEPs-info[{(urn:opendaylight:genius:itm:op?revision=2016-04-06)DPN-ID=234113743044535}]/tunnel-end-points/tunnel-end-points[{(urn:opendaylight:genius:itm:op?revision=2016-04-06)portname=tunnel_port, (urn:opendaylight:genius:itm:op?revision=2016-04-06)VLAN-ID=0, (urn:opendaylight:genius:itm:op?revision=2016-04-06)ip-address=10.29.13.141, (urn:opendaylight:genius:itm:op?revision=2016-04-06)tunnel-type=(urn:opendaylight:genius:interfacemanager?revision=2016-04-06)tunnel-type-vxlan}]. 2017-11-27 22:43:29,452 | WARN | ult-dispatcher-6 | ConcurrentDOMDataBroker | 184 - org.opendaylight.controller.sal-distributed-datastore - 1.5.3.SNAPSHOT | Tx: DOM-159119 Error during phase CAN_COMMIT, starting Abort OptimisticLockFailedException{message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:751)[184:org.opendaylight.controller.sal-distributed-datastore:1.5.3.SNAPSHOT]</pre> </div></div> <p>Possibly related to this gerrit : <a href="https://git.opendaylight.org/gerrit/#/c/65726/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/65726/</a></p> <p>Yes Sam, the exception is because of the gerrit <a href="https://git.opendaylight.org/gerrit/#/c/65726/3" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/65726/3</a>. <br/> TransportZoneNotificationUtil is performing tep removal and transportzone deletion using <b>SingleTransactionDataBroker.</b><br/> Since both threads are modifying the same piece of data, ConflictingModificationAppliedException is thrown.<br/> Moreover, just transportzone deletion is sufficient to perform the needed operation as that will take care of tep removal too.</p> <p>Pushed patch <a href="https://git.opendaylight.org/gerrit/#/c/66160/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/66160/</a>, which removed deletion of vtep. Local testing with just deleting the transport zone also removes the tep, as Dimple said above. Note: I did not see any exceptions in my local testing with original patch that deleted vtep.</p> <p>I just started a CSIT run with patch 66160 to verify that this exception has been resolved. </p> <p><a href="https://jenkins.opendaylight.org/sandbox/job/netvirt-csit-1node-openstack-ocata-upstream-stateful-oxygen/1/console" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/sandbox/job/netvirt-csit-1node-openstack-ocata-upstream-stateful-oxygen/1/console</a></p> <p>Job completed, verified that this exception is not in the exception log file, so patch should be good to go.</p> <p> </p> <p><a href="https://logs.opendaylight.org/sandbox/jenkins091/netvirt-csit-1node-openstack-ocata-upstream-stateful-oxygen/3/odl_1/odl1_exception.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/sandbox/jenkins091/netvirt-csit-1node-openstack-ocata-upstream-stateful-oxygen/3/odl_1/odl1_exception.log.gz</a></p> <p>this popped up again in: <a href="https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-openstack-ocata-upstream-stateful-carbon/245/odl_1/odl1_exception.log.gz." class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-openstack-ocata-upstream-stateful-carbon/245/odl_1/odl1_exception.log.gz.</a></p> <p>I was able to reproduce this with genius debug logs in csit (Thanks Jamo for the pointers!).</p> <p> </p> <p>I see 2 threads deleting the dpn at the (almost) same time. I haven't been able to get to the bottom of this yet, will likely be after the Holiday break before I am able to look at this again.</p> <p> </p> <p>Meanwhile, if anyone wants to take a look, debug logs can be found here (not sure for how long they will be available):</p> <p><a href="https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/netvirt-csit-1node-openstack-ocata-vic-upstream-stateful-carbon/1/odl_1/odl1_karaf.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/netvirt-csit-1node-openstack-ocata-vic-upstream-stateful-carbon/1/odl_1/odl1_karaf.log.gz</a></p> <p> </p> <p> </p> <p> </p> <p>2017-12-19 19:58:16,955 | DEBUG | nPool-1-worker-7 | ItmTepRemoveWorker | 327 - org.opendaylight.genius.itm-impl - 0.2.3.SNAPSHOT | Invoking Internal Tunnel delete method with DpnList to be deleted [DPNTEPsInfo [_dPNID=242939118659093, _key=DPNTEPsInfoKey <span class="error">&#91;_dPNID=242939118659093&#93;</span>, _tunnelEndPoints=[TunnelEndPoints [_gwIpAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=0.0.0.0&#93;</span>], _interfaceName=242939118659093:tunnel_port:0, _ipAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.170.59&#93;</span>], _key=TunnelEndPointsKey [_portname=tunnel_port, _vLANID=0, _ipAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.170.59&#93;</span>], _tunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan], _optionTunnelTos=0, _portname=tunnel_port, _subnetMask=IpPrefix [_ipv4Prefix=Ipv4Prefix <span class="error">&#91;_value=0.0.0.0/0&#93;</span>], _tunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan, _tzMembership=[TzMembership [_key=TzMembershipKey <span class="error">&#91;_zoneName=854f98a8-febd-40bd-9c0d-79f692c9e5d1&#93;</span>, _zoneName=854f98a8-febd-40bd-9c0d-79f692c9e5d1, augmentation=[]]], _vLANID=0, _optionOfTunnel=false, augmentation=[]]], augmentation=[]]] ; Meshed DpnList [DPNTEPsInfo{getDPNID=242939118659093, getTunnelEndPoints=[TunnelEndPoints{getGwIpAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=0.0.0.0&#93;</span>], getInterfaceName=242939118659093:tunnel_port:0, getIpAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.170.59&#93;</span>], getOptionTunnelTos=0, getPortname=tunnel_port, getSubnetMask=IpPrefix [_ipv4Prefix=Ipv4Prefix <span class="error">&#91;_value=0.0.0.0/0&#93;</span>], getTunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan, getTzMembership=<span class="error">&#91;TzMembership{getZoneName=48ca5c45-8aac-4dc5-abc3-5244f1222141, augmentations={}}, TzMembership\{getZoneName=826636be-414b-45a9-9fd3-a79bff0e8cf3, augmentations={}}, TzMembership\{getZoneName=a23a3eca-81b6-4c3b-b351-d7d2dc7a609a, augmentations={}}, TzMembership\{getZoneName=98dfda45-76b3-4e53-800c-4fa345117b16, augmentations={}}, TzMembership\{getZoneName=c680e4c7-c3d2-46c8-a70c-a0f91113559c, augmentations={}}, TzMembership\{getZoneName=ae856023-23f2-4cb5-adec-c838c0f96cf5, augmentations={}}, TzMembership\{getZoneName=5b900177-b992-4650-9ef6-1270410a4f2d, augmentations={}}, TzMembership\{getZoneName=854f98a8-febd-40bd-9c0d-79f692c9e5d1, augmentations={}}, TzMembership\{getZoneName=6b2c78d3-5e72-4dbe-8470-b4f8cb77445f, augmentations={}}, TzMembership\{getZoneName=5288709e-b111-42d0-afdf-5b5280c27431, augmentations={}}, TzMembership\{getZoneName=5a7d6cca-e942-4a3a-922a-a96012c837cc, augmentations={}}, TzMembership\{getZoneName=729d4f04-9b7c-48ac-8577-53d5093c5a8b, augmentations={}}, TzMembership\{getZoneName=37b5741d-443c-479b-b280-ae9aca1fd1f0, augmentations={}}, TzMembership\{getZoneName=558b2444-271a-4657-9ff2-752aed518a01, augmentations={}}, TzMembership\{getZoneName=495f0211-e4e1-4bd9-a8b7-fcd7d979ea52, augmentations={}}, TzMembership\{getZoneName=22ace521-3748-438e-903a-b9223ed47317, augmentations={}}, TzMembership\{getZoneName=8b6cde1a-d0b2-412b-8b72-871f8d70f371, augmentations={}}, TzMembership\{getZoneName=48c8caae-874a-4ecf-8a6e-fcade26d142c, augmentations={}}, TzMembership\{getZoneName=7124f2df-e8a5-4a97-bf2e-532304170f4c, augmentations={}}, TzMembership\{getZoneName=8cbb80e9-8c52-43a7-af39-6e036aca8f3f, augmentations={}}, TzMembership\{getZoneName=d86a97ea-36d3-4596-98e9-83a4eaa7cd86, augmentations={}}, TzMembership\{getZoneName=a6e45bed-ff72-4fed-808a-00cc82181f66, augmentations={}}&#93;</span>, getVLANID=0, isOptionOfTunnel=false, augmentations={}}], augmentations={}}] <br/> 2017-12-19 19:58:16,955 | TRACE | nPool-1-worker-7 | DataStoreJobCoordinator | 292 - org.opendaylight.genius.mdsalutil-api - 0.2.3.SNAPSHOT | Job 854f98a8-febd-40bd-9c0d-79f692c9e5d1 took 2ms to complete<br/> 2017-12-19 19:58:16,956 | DEBUG | nPool-1-worker-1 | ItmTepRemoveWorker | 327 - org.opendaylight.genius.itm-impl - 0.2.3.SNAPSHOT | Invoking Internal Tunnel delete method with DpnList to be deleted [DPNTEPsInfo [_dPNID=242939118659093, _key=DPNTEPsInfoKey <span class="error">&#91;_dPNID=242939118659093&#93;</span>, _tunnelEndPoints=[TunnelEndPoints [_gwIpAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=0.0.0.0&#93;</span>], _interfaceName=242939118659093:tunnel_port:0, _ipAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.170.59&#93;</span>], _key=TunnelEndPointsKey [_portname=tunnel_port, _vLANID=0, _ipAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.170.59&#93;</span>], _tunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan], _optionTunnelTos=0, _portname=tunnel_port, _subnetMask=IpPrefix [_ipv4Prefix=Ipv4Prefix <span class="error">&#91;_value=0.0.0.0/0&#93;</span>], _tunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan, _tzMembership=[TzMembership [_key=TzMembershipKey <span class="error">&#91;_zoneName=d86a97ea-36d3-4596-98e9-83a4eaa7cd86&#93;</span>, _zoneName=d86a97ea-36d3-4596-98e9-83a4eaa7cd86, augmentation=[]]], _vLANID=0, _optionOfTunnel=false, augmentation=[]]], augmentation=[]]] ; Meshed DpnList [DPNTEPsInfo{getDPNID=242939118659093, getTunnelEndPoints=[TunnelEndPoints{getGwIpAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=0.0.0.0&#93;</span>], getInterfaceName=242939118659093:tunnel_port:0, getIpAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.170.59&#93;</span>], getOptionTunnelTos=0, getPortname=tunnel_port, getSubnetMask=IpPrefix [_ipv4Prefix=Ipv4Prefix <span class="error">&#91;_value=0.0.0.0/0&#93;</span>], getTunnelType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.rev160406.TunnelTypeVxlan, getTzMembership=<span class="error">&#91;TzMembership{getZoneName=48ca5c45-8aac-4dc5-abc3-5244f1222141, augmentations={}}, TzMembership\{getZoneName=826636be-414b-45a9-9fd3-a79bff0e8cf3, augmentations={}}, TzMembership\{getZoneName=a23a3eca-81b6-4c3b-b351-d7d2dc7a609a, augmentations={}}, TzMembership\{getZoneName=98dfda45-76b3-4e53-800c-4fa345117b16, augmentations={}}, TzMembership\{getZoneName=c680e4c7-c3d2-46c8-a70c-a0f91113559c, augmentations={}}, TzMembership\{getZoneName=ae856023-23f2-4cb5-adec-c838c0f96cf5, augmentations={}}, TzMembership\{getZoneName=5b900177-b992-4650-9ef6-1270410a4f2d, augmentations={}}, TzMembership\{getZoneName=854f98a8-febd-40bd-9c0d-79f692c9e5d1, augmentations={}}, TzMembership\{getZoneName=6b2c78d3-5e72-4dbe-8470-b4f8cb77445f, augmentations={}}, TzMembership\{getZoneName=5288709e-b111-42d0-afdf-5b5280c27431, augmentations={}}, TzMembership\{getZoneName=5a7d6cca-e942-4a3a-922a-a96012c837cc, augmentations={}}, TzMembership\{getZoneName=729d4f04-9b7c-48ac-8577-53d5093c5a8b, augmentations={}}, TzMembership\{getZoneName=37b5741d-443c-479b-b280-ae9aca1fd1f0, augmentations={}}, TzMembership\{getZoneName=558b2444-271a-4657-9ff2-752aed518a01, augmentations={}}, TzMembership\{getZoneName=495f0211-e4e1-4bd9-a8b7-fcd7d979ea52, augmentations={}}, TzMembership\{getZoneName=22ace521-3748-438e-903a-b9223ed47317, augmentations={}}, TzMembership\{getZoneName=8b6cde1a-d0b2-412b-8b72-871f8d70f371, augmentations={}}, TzMembership\{getZoneName=48c8caae-874a-4ecf-8a6e-fcade26d142c, augmentations={}}, TzMembership\{getZoneName=7124f2df-e8a5-4a97-bf2e-532304170f4c, augmentations={}}, TzMembership\{getZoneName=8cbb80e9-8c52-43a7-af39-6e036aca8f3f, augmentations={}}, TzMembership\{getZoneName=d86a97ea-36d3-4596-98e9-83a4eaa7cd86, augmentations={}}, TzMembership\{getZoneName=a6e45bed-ff72-4fed-808a-00cc82181f66, augmentations={}}&#93;</span>, getVLANID=0, isOptionOfTunnel=false, augmentations={}}], augmentations={}}] <br/> 2017-12-19 19:58:16,956 | TRACE | nPool-1-worker-1 | DataStoreJobCoordinator | 292 - org.opendaylight.genius.mdsalutil-api - 0.2.3.SNAPSHOT | Job d86a97ea-36d3-4596-98e9-83a4eaa7cd86 took 3ms to complete<br/> 2017-12-19 19:58:16,956 | WARN | rd-dispatcher-31 | ShardDataTree | 184 - org.opendaylight.controller.sal-distributed-datastore - 1.5.3.SNAPSHOT | member-1-shard-default-config: Store Tx member-1-datastore-config-fe-0-txn-13613-0: Conflicting modification for path /(urn:opendaylight:genius:itm:op?revision=2016-04-06)dpn-endpoints/DPN-TEPs-info/DPN-TEPs-info<span class="error">&#91;\{(urn:opendaylight:genius:itm:op?revision=2016-04-06)DPN-ID=242939118659093}&#93;</span>/tunnel-end-points/tunnel-end-points<span class="error">&#91;\{(urn:opendaylight:genius:itm:op?revision=2016-04-06)portname=tunnel_port, (urn:opendaylight:genius:itm:op?revision=2016-04-06)VLAN-ID=0, (urn:opendaylight:genius:itm:op?revision=2016-04-06)ip-address=10.30.170.59, (urn:opendaylight:genius:itm:op?revision=2016-04-06)tunnel-type=(urn:opendaylight:genius:interfacemanager?revision=2016-04-06)tunnel-type-vxlan}&#93;</span>.<br/> 2017-12-19 19:58:16,958 | ERROR | lt-dispatcher-33 | LocalThreePhaseCommitCohort | 184 - org.opendaylight.controller.sal-distributed-datastore - 1.5.3.SNAPSHOT | Failed to prepare transaction member-1-datastore-config-fe-0-txn-13613-0 on backend<br/> OptimisticLockFailedException{message=Optimistic lock failed., errorList=[RpcError <span class="error">&#91;message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.&#93;</span>]}<br/> at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:751)<span class="error">&#91;184:org.opendaylight.controller.sal-distributed-datastore:1.5.3.SNAPSHOT&#93;</span></p> Development Rank 0|i038dj: