[OPNFLWPLUG-858] Data validation failed for path /(nodes/node/node/../group-id Created: 20/Feb/17  Updated: 19/Nov/19  Resolved: 02/Apr/18

Status: Resolved
Project: OpenFlowPlugin
Component/s: General
Affects Version/s: Oxygen, Fluorine
Fix Version/s: Oxygen, Fluorine

Type: Bug Priority: Highest
Reporter: Sam Hague Assignee: Sunil Kumar M S
Resolution: Done Votes: 0
Labels: csit:3node, csit:exception, patch_merged
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: File odl1_karaf.132.tar.gz    
Issue Links:
Duplicate
is duplicated by OPNFLWPLUG-944 ModifiedNodeDoesNotExistException: Ca... Resolved
Relates
relates to NETVIRT-996 All work to to gain clustering stability Open
relates to NETVIRT-1013 OptimisticLockFailedException: Interf... Resolved
Epic Link: Clustering Stability
External issue ID: 7826

 Description   
2017-02-20 01:55:53,344 | WARN  | ult-dispatcher-3 | ShardDataTree                    | 176 - org.opendaylight.controller.sal-distributed-datastore - 1.4.3.SNAPSHOT | member-1-shard-inventory-operational: Store Tx member-1-datastore-operational-fe-0-chn-8-txn-5754: Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:229549806101377}]/AugmentationIdentifier{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210023}].
org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:229549806101377}]/AugmentationIdentifier{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210023}] does not exist. Cannot apply modification to its children.
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:281)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextTransaction(ShardDataTree.java:524)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:560)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:81)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:98)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:237)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:201)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:437)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:243)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:291)[171:org.opendaylight.controller.sal-akka-raft:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:29)[170:org.opendaylight.controller.sal-clustering-commons:1.4.3.SNAPSHOT]
	at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[164:com.typesafe.akka.persistence:2.4.7]
	at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:97)[170:org.opendaylight.controller.sal-clustering-commons:1.4.3.SNAPSHOT]
	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[158:com.typesafe.akka.actor:2.4.7]
	at akka.actor.Actor$class.aroundReceive(Actor.scala:484)[158:com.typesafe.akka.actor:2.4.7]
	at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:633)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:179)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[158:com.typesafe.akka.actor:2.4.7]
	at akka.actor.ActorCell.invoke(ActorCell.scala:495)[158:com.typesafe.akka.actor:2.4.7]
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[158:com.typesafe.akka.actor:2.4.7]
	at akka.dispatch.Mailbox.run(Mailbox.scala:224)[158:com.typesafe.akka.actor:2.4.7]
	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[158:com.typesafe.akka.actor:2.4.7]
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
2017-02-20 01:55:53,345 | ERROR | ult-dispatcher-3 | TransactionChainProxy            | 176 - org.opendaylight.controller.sal-distributed-datastore - 1.4.3.SNAPSHOT | Tx: member-1-datastore-operational-fe-0-chn-8-txn-5755 - ready future failed for previous Tx member-1-datastore-operational-fe-0-chn-8-txn-5755
2017-02-20 01:55:53,346 | WARN  | lt-dispatcher-22 | ConcurrentDOMDataBroker          | 176 - org.opendaylight.controller.sal-distributed-datastore - 1.4.3.SNAPSHOT | Tx: DOM-CHAIN-7-5754 Error during phase CAN_COMMIT, starting Abort
TransactionCommitFailedException{message=Data did not pass validation., errorList=[RpcError [message=Data did not pass validation., severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:229549806101377}]/AugmentationIdentifier{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210023}] does not exist. Cannot apply modification to its children.]]}
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextTransaction(ShardDataTree.java:540)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:560)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:81)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:98)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:237)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:201)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:437)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:243)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:291)[171:org.opendaylight.controller.sal-akka-raft:1.4.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:29)[170:org.opendaylight.controller.sal-clustering-commons:1.4.3.SNAPSHOT]
	at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[164:com.typesafe.akka.persistence:2.4.7]
	at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:97)[170:org.opendaylight.controller.sal-clustering-commons:1.4.3.SNAPSHOT]
	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[158:com.typesafe.akka.actor:2.4.7]
	at akka.actor.Actor$class.aroundReceive(Actor.scala:484)[158:com.typesafe.akka.actor:2.4.7]
	at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:633)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:179)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[164:com.typesafe.akka.persistence:2.4.7]
	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[158:com.typesafe.akka.actor:2.4.7]
	at akka.actor.ActorCell.invoke(ActorCell.scala:495)[158:com.typesafe.akka.actor:2.4.7]
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[158:com.typesafe.akka.actor:2.4.7]
	at akka.dispatch.Mailbox.run(Mailbox.scala:224)[158:com.typesafe.akka.actor:2.4.7]
	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[158:com.typesafe.akka.actor:2.4.7]
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[154:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:229549806101377}]/AugmentationIdentifier{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210023}] does not exist. Cannot apply modification to its children.
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:281)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[80:org.opendaylight.yangtools.yang-data-impl:1.0.3.SNAPSHOT]
	at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextTransaction(ShardDataTree.java:524)[176:org.opendaylight.controller.sal-distributed-datastore:1.4.3.SNAPSHOT]
	... 26 more


 Comments   
Comment by Sam Hague [ 20/Feb/17 ]

Attachment odl1_karaf.132.tar.gz has been added with description: karaf logs from job 132

Comment by Abhinav Gupta [ 10/Mar/17 ]

Log location: https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-boron/132/archives/odl1_karaf.log.gz

Comment by Abhinav Gupta [ 10/Mar/17 ]

Analyzed the last 6 CSIT runs (170-175), I see this issue occuring just once, so it's quite sporadic.
It's about group 2100xx (ELAN/ETree Local BC Group) creation/deletion landing into concurrent modification exception.
Have checked with Peri, a recent fix has gone in which might fix it: https://git.opendaylight.org/gerrit/#/c/53059/
Will revisit this with the next 3-4 runs after the above commit merges in stable/boron.

Comment by Janki Chhatbar [ 15/Mar/17 ]

This is seen in Carbon too.
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-carbon/196/archives/odl1_karaf.log.gz

Comment by Periyasamy Palanisamy [ 27/Mar/17 ]

Swati, since https://git.opendaylight.org/gerrit/#/c/53059/ is merge, can you check this error still logged in karaf.log ? If not, please move it to fixed.

Comment by Janki Chhatbar [ 30/Mar/17 ]

Exception not logged at https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-carbon/274/archives/odl1_karaf.log.gz.

Moving it to fixed.

Comment by Swati Udhavrao Niture [ 30/Mar/17 ]

yes, ok.

Thanks Janki.

Comment by Janki Chhatbar [ 07/Apr/17 ]

Swati, this is again seen in https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-carbon/309/archives/odl1_karaf.log.gz.

I have reopened the bug. Can you please have a look.

Thanks

Comment by Swati Udhavrao Niture [ 07/Apr/17 ]

(In reply to Janki Chhatbar from comment #7)
> Swati, this is again seen in
> https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-
> newton-nodl-v2-upstream-stateful-carbon/309/archives/odl1_karaf.log.gz.
>
> I have reopened the bug. Can you please have a look.
>
> Thanks

Ok Janki.

Comment by Janki Chhatbar [ 17/Apr/17 ]

Seen in Nitrogen https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/12/archives/odl1_karaf.log.gz

Comment by Janki Chhatbar [ 25/Apr/17 ]

https://git.opendaylight.org/gerrit/#/c/54589/

Comment by Janki Chhatbar [ 22/May/17 ]

Seen in https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-carbon/597/archives/odl1_karaf.log.gz

Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:84565533607347}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=150006}

].
org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:84565533607347}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=150006}

] does not exist. Cannot apply modification to its children.

Comment by Janki Chhatbar [ 14/Jul/17 ]

Seen in latest Nitrogen build
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/123/odl1_karaf.log.gz

Comment by Janki Chhatbar [ 08/Aug/17 ]

Seen in latest build:

Suites having this exception:
1. suites/openstack/connectivity/02_l3_tests.robot

  • Test: Add Interfaces To Router

2. suites/openstack/connectivity/04_security_group_tests.robot

  • Test: Add Interfaces To Router

3. suites/openstack/tempest/tempest.robot

  • Test: tempest.api.network
  • Test: tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os

4. suites/netvirt/Netvirt_Vpnservice/03_basic_ipv6_vpnservice.robot

  • Test: Dissociate L3VPN From Routers

5. suites/netvirt/ElanService/ElanService.robot

Comment by Janki Chhatbar [ 08/Aug/17 ]

Seen in latest build:
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/165/odl1_karaf.log.gz

Comment by Sam Hague [ 12/Aug/17 ]

still in latest: https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-snat-conntrack-nitrogen/99/odl1_karaf.log.gz

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-nitrogen/132/odl1_karaf.log.gz

Comment by Sam Hague [ 29/Aug/17 ]

debugging patch: https://git.opendaylight.org/gerrit/#/c/62184/

Comment by Sam Hague [ 13/Sep/17 ]

We analyzed the exception today here.
The Openflow-inventory Operational Datastore is not managed by Genius / Netvirt.

The management of data on that DS is driven by OpenFlowPlugin and OVSDB.

The same inventory-node is present in config-DS equivalent openflow:inventory-nodes.
However, that inventory-node is absent in Operational DS, which results in the group-id exception being thrown.

I request that we get help from OpenFlowPlugin team to take a look on why there is a
transaction initiated on an inventory-node that does not exist?

We referred the following links in the bug:

Config DS has the node correctly:
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-nitrogen/132/log.html.gz#s1-s2-s1-t1-k3-k1-k2-k1-k2-k33-k4

Node: openflow:1041420389861

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-nitrogen/132/odl1_karaf.log.gz

2017-08-11 16:58:50,470 | WARN | rd-dispatcher-91 | ShardDataTree | 211 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | member-1-shard-inventory-operational: Store Tx member-1-datastore-operational-fe-0-chn-8-txn-4725-0: Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:1041420389861}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210004}

].

org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:1041420389861}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210004}

] does not exist. Cannot apply modification to its children.

Comment by Sunil Kumar M S [ 19/Sep/17 ]

patch https://git.opendaylight.org/gerrit/#/c/62792/6 should fix this.

Comment by Sam Hague [ 19/Sep/17 ]

(In reply to Sunil Kumar M S from comment #19)
> patch https://git.opendaylight.org/gerrit/#/c/62792/6 should fix this.

This patch did not seem to help with bug-7826. It also looks to have added a new exception:

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-gate-stateful-nitrogen/17/odl1_exception.log.gz

2017-09-19 15:07:08,498 | INFO | nsole user karaf | core | 112 - org.apache.karaf.log.core - 4.0.9 | ROBOT MESSAGE: Starting test Delete Vm Instances
OptimisticLockFailedException

{message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was deleted by other transaction.]]}

Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was deleted by other transaction.
OptimisticLockFailedException

{message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was deleted by other transaction.]]}

Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was deleted by other transaction.
org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:176078904599989}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)table/table[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=44}

]/flow does not exist. Cannot apply modification to its children.
TransactionCommitFailedException{message=Data did not pass validation., errorList=[RpcError [message=Data did not pass validation., severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:176078904599989}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)table/table[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=44}

]/flow does not exist. Cannot apply modification to its children.]]}

Comment by Janki Chhatbar [ 11/Oct/17 ]

Seen in latest build:
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/197/odl1_karaf.log.gz

for group-id=150002

Comment by Janki Chhatbar [ 11/Oct/17 ]

Group id - 150005

2017-09-06 04:00:39,282 | INFO | nPool-1-worker-0 | dIngressServicesConfigBindHelper | 232 - org.opendaylight.genius.interfacemanager-impl - 0.3.0.SNAPSHOT | binding ingress service vpn.6bc09103-a342-47fe-a88c-1fb8ebdb0816.d0388625-4ecd-477b-ab83-c3378b013a2a for vlan port: d0388625-4ecd-477b-ab83-c3378b013a2a
2017-09-06 04:00:39,286 | WARN | rd-dispatcher-22 | ShardDataTree | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | member-1-shard-inventory-operational: Store Tx member-1-datastore-operational-fe-0-chn-8-txn-2887-0: Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=150005}

].
org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=150005}

] does not exist. Cannot apply modification to its children.
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:281)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:723)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:729)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[197:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]

Comment by Janki Chhatbar [ 11/Oct/17 ]

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/197/odl1_karaf.log.gz

Group id - 150012

2017-09-06 04:06:42,323 | INFO | eChangeHandler-0 | VpnOpStatusListener | 321 - org.opendaylight.netvirt.vpnmanager-impl - 0.5.0.SNAPSHOT | update: Processing update for vpn 61bffd7f-170e-4156-b3e4-0885521f219e with rd 61bffd7f-170e-4156-b3e4-0885521f219e
2017-09-06 04:06:42,324 | WARN | rd-dispatcher-91 | ShardDataTree | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | member-1-shard-inventory-operational: Store Tx member-1-datastore-operational-fe-0-chn-9-txn-830-0: Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=150012}

].
org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=150012}

] does not exist. Cannot apply modification to its children.
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:281)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:723)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:729)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[197:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]

Comment by Janki Chhatbar [ 11/Oct/17 ]

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/197/odl1_karaf.log.gz

Group id = 210022

2017-09-06 04:27:36,311 | INFO | nPool-1-worker-3 | NeutronPortChangeListener | 311 - org.opendaylight.netvirt.neutronvpn-impl - 0.5.0.SNAPSHOT | Of-port-interface creation for port 495c73aa-f94f-47e4-96a1-b30b8be07f38
2017-09-06 04:27:36,311 | WARN | rd-dispatcher-91 | ShardDataTree | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | member-1-shard-inventory-operational: Store Tx member-1-datastore-operational-fe-0-chn-10-txn-2865-0: Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210022}

].
org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210022}

] does not exist. Cannot apply modification to its children.
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:281)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:723)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:729)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[197:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
2017-09-06 04:27:36,312 | ERROR | lt-dispatcher-21 | TransactionChainProxy | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | Tx: member-1-datastore-operational-fe-0-chn-10-txn-2866-0 - ready future failed for previous Tx member-1-datastore-operational-fe-0-chn-10-txn-2866-0

Comment by Janki Chhatbar [ 11/Oct/17 ]

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/197/odl1_karaf.log.gz

Group id = 210018

2017-09-06 04:52:14,519 | INFO | eChangeHandler-0 | TransportZoneNotificationUtil | 301 - org.opendaylight.netvirt.elanmanager-impl - 0.5.0.SNAPSHOT | Transport zone 1a4c4e0a-d0fa-44fe-9c09-d63540750900 updated due to dpn 9011006519764 handling.
2017-09-06 04:52:14,809 | INFO | CountersThread | CountersDumperThread | 243 - org.opendaylight.infrautils.counters-impl - 1.2.0.SNAPSHOT | dsjcc.jobs_cleared: +23 , dsjcc.jobs_incomplete: 2 , dsjcc.jobs_pending: 1 , dsjcc.jobs_created: +25 , nsc.subnetmap_update: +1
2017-09-06 04:52:14,844 | WARN | rd-dispatcher-42 | ShardDataTree | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | member-1-shard-inventory-operational: Store Tx member-1-datastore-operational-fe-0-chn-11-txn-3320-0: Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210018}

].
org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:9011006519764}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter]}

/(urn:opendaylight:flow:inventory?revision=2013-08-19)group/group[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)group-id=210018}

] does not exist. Cannot apply modification to its children.
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:281)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.checkApplicable(StructuralContainerModificationStrategy.java:99)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:305)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:288)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:125)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:35)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:723)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:729)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[197:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
2017-09-06 04:52:14,846 | ERROR | ult-dispatcher-2 | TransactionChainProxy | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | Tx: member-1-datastore-operational-fe-0-chn-11-txn-3321-0 - ready future failed for previous Tx member-1-datastore-operational-fe-0-chn-11-txn-3321-0

Comment by Janki Chhatbar [ 11/Oct/17 ]

Please ignore all my comments dated 11 Oct. They are based on old logs. Apologies for the confusion.

Comment by Sam Hague [ 11/Oct/17 ]

https://git.opendaylight.org/gerrit/63896

Comment by Abhijit Kumbhare [ 16/Oct/17 ]

Anil - can you add yourself to the patch reviewers on this?

Comment by Abhijit Kumbhare [ 30/Oct/17 ]

Update in OpenFlow Plugin meeting:
A new patch (63896) for this has been provided but currently failing single feature test. Apparently does not happen in Sunil's local env - and has to depend on the CSIT which is taking time.

Comment by Sunil Kumar M S [ 31/Oct/17 ]

patch under review as issue is not observed in recent CSIT run.

Comment by Sam Hague [ 14/Nov/17 ]

carbon patch is merged and fixes the group-id exception. https://git.opendaylight.org/gerrit/63896
nitrogen wip: https://git.opendaylight.org/gerrit/63766
master: needs patch

Comment by Sam Hague [ 20/Nov/17 ]

Something regressed in carbon as the issue is back on all latest csit runs.

Comment by Abhijit Kumbhare [ 20/Nov/17 ]

Can you take a look at it Sunil?

Comment by Sunil Kumar M S [ 27/Nov/17 ]

This seems to be new issue, where there is some code changes on netvirt which is to write group to device directly and then write the same group to config datastore. In the mean while if the statistic collection is triggered this issue seems to occur.  

Comment by Sam Hague [ 28/Nov/17 ]

Sunil, would this be another fix in openflowplugin to address this new issue?

Also, any update on the nitrogen and master ports of the original patch?

Comment by Luis Gomez [ 05/Dec/17 ]

FYI patch https://git.opendaylight.org/gerrit/63896 fixed a very nasty memory leak in group stats collection, so we need the cherry-pick in all branches.

Comment by Sam Hague [ 05/Dec/17 ]

still seen on all branches: https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-carbon/214/odl_1/odl1_exception.log.gz and https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-oxygen/483/odl_1/odl1_exception.log.gz

Comment by Sam Hague [ 05/Dec/17 ]

Luis, that patch you said needs cherry-pick, but then you also reverted the patch. What was the thinking?

Comment by Luis Gomez [ 05/Dec/17 ]

To verify the patch addresses the leak. My patch only lived for the test I did, now it is abandoned.

Comment by Sunil Kumar M S [ 07/Dec/17 ]

carbon - https://git.opendaylight.org/gerrit/#/c/66244/

Nitrogen - https://git.opendaylight.org/gerrit/#/c/63766/28

Master - https://git.opendaylight.org/gerrit/#/c/66207/

Patches raised on all three branches. 

Comment by Sam Hague [ 09/Jan/18 ]

old patches abandoned and new patches pushed:

carbon: https://git.opendaylight.org/gerrit/#/c/66304/

nitrogen: https://git.opendaylight.org/gerrit/#/c/66301/

oxygen unchanged: Master - https://git.opendaylight.org/gerrit/#/c/66207/

Comment by Anil Vishnoi [ 13/Feb/18 ]

sunilkumarms can you please rebase the above patches so that we can review and merge.

Comment by Sunil Kumar M S [ 13/Feb/18 ]

Hello Anil, 

nitrogen: https://git.opendaylight.org/gerrit/#/c/66301/ patch is rebased but CSIT is unstable. You can take a look and give comments. 

Comment by Sam Hague [ 07/Mar/18 ]

Still seeing this in oxygen:

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-openstack-queens-upstream-stateful-oxygen/200/odl_1/odl1_karaf.log.gz

 

Comment by Sunil Kumar M S [ 08/Mar/18 ]

shague this patch on oxygen https://git.opendaylight.org/gerrit/#/c/69025/ is under review. 

Comment by Daniel Farrell [ 09/Mar/18 ]

sunilkumarms - 69025 didn't pass the test-openflowplugin-core job you triggered, I retriggered the same just being optimistic and trying to push something forward.

Comment by Daniel Farrell [ 09/Mar/18 ]

sunilkumarms - ecelgp is saying the CSIT results look good. Ready to merge? Is Avishnoi the right OFP committer to merge it?

Comment by Daniel Farrell [ 09/Mar/18 ]

Patch has +2, I am pinging LF admins on IRC to get it merged.

Comment by Daniel Farrell [ 09/Mar/18 ]

Merged. Thanks ecelgp, Avishnoi and abelur!

Comment by Daniel Farrell [ 09/Mar/18 ]

Can someone verify this is fixed and close the bug?

Comment by Daniel Farrell [ 12/Mar/18 ]

shague - To verify this is fixed, close bug if so.

Comment by Sam Hague [ 12/Mar/18 ]

Oxygen NetVirt csit is looking good. Good stuff!

Generated at Wed Feb 07 20:33:34 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.