[NETVIRT-829] IllegalArgumentException: vrfEntry is missing mandatory descendant origin Created: 08/Aug/17  Updated: 30/Oct/17  Resolved: 11/Oct/17

Status: Resolved
Project: netvirt
Component/s: General
Affects Version/s: Nitrogen
Fix Version/s: None

Type: Bug
Reporter: Janki Chhatbar Assignee: Kiran N Upadhyaya
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: File odl1_karaf.log.tar.xz    
Issue Links:
Duplicate
is duplicated by NETVIRT-903 IllegalArgumentException: All keys mu... Resolved
is duplicated by NETVIRT-921 TransactionCommitFailedException: Job... Resolved
External issue ID: 8953

 Comments   
Comment by Janki Chhatbar [ 08/Aug/17 ]

Seen in https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/165/odl1_karaf.log.gz

Failing tests:

1. suites/netvirt/ElanService/ElanService.robot

  • Test: Verify Datapath After OVS Restart
Comment by Janki Chhatbar [ 08/Aug/17 ]

2017-08-07 22:16:13,366 | INFO | eChangeHandler-0 | VpnInterfaceOpListener | 319 - org.opendaylight.netvirt.vpnmanager-impl - 0.5.0.SNAPSHOT | VpnInterfaceOpListener updated: original VpnInterface{getDpnId=229695899772772, getName=229695899772772:br-physnet1-pa:trunk, getVpnInstanceName=c4d47f1c-ba83-4c85-8356-6b1827ac1410, isScheduledForRemove=true, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.Adjacencies=Adjacencies{getAdjacency=[Adjacency{getAdjacencyType=PrimaryAdjacency, getIpAddress=10.10.10.250/32, getLabel=100034, getMacAddress=F6:00:00:FF:01:01, getNextHopIpList=[], getSubnetId=Uuid [_value=c8fb0c64-9251-48a8-9880-f39f3da03654], isPhysNetworkFunc=true, augmentations={}}]}}} updated VpnInterface{getDpnId=229695899772772, getName=229695899772772:br-physnet1-pa:trunk, getVpnInstanceName=c4d47f1c-ba83-4c85-8356-6b1827ac1410, isScheduledForRemove=true, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.Adjacencies=Adjacencies{getAdjacency=[Adjacency{getAdjacencyType=PrimaryAdjacency, getIpAddress=10.10.10.250/32, getLabel=100034, getMacAddress=F6:00:00:FF:01:01, getNextHopIpList=[10.29.13.90], getSubnetId=Uuid [_value=c8fb0c64-9251-48a8-9880-f39f3da03654], isPhysNetworkFunc=true, augmentations={}}]}}}
2017-08-07 22:16:13,366 | ERROR | rd-dispatcher-39 | SimpleShardDataTreeCohort | 210 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | Transaction member-1-datastore-config-fe-0-txn-138786-0 failed to prepare
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:395)[26:com.google.guava:22.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer$Strict.enforceOnData(MandatoryLeafEnforcer.java:43)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer.enforceOnTreeNode(MandatoryLeafEnforcer.java:58)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyWrite(ListEntryModificationStrategy.java:49)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:210)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyTouch(ListEntryModificationStrategy.java:55)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyMerge(ListEntryModificationStrategy.java:40)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:71)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:200)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:820)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:103)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(CohortEntry.java:101)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(ShardCommitCoordinator.java:295)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:242)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:725)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:721)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:325)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[195:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:31)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[36:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[33:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[33:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]

2017-08-07 22:16:13,367 | ERROR | rd-dispatcher-39 | Shard | 203 - org.opendaylight.controller.sal-clustering-commons - 1.6.0.SNAPSHOT | member-1-shard-default-config An exception occurred while preCommitting transaction member-1-datastore-config-fe-0-txn-138786-0
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:395)[26:com.google.guava:22.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer$Strict.enforceOnData(MandatoryLeafEnforcer.java:43)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer.enforceOnTreeNode(MandatoryLeafEnforcer.java:58)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyWrite(ListEntryModificationStrategy.java:49)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:210)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyTouch(ListEntryModificationStrategy.java:55)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyMerge(ListEntryModificationStrategy.java:40)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:71)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:200)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:820)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:103)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(CohortEntry.java:101)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(ShardCommitCoordinator.java:295)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:242)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:725)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:721)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:325)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[195:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:31)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[36:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[33:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[33:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
2017-08-07 22:16:13,368 | WARN | ult-dispatcher-5 | ConcurrentDOMDataBroker | 210 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | Tx: DOM-221395 Error during phase CAN_COMMIT, starting Abort
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:395)[26:com.google.guava:22.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer$Strict.enforceOnData(MandatoryLeafEnforcer.java:43)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer.enforceOnTreeNode(MandatoryLeafEnforcer.java:58)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyWrite(ListEntryModificationStrategy.java:49)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:210)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyTouch(ListEntryModificationStrategy.java:55)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyMerge(ListEntryModificationStrategy.java:40)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:71)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:200)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:820)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:103)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(CohortEntry.java:101)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(Sha...

Comment by Janki Chhatbar [ 08/Aug/17 ]

2017-08-07 22:16:13,368 | ERROR | lt-dispatcher-45 | LocalThreePhaseCommitCohort | 210 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | Failed to prepare transaction member-1-datastore-config-fe-0-txn-138786-0 on backend
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:395)[26:com.google.guava:22.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer$Strict.enforceOnData(MandatoryLeafEnforcer.java:43)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer.enforceOnTreeNode(MandatoryLeafEnforcer.java:58)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyWrite(ListEntryModificationStrategy.java:49)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:210)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyTouch(ListEntryModificationStrategy.java:55)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyMerge(ListEntryModificationStrategy.java:40)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:71)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:200)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:820)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:103)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(CohortEntry.java:101)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(ShardCommitCoordinator.java:295)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:242)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:725)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:721)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:325)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[195:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:31)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[36:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[33:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[33:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
2017-08-07 22:16:13,373 | ERROR | nPool-1-worker-0 | MDSALUtil | 236 - org.opendaylight.genius.mdsalutil-api - 0.3.0.SNAPSHOT | Error writing to datastore (path, data) : (KeyedInstanceIdentifier

{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.fibmanager.rev150330.vrfentrybase.RoutePaths, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.fibmanager.rev150330.FibEntries, org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.fibmanager.rev150330.fibentries.VrfTables[key=VrfTablesKey [_routeDistinguisher=c4d47f1c-ba83-4c85-8356-6b1827ac1410]], org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.fibmanager.rev150330.vrfentries.VrfEntry[key=VrfEntryKey [_destPrefix=10.10.10.250/32]], org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.fibmanager.rev150330.vrfentrybase.RoutePaths[key=RoutePathsKey [_nexthopAddress=10.29.13.90]]]}

, RoutePaths [_key=RoutePathsKey [_nexthopAddress=10.29.13.90], _label=100034, _nexthopAddress=10.29.13.90, augmentation=[]])
2017-08-07 22:16:13,373 | ERROR | nPool-1-worker-0 | DataStoreJobCoordinator | 236 - org.opendaylight.genius.mdsalutil-api - 0.3.0.SNAPSHOT | Exception when executing jobEntry: JobEntry

{key='VPNINTERFACE-229695899772772:br-physnet1-pa:trunk', mainWorker=org.opendaylight.netvirt.vpnmanager.TunnelInterfaceStateListener$UpdateVpnInterfaceOnTunnelEvent@6939bae3, rollbackWorker=null, retryCount=0, futures=null}

java.lang.RuntimeException: TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin]]}
at org.opendaylight.genius.mdsalutil.MDSALUtil.syncWrite(MDSALUtil.java:585)[236:org.opendaylight.genius.mdsalutil-api:0.3.0.SNAPSHOT]
at org.opendaylight.netvirt.fibmanager.FibUtil.updateRoutePathForFibEntry(FibUtil.java:431)[301:org.opendaylight.netvirt.fibmanager-impl:0.5.0.SNAPSHOT]
at org.opendaylight.netvirt.fibmanager.FibManagerImpl.updateRoutePathForFibEntry(FibManagerImpl.java:156)[301:org.opendaylight.netvirt.fibmanager-impl:0.5.0.SNAPSHOT]
at Proxy1a0e62b6_6ba8_44a9_8779_79babbf2a542.updateRoutePathForFibEntry(Unknown Source)[:]
at Proxyee1341d4_ee49_4a43_9858_d398cb4d7527.updateRoutePathForFibEntry(Unknown Source)[:]
at org.opendaylight.netvirt.vpnmanager.VpnInterfaceManager.updateVpnInterfaceOnTepAdd(VpnInterfaceManager.java:782)[319:org.opendaylight.netvirt.vpnmanager-impl:0.5.0.SNAPSHOT]
at org.opendaylight.netvirt.vpnmanager.TunnelInterfaceStateListener$UpdateVpnInterfaceOnTunnelEvent.call(TunnelInterfaceStateListener.java:433)[319:org.opendaylight.netvirt.vpnmanager-impl:0.5.0.SNAPSHOT]
at org.opendaylight.netvirt.vpnmanager.TunnelInterfaceStateListener$UpdateVpnInterfaceOnTunnelEvent.call(TunnelInterfaceStateListener.java:404)[319:org.opendaylight.netvirt.vpnmanager-impl:0.5.0.SNAPSHOT]
at org.opendaylight.genius.datastoreutils.DataStoreJobCoordinator$MainTask.run(DataStoreJobCoordinator.java:285)[236:org.opendaylight.genius.mdsalutil-api:0.3.0.SNAPSHOT]
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)[:1.8.0_131]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)[:1.8.0_131]
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)[:1.8.0_131]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)[:1.8.0_131]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)[:1.8.0_131]
Caused by: TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin]]}
at org.opendaylight.controller.md.sal.dom.broker.impl.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37)[200:org.opendaylight.controller.sal-broker-impl:1.6.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18)[200:org.opendaylight.controller.sal-broker-impl:1.6.0.SNAPSHOT]
at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:103)[364:org.opendaylight.yangtools.util:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:231)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.access$000(ConcurrentDOMDataBroker.java:46)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:126)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1228)[26:com.google.guava:22.0.0]
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:399)[26:com.google.guava:22.0.0]
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:911)[26:com.google.guava:22.0.0]
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:822)[26:com.google.guava:22.0.0]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:686)[26:com.google.guava:22.0.0]
at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:54)[26:com.google.guava:22.0.0]
at org.opendaylight.controller.cluster.datastore.SingleCommitCohortProxy$1.onComplete(SingleCommitCohortProxy.java:58)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at akka.dispatch.OnComplete.internal(Future.scala:258)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.OnComplete.internal(Future.scala:256)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:186)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:183)[33:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)[33:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)[33:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[388:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
Caused by: java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:395)[26:com.google.guava:22.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer$Strict.enforceOnData(MandatoryLeafEnforcer.java:43)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer.enforceOnTreeNode(MandatoryLeafEnforcer.java:58)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyWrite(ListEntryModificationStrategy.java:49)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:210)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyTouch(ListEntryModificationStrategy.java:55)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyMerge(ListEntryModificationStrategy.java:40)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:71)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:200)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53)[369:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:820)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:103)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(CohortEntry.java:101)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(ShardCommitCoordinator.java:295)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:242)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:725)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:721)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:325)[210:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[195:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:31)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[36:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[203:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[33:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[36:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[36:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[33:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[33:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[33:com.typesafe.akka.actor:2.4.18]
... 4 more

Comment by Sam Hague [ 12/Aug/17 ]

latest: https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-nitrogen/132/odl1_karaf.log.gz

Comment by Vivekanandan Narasimhan [ 22/Aug/17 ]

While analyzing NETVIRT-829 through the CSIT logs, we noticed that external network ‘physnet1’(UUID: 42107efb-a96d-4806-bfe6-5f011cb80f54) created during tempest tests is never deleted.
This seems to be causing stale VPN operational data, leading to the exception mentioned in the bug in a later Test case(Elan Service Suite).

Our observations:
The last test case in tempest shows that the ‘physnet1’ network is not deleted from the CONFIG neutron data-store.
However, the dump from the very next test case in NeutronSecurityGrp Suite does not contain this external network in config neutron data store.
How is this network (and other resources created by tempest) being cleaned up exactly?

If this external network has NOT gone through a valid deletion cycle, then this will have lead to stale operational data causing the reported CSIT exception. In this case, this bug can be resolved by the tempest suite cleaning up its resources once the tests are done.
If the network DID go through a valid deletion cycle, then external network deletion has some problems in the code and will need to be fixed.

The CSIT link for reference:
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-nitrogen/132/log.html.gz#s1-s1-s1

Comment by Kiran N Upadhyaya [ 23/Aug/17 ]

https://git.opendaylight.org/gerrit/#/c/62186/

Comment by Kiran N Upadhyaya [ 12/Sep/17 ]

https://git.opendaylight.org/gerrit/#/c/62957/

Comment by Sam Hague [ 19/Sep/17 ]

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-gate-stateful-oxygen/41/

Comment by Sam Hague [ 19/Sep/17 ]

Attachment odl1_karaf.log.tar.xz has been added with description: odl1_karaf.log

Comment by Vivekanandan Narasimhan [ 19/Sep/17 ]

Raised this to resolve this issue further:
https://git.opendaylight.org/gerrit/63238

Comment by Sam Hague [ 19/Sep/17 ]

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-gate-stateful-oxygen/41/odl1_exception.log.gz

2017-09-19 09:10:11,275 | INFO | nsole user karaf | core | 112 - org.apache.karaf.log.core - 4.0.9 | ROBOT MESSAGE: Starting test Verify Datapath After OVS Restart
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin

2017-09-19 09:10:14,584 | ERROR | nPool-2-worker-1 | DataStoreJobCoordinator | 243 - org.opendaylight.genius.mdsalutil-api - 0.4.0.SNAPSHOT | Exception when executing jobEntry: JobEntry

{key='VPNINTERFACE-114880759651325:br-physnet1-pa:trunk', mainWorker=org.opendaylight.netvirt.vpnmanager.TunnelInterfaceStateListener$UpdateVpnInterfaceOnTunnelEvent@3b05404f, rollbackWorker=null, retryCount=0, futures=null}

java.lang.RuntimeException: TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION,

Comment by Kiran N Upadhyaya [ 25/Sep/17 ]

https://git.opendaylight.org/gerrit/63482

Comment by Janki Chhatbar [ 11/Oct/17 ]

Reopening, seen in
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/197/odl1_karaf.log.gz

2017-09-06 04:53:58,644 | INFO | nPool-1-worker-0 | VpnInterfaceManager | 321 - org.opendaylight.netvirt.vpnmanager-impl - 0.5.0.SNAPSHOT | updateVpnInterfaceOnTepAdd: Updated label mapper : label 100028 dpn 9011006519764 prefix 10.10.10.250/32 nexthoplist [10.29.14.90] vpn 098e7860-5da5-45e4-b1c0-34ae67c433bd vpnid 100010 rd 098e7860-5da5-45e4-b1c0-34ae67c433bd interface 9011006519764:br-physnet1-pa:trunk
2017-09-06 04:53:58,644 | INFO | eChangeHandler-0 | TunnelInterfaceStateListener | 321 - org.opendaylight.netvirt.vpnmanager-impl - 0.5.0.SNAPSHOT | handleTunnelEventForDPN: Remote DpnId 9011006519764 VpnId 100010 rd 098e7860-5da5-45e4-b1c0-34ae67c433bd VpnInterface VpnInterface{getDpnId=9011006519764, getName=9011006519764:br-physnet1-pa:trunk, getVpnInstanceName=098e7860-5da5-45e4-b1c0-34ae67c433bd, isScheduledForRemove=true, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.Adjacencies=Adjacencies{getAdjacency=[Adjacency{getAdjacencyType=PrimaryAdjacency, getIpAddress=10.10.10.250/32, getLabel=100028, getMacAddress=F6:00:00:FF:01:01, getNextHopIpList=[10.29.14.90], getSubnetId=Uuid [_value=61bffd7f-170e-4156-b3e4-0885521f219e], isPhysNetworkFunc=true, augmentations={}}]}}} srcTepIp 10.29.15.141 destTepIp 10.29.14.90
2017-09-06 04:53:58,647 | ERROR | rd-dispatcher-25 | SimpleShardDataTreeCohort | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | Transaction member-1-datastore-config-fe-0-txn-151828-0 failed to prepare
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[

{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}

] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:395)[27:com.google.guava:22.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer$Strict.enforceOnData(MandatoryLeafEnforcer.java:43)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer.enforceOnTreeNode(MandatoryLeafEnforcer.java:58)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyWrite(ListEntryModificationStrategy.java:49)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:210)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyTouch(ListEntryModificationStrategy.java:55)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyMerge(ListEntryModificationStrategy.java:40)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:71)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:200)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:820)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:103)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(CohortEntry.java:101)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(ShardCommitCoordinator.java:295)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:242)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:725)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:729)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[197:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
2017-09-06 04:53:58,648 | ERROR | rd-dispatcher-25 | Shard | 205 - org.opendaylight.controller.sal-clustering-commons - 1.6.0.SNAPSHOT | member-1-shard-default-config An exception occurred while preCommitting transaction member-1-datastore-config-fe-0-txn-151828-0

Comment by Vivekanandan Narasimhan [ 11/Oct/17 ]

Janki,

Why are you re-opening bugs with 6th September logs.

The fix for this landed only during end of September.

Please check for logs after Oct 1, to make sure this issue continues to exist.

Comment by Janki Chhatbar [ 11/Oct/17 ]

Oh, sorry did not realise the date. I just took the last Nitrogen build which happens to be on Sept.

Please ignore my comments. Closing it again

Generated at Wed Feb 07 20:22:34 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.