Metadata not available for modification NodeModification

This may be a bug in yangtools or md-sal or clustering, but similar CONTROLLER-705 seemed to be caused by application, so I am filling this against BGP first.

When trying to connect 600 BGP peers (each with one route to advertise), the exception below appears in karaf.log (I will try to reproduce with more verbose logging).

This appears both on Lithium-SR1 and on recent Lithium snapshot builds.

The suite fails due to bugs in test code, so it is currently unknown what consequences the exception has on BGP functionality.

2015-10-16 10:05:57,293 | ERROR | ult-dispatcher-2 | Dispatcher | 177 - com.typesafe.akka.slf4j - 2.3.10 | Metadata not available for modification NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)peer[

{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)peer-id=bgp://127.0.0.150}

], modificationType=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)effective-rib-in=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)effective-rib-in, modificationType=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables, modificationType=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables[

{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)ipv4-address-family, (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)unicast-subsequent-address-family}

]=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables[

{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)ipv4-address-family, (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)unicast-subsequent-address-family}

], modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)attributes=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)attributes, modificationType=WRITE, childModification={}], (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)routes=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)routes, modificationType=WRITE, childModification={}], (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi, modificationType=WRITE, childModification={}], (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi, modificationType=WRITE, childModification={}]}]}]}]}]
java.lang.IllegalArgumentException: Metadata not available for modification NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)peer[

{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)peer-id=bgp://127.0.0.150}

], modificationType=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)effective-rib-in=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)effective-rib-in, modificationType=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables, modificationType=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables[

{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)ipv4-address-family, (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)unicast-subsequent-address-family}

]=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)tables[

{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)ipv4-address-family, (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi=(urn:opendaylight:params:xml:ns:yang:bgp-types?revision=2013-09-19)unicast-subsequent-address-family}

], modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)attributes=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)attributes, modificationType=WRITE, childModification={}], (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)routes=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)routes, modificationType=WRITE, childModification={}], (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)afi, modificationType=WRITE, childModification={}], (urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2013-09-25)safi, modificationType=WRITE, childModification={}]}]}]}]}]
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)[72:com.google.guava:18.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:196)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:117)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:152)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:198)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:117)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:152)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:198)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:117)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:152)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:198)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:117)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:152)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:198)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:117)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:152)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:198)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTreeModification.newModification(InMemoryDataTreeModification.java:199)[97:org.opendaylight.yangtools.yang-data-impl:0.7.3.SNAPSHOT]
at org.opendaylight.controller.sal.core.spi.data.SnapshotBackedWriteTransaction.<init>(SnapshotBackedWriteTransaction.java:50)[165:org.opendaylight.controller.sal-core-spi:1.2.3.SNAPSHOT]
at org.opendaylight.controller.sal.core.spi.data.SnapshotBackedReadWriteTransaction.<init>(SnapshotBackedReadWriteTransaction.java:35)[165:org.opendaylight.controller.sal-core-spi:1.2.3.SNAPSHOT]
at org.opendaylight.controller.sal.core.spi.data.AbstractSnapshotBackedTransactionChain.newReadWriteTransaction(AbstractSnapshotBackedTransactionChain.java:142)[165:org.opendaylight.controller.sal-core-spi:1.2.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.LocalTransactionChain.newReadWriteTransaction(LocalTransactionChain.java:71)[191:org.opendaylight.controller.sal-distributed-datastore:1.2.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory.createLocalTransactionContext(AbstractTransactionContextFactory.java:196)[191:org.opendaylight.controller.sal-distributed-datastore:1.2.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory.maybeCreateLocalTransactionContext(AbstractTransactionContextFactory.java:58)[191:org.opendaylight.controller.sal-distributed-datastore:1.2.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory.onFindPrimaryShardSuccess(AbstractTransactionContextFactory.java:73)[191:org.opendaylight.controller.sal-distributed-datastore:1.2.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory.access$000(AbstractTransactionContextFactory.java:35)[191:org.opendaylight.controller.sal-distributed-datastore:1.2.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory$1.onComplete(AbstractTransactionContextFactory.java:108)[191:org.opendaylight.controller.sal-distributed-datastore:1.2.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.AbstractTransactionContextFactory$1.onComplete(AbstractTransactionContextFactory.java:104)[191:org.opendaylight.controller.sal-distributed-datastore:1.2.3.SNAPSHOT]
at akka.dispatch.OnComplete.internal(Future.scala:248)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.OnComplete.internal(Future.scala:245)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:175)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:172)[176:com.typesafe.akka.actor:2.3.10]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)[173:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:90)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:88)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:88)[176:com.typesafe.akka.actor:2.3.10]
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)[173:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:88)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)[176:com.typesafe.akka.actor:2.3.10]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:401)[176:com.typesafe.akka.actor:2.3.10]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[173:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:1253)[173:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1346)[173:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[173:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[173:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]