Medium Seen in https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/bgpcep-csit-1node-userfeatures-all-phosphorus/180/, the tests seem to succeed, but we have the following splat in https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/bgpcep-csit-1node-userfeatures-all-phosphorus/180/odl_1/odl1_karaf.log.gz
2021-09-29T05:35:31,003 | WARN | opendaylight-cluster-data-akka.actor.default-dispatcher-14 | LocalThreePhaseCommitCohort | 291 - org.opendaylight.controller.sal-distributed-datastore - 4.0.3 | Failed to prepare transaction member-1-datastore-operational-fe-0-chn-250-txn-4-0 on backend
org.opendaylight.mdsal.common.api.OptimisticLockFailedException: Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)bgp-rib/rib/rib[{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)id=example-bgp-rib}]/peer/peer[{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)peer-id=bgp://127.0.0.5}]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$3(ShardDataTree.java:812) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:850) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:797) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:952) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:98) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:109) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:239) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:201) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:826) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:383) ~[bundleFile:?]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) ~[bundleFile:?]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:189) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:188) ~[bundleFile:?]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?]
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:241) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:189) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:188) ~[bundleFile:?]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?]
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:241) ~[bundleFile:?]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:93) ~[bundleFile:?]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:29) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:189) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:188) ~[bundleFile:?]
at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:27) ~[bundleFile:?]
at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?]
at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?]
at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) ~[bundleFile:?]
at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:923) ~[bundleFile:?]
at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:251) ~[bundleFile:?]
at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:250) ~[bundleFile:?]
at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) ~[bundleFile:?]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) ~[bundleFile:?]
at akka.actor.ActorCell.invoke(ActorCell.scala:548) ~[bundleFile:?]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[bundleFile:?]
at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[bundleFile:?]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[bundleFile:?]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) [?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) [?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) [?:?]
Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was deleted by other transaction.
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:367) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:133) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:415) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:374) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:133) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:415) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:374) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:133) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:415) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:374) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:133) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:415) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:374) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:133) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:415) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:374) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:133) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:37) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$3(ShardDataTree.java:804) ~[bundleFile:?]
... 42 more
Investigate the reason for the failure, which seems to be related to this:
2021-09-29T05:35:30,998 | INFO | epollEventLoopGroup-5-8 | BGPPeer | 234 - org.opendaylight.bgpcep.bgp-rib-impl - 0.16.7 | Session with peer 127.0.0.5 went down
2021-09-29T05:35:30,998 | INFO | epollEventLoopGroup-5-8 | BGPPeer | 234 - org.opendaylight.bgpcep.bgp-rib-impl - 0.16.7 | Closing session with peer
2021-09-29T05:35:30,998 | INFO | epollEventLoopGroup-5-8 | AbstractPeer | 234 - org.opendaylight.bgpcep.bgp-rib-impl - 0.16.7 | Closed per Peer /(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)bgp-rib/rib/rib[{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)id=example-bgp-rib}]/peer/peer[{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)peer-id=bgp://127.0.0.5}] removed
2021-09-29T05:35:31,000 | WARN | opendaylight-cluster-data-shard-dispatcher-28 | ShardDataTree | 291 - org.opendaylight.controller.sal-distributed-datastore - 4.0.3 | member-1-shard-default-operational: Store Tx member-1-datastore-operational-fe-0-chn-250-txn-4-0: Conflicting modification for path /(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)bgp-rib/rib/rib[{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)id=example-bgp-rib}]/peer/peer[{(urn:opendaylight:params:xml:ns:yang:bgp-rib?revision=2018-03-29)peer-id=bgp://127.0.0.5}].