[BGPCEP-500] ApplicationPeer's write transaction raced Created: 28/Jul/16  Updated: 03/Mar/19  Resolved: 03/Aug/16

Status: Resolved
Project: bgpcep
Component/s: BGP
Affects Version/s: Bugzilla Migration
Fix Version/s: Bugzilla Migration

Type: Bug
Reporter: Milos Fabian Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

External issue ID: 6300

 Description   

Testing BGP Application Peer performance with bgp-benchmark-app (500k routes injection), the following error occured repeatedly:

2016-07-28 21:17:41,114 | ERROR | on-dispatcher-45 | DataTreeChangeListenerActor | 170 - org.opendaylight.controller.sal-distributed-datastore - 1.3.3.SNAPSHOT | Error notifying listener org.opendaylight.protocol.bgp.rib.impl.ApplicationPeer@16df1141
java.lang.IllegalStateException: New transaction PingPongTransaction

{delegate=org.opendaylight.controller.cluster.databroker.DOMBrokerReadWriteTransaction@645b4850}

raced with transacion null
at org.opendaylight.controller.md.sal.dom.broker.impl.PingPongTransactionChain.slowAllocateTransaction(PingPongTransactionChain.java:136)[143:org.opendaylight.controller.sal-broker-impl:1.3.3.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.PingPongTransactionChain.allocateTransaction(PingPongTransactionChain.java:148)[143:org.opendaylight.controller.sal-broker-impl:1.3.3.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.PingPongTransactionChain.newReadWriteTransaction(PingPongTransactionChain.java:347)[143:org.opendaylight.controller.sal-broker-impl:1.3.3.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.broker.impl.PingPongTransactionChain.newWriteOnlyTransaction(PingPongTransactionChain.java:378)[143:org.opendaylight.controller.sal-broker-impl:1.3.3.SNAPSHOT]
at org.opendaylight.protocol.bgp.rib.impl.ApplicationPeer.onDataTreeChanged(ApplicationPeer.java:95)[260:org.opendaylight.bgpcep.bgp-rib-impl:0.5.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataChanged(DataTreeChangeListenerActor.java:53)[170:org.opendaylight.controller.sal-distributed-datastore:1.3.3.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:37)[170:org.opendaylight.controller.sal-distributed-datastore:1.3.3.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:34)[166:org.opendaylight.controller.sal-clustering-commons:1.3.3.SNAPSHOT]
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)[155:com.typesafe.akka.actor:2.3.14]
at akka.actor.Actor$class.aroundReceive(Actor.scala:467)[155:com.typesafe.akka.actor:2.3.14]
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)[155:com.typesafe.akka.actor:2.3.14]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)[155:com.typesafe.akka.actor:2.3.14]
at akka.actor.ActorCell.invoke(ActorCell.scala:487)[155:com.typesafe.akka.actor:2.3.14]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)[155:com.typesafe.akka.actor:2.3.14]
at akka.dispatch.Mailbox.run(Mailbox.scala:220)[155:com.typesafe.akka.actor:2.3.14]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)[155:com.typesafe.akka.actor:2.3.14]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[152:org.scala-lang.scala-library:2.11.7.v20150622-112736-1fbce4612c]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[152:org.scala-lang.scala-library:2.11.7.v20150622-112736-1fbce4612c]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[152:org.scala-lang.scala-library:2.11.7.v20150622-112736-1fbce4612c]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[152:org.scala-lang.scala-library:2.11.7.v20150622-112736-1fbce4612c]

The regression was introduced by the patch https://git.opendaylight.org/gerrit/#/c/42295/

 Comments   
Comment by Milos Fabian [ 28/Jul/16 ]

stable/beryllium: https://git.opendaylight.org/gerrit/#/c/42737/

Comment by A H [ 29/Jul/16 ]

A patch for this bug was submitted:

https://git.opendaylight.org/gerrit/#/c/42737/

To better assess the impact of this bug and fix, could someone from your team please help us identify the following:

Severity: Could you elaborate on the severity of this bug? Is this a BLOCKER such that we cannot release Beryllium without it? Is there a workaround such that we can write a release note and fix in future Beryllium SR4?
Testing: Could you also elaborate on the testing or this patch? How extensively has this patch been tested? Is it covered by any unit tests or system tests?
Impact: Does this fix impact any dependent projects?

Comment by Milos Fabian [ 29/Jul/16 ]

(In reply to A H from comment #2)
> A patch for this bug was submitted:
>
> https://git.opendaylight.org/gerrit/#/c/42737/
>
> To better assess the impact of this bug and fix, could someone from your
> team please help us identify the following:
>
> Severity: Could you elaborate on the severity of this bug? Is this a
> BLOCKER such that we cannot release Beryllium without it? Is there a
> workaround such that we can write a release note and fix in future Beryllium
> SR4?
> Testing: Could you also elaborate on the testing or this patch? How
> extensively has this patch been tested? Is it covered by any unit tests or
> system tests?
> Impact: Does this fix impact any dependent projects?

Severity: Yes, it is blocker. No workaround exists, I would prefer to include this change in SR3, as this is an important BGP feature. It is SR2 -> SR3 regression.
Testing: I did manual testing using ODL BGP Benchmark Java application (part of the distribution) - rerun several times. Use-case is covered by system test: https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-periodic-bgp-ingest-only-beryllium/268/robot/bgpcep-bgp-ingest.txt/Bgp%20App%20Peer%20Prefixcount/Wait_For_Ipv4_Topology_Is_Filled/
Impact: No impact - no dependent projects in ODL.

Comment by Milos Fabian [ 01/Aug/16 ]

master: https://git.opendaylight.org/gerrit/#/c/42877/

Comment by A H [ 01/Aug/16 ]

Have we verified if the bug has been fixed in the latest build for Beryllium SR3?

https://wiki.opendaylight.org/view/Simultaneous_Release:Beryllium_Release_Plan#Beryllium_SR3_Download

Comment by A H [ 03/Aug/16 ]

Verified as fixed in beryllium per this email thread: https://lists.opendaylight.org/pipermail/bgpcep-dev/2016-August/000703.html

Generated at Wed Feb 07 19:13:16 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.