[CONTROLLER-1688] NPE during reconnected direct commit Created: 18/May/17  Updated: 25/Jul/23  Resolved: 19/May/17

Status: Resolved
Project: controller
Component/s: clustering
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Robert Varga Assignee: Robert Varga
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


External issue ID: 8507

 Description   

After reconnect and replay of modifications, we have observed a NPE failure here: https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon-2/1/archives/log.html.gz#s1-s20-t3-k2-k10-k1.

The log snippet is:

2017-05-18 11:57:09,793 | WARN | lt-dispatcher-20 | TransmitQueue | 197 - org.opendaylight.controller.cds-access-client - 1.1.0.SNAPSHOT | No request matching Envelope{sessionId=1, txSequence=bb7, message=TransactionP
urgeResponse{target=member-2-datastore-config-fe-0-chn-2-txn-1421-0, sequence=1}} found, ignoring response
2017-05-18 11:57:09,834 | WARN | lt-dispatcher-21 | ConcurrentDOMDataBroker | 199 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Tx: DOM-CHAIN-1-2146 Error during phase CAN_COMMIT, starting Abort
java.lang.NullPointerException
at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:336)
at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:319)
at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:90)
at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:141)
at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:154)
at org.opendaylight.controller.cluster.datastore.LeaderFrontendState.handleTransactionRequest(LeaderFrontendState.java:198)
at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:461)
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:292)
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:31)
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)
at akka.actor.Actor$class.aroundReceive(Actor.scala:497)
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:664)
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

The path taken through the code indicates a direct commit ModifyTransaction, which is correctly triggering the canCommit path, at which point it encounters both openTransaction and readyCohort being null.



 Comments   
Comment by A H [ 18/May/17 ]

Is there an ETA for when a fix can be merged and this bug resolved for stable/carbon branch?

Comment by Robert Varga [ 18/May/17 ]

Prototype patch: https://git.opendaylight.org/gerrit/57387

It needs finishing up and some more logs. ETA tomorrow morning CET.

Comment by Robert Varga [ 18/May/17 ]

Err, that's when the patch will be ready for review.

Generated at Wed Feb 07 19:56:11 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.