[CONTROLLER-1659] C: cluster singleton - rpc failing after leader islolation

Tue, 25 Jul 2023 08:24:34 +0000

robot:
https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/665/archives/log.html.gz#s1-s44-t3-k2-k8-k1-k1-k3-k2-k1-k1-k2-k1-k4-k7-k1

logs:
https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/665/archives/odl3_karaf.log.gz
2017-05-07 18:01:12,864 | INFO | h for user karaf | command | 266 - org.apache.karaf.log.command - 3.0.8 | ROBOT MESSAGE: Starting test Isolate_Owner_Node

The singleton owner (node1) was isolated and new leader was elected (node2).
Then tests try for 1 minute to invoke rpc on node 3, but it fails with
<errors xmlns="urn:ietf:params:xml:ns:yang:ietf-restconf"><error><error-type>application</error-type><error-tag>operation-failed</error-tag><error-message>The operation encountered an unexpected error while executing.</error-message><error-info>akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://opendaylight-cluster-data@10.29.12.160:2550/user/rpc/broker#1723639021]] after [15000 ms]. Sender[null] sent message of type "org.opendaylight.controller.remote.rpc.messages.ExecuteRpc".
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)
at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:329)
at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:280)
at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:284)
at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:236)
at java.lang.Thread.run(Thread.java:745)
</error-info></error></errors>

It is similar to 8207 but 1 minute should be enough for rpc to be routed towards new owner.

OpenDaylight JIRA

[CONTROLLER-1659] C: cluster singleton - rpc failing after leader islolation