https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/CONTROLLER-804 controller <p>For steps checkout the clustering disaster recovery scripts in the integration repo.</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/11021/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/11021/</a></p> <p>Operating System: Mac OS<br/> Platform: PC</p> CONTROLLER-804

Clustering : Restarting a controller node in a 3 node cluster causes it to not be able to find the leader

Bug Resolved Cannot Reproduce Moiz Raja Moiz Raja Thu, 11 Sep 2014 04:41:50 +0000 Tue, 16 Sep 2014 00:03:42 +0000 Tue, 16 Sep 2014 00:03:42 +0000 Helium mdsal 0 3 <p>2014-09-11 02:59:09,916 | WARN | lt-dispatcher-18 | ShardManager | 152 - com.typesafe.akka.slf4j - 2.3.4 | akka://opendaylight-cluster-data/user/shardmanager-config | Supervisor Strategy of resume applied<br/> at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2412)<br/> at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2380)<br/> at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)<br/> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)<br/> at com.google.common.cache.LocalCache.get(LocalCache.java:4000)<br/> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004)<br/> at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)<br/> at org.opendaylight.yangtools.yang.data.impl.schema.tree.DataNodeContainerModificationStrategy.getChild(DataNodeContainerModificationStrategy.java:81)<br/> at org.opendaylight.yangtools.yang.data.impl.schema.tree.DataNodeContainerModificationStrategy$ContainerModificationStrategy.getChild(DataNodeContainerModificationStrategy.java:119)<br/> at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.getChild(RootModificationApplyOperation.java:66)<br/> at org.opendaylight.yangtools.yang.data.impl.schema.tree.TreeNodeUtils.findNodeChecked(TreeNodeUtils.java:53)<br/> at org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTreeModification.resolveModificationStrategy(InMemoryDataTreeModification.java:137)<br/> at org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTreeModification.resolveModificationFor(InMemoryDataTreeModification.java:143)<br/> at org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:73)<br/> at org.opendaylight.controller.md.sal.dom.store.impl.SnapshotBackedWriteTransaction.merge(SnapshotBackedWriteTransaction.java:85)<br/> at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:37)<br/> at org.opendaylight.controller.cluster.datastore.modification.MutableCompositeModification.apply(MutableCompositeModification.java:33)<br/> at org.opendaylight.controller.cluster.datastore.Shard.commit(Shard.java:330)<br/> at org.opendaylight.controller.cluster.datastore.Shard.applyState(Shard.java:444)<br/> at org.opendaylight.controller.cluster.raft.RaftActor.onReceiveRecover(RaftActor.java:151)<br/> at org.opendaylight.controller.cluster.datastore.Shard.onReceiveRecover(Shard.java:175)<br/> at akka.persistence.UntypedPersistentActor$$anonfun$receiveRecover$1.applyOrElse(Eventsourced.scala:433)<br/> at scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)<br/> at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)<br/> at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)<br/> at akka.persistence.Eventsourced$$anonfun$akka$persistence$Eventsourced$$recoveryBehavior$1.applyOrElse(Eventsourced.scala:168)<br/> at akka.persistence.Recovery$State$$anonfun$processPersistent$1.apply(Recovery.scala:33)<br/> at akka.persistence.Recovery$State$$anonfun$processPersistent$1.apply(Recovery.scala:33)<br/> at akka.persistence.Recovery$class.withCurrentPersistent(Recovery.scala:176)<br/> at akka.persistence.UntypedPersistentActor.withCurrentPersistent(Eventsourced.scala:428)<br/> at akka.persistence.Recovery$State$class.processPersistent(Recovery.scala:33)<br/> at akka.persistence.Recovery$$anon$1.processPersistent(Recovery.scala:95)<br/> at akka.persistence.Recovery$$anon$1.aroundReceive(Recovery.scala:101)<br/> at akka.persistence.Recovery$class.aroundReceive(Recovery.scala:256)<br/> at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(Eventsourced.scala:428)<br/> at akka.persistence.Eventsourced$$anon$1.aroundReceive(Eventsourced.scala:35)<br/> at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:369)<br/> at akka.persistence.UntypedPersistentActor.aroundReceive(Eventsourced.scala:428)<br/> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)<br/> at akka.actor.ActorCell.invoke(ActorCell.scala:487)<br/> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)<br/> at akka.dispatch.Mailbox.run(Mailbox.scala:220)<br/> at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)<br/> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)<br/> at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)<br/> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)<br/> at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)<br/> 2014-09-11 02:59:09,916 | WARN | lt-dispatcher-18 | OneForOneStrategy | 152 - com.typesafe.akka.slf4j - 2.3.4 | akka://opendaylight-cluster-data/user/shardmanager-config/member-3-shard-default-config | CacheLoader returned null for key (urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people?revision=2014-08-18)car-people.</p> <p>This happens to be an issue with configuration. The issue was kind of like this. </p> <p>Let's say you have 3 nodes and node 1 was setup to be the only seed node. Now you bring down the seed node and bring it back up. When the leader node comes back up it tries to join cluster however the cluster has changed because a new leader was elected. </p> <p>The fix for this is one of the following,</p> <p>a. When the leader is restarted set the seed node for it as one of the existing nodes in the cluster (the one that is running)<br/> b. In a 3 node cluster provide all nodes as the seed nodes.</p> Development External issue ID 1803 External issue URL ODL SR Target Milestone Priority Rank 0|i02mnr: