https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/CONTROLLER-1630 controller <p>I frequently see cases where the follower prints the following after cluster is recovered. It prints it forever and does not seem to recover.</p> <p>2017-04-07 11:24:41,471 | INFO | lt-dispatcher-20 | Shard | 187 - org.opendaylight.controller.sal-akka-raft - 1.4.3.Boron-SR3 | member-2-shard-default-operational (Follower): The log is not empty but the prevLogIndex 1685 was not found in it - lastIndex: 1685, snapshotIndex: -1<br/> 2017-04-07 11:24:41,471 | INFO | lt-dispatcher-20 | Shard | 187 - org.opendaylight.controller.sal-akka-raft - 1.4.3.Boron-SR3 | member-2-shard-default-operational (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply <span class="error">&#91;term=23, success=false, followerId=member-2-shard-default-operational, logLastIndex=1685, logLastTerm=9, forceInstallSnapshot=false, payloadVersion=5, raftVersion=3&#93;</span></p> <p>Operating System: All<br/> Platform: All</p> CONTROLLER-1630

Follower not sync'ing up after rejoining cluster

Bug Resolved Cannot Reproduce Unassigned Srini Seetharaman Mon, 10 Apr 2017 15:03:36 +0000 Tue, 25 Jul 2023 08:24:29 +0000 Tue, 22 May 2018 10:50:59 +0000 clustering 0 2 <p>I think a simple solution is to treat that particular out-of-sync scenario such that the leader forces an install snapshot.</p> <p>However the out-of-sync scenario may be a symptom of another issue which resulted in an index gap. In order to determine that, we need to know what led up to the issue. It would be helpful to reproduce with org.opendaylight.controller.cluster.datastore.Shard debug enabled and provide the logs and the exact steps that were taken to reproduce along with time stamps.</p> <p>Looking at the log message, the leader's previous index, 1685, is the same as the follower's last index so it <b>should</b> find in the index in the journal log. Since the snapshotIndex was -1, there was no prior snapshot which means this check was satisfied in AbstractReplicatedLogImpl#get:</p> <p> if (adjustedIndex &lt; 0 || adjustedIndex &gt;= journal.size()) </p> { // physical index should be less than list size and &gt;= 0 return null; } <p>and lookup failed. Clearly the actual journal size is less than 1685 (adjustedIndex) even though the last entry's index is 1685 which means there's a gap in the journal list/array. The only known way this happens in Boron (that I recall) is if the operational persistent flag is changed from false to true or if one node has persistence enabled while another doesn't. If that wasn't the case in this scenario, then there's another issue and we'd need to reproduce with debug on.</p> <p>Does this happen with a newer release train (Carbon/Nitrogen)?</p> Development External issue ID 8199 External issue URL Rank 0|i02rrb: