[CONTROLLER-973] Clustering : Raft Leader should recognize that it has been isolated and change to a passive state Created: 29/Oct/14 Updated: 25/Jul/23 Resolved: 01/Dec/14 |
|
| Status: | Resolved |
| Project: | controller |
| Component/s: | mdsal |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | ||
| Reporter: | Moiz Raja | Assignee: | Kamal Rameshan |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Operating System: All |
||
| Comments |
| Comment by Moiz Raja [ 31/Oct/14 ] |
|
Let's say you have a three node clusters with node A, B and C where node A was a Leader. Due to network partition node A got isolated and node B and C were put in a separate partition. Since elections are only trigger when a heart beat is not received by a Follower node A will continue to remain a Leader. On the other partition however node B and C will trigger a new election and one of them will get voted in as a Leader. The initial concern when filing this bug was that now you could have a situation where you have 2 Leaders in the cluster one in the minority partition and the other in the majority partition. On analysis it seems that this should not cause a problem as on the minority partition the Leader will not be able to apply commits to the state because it would not get a response from it's followers. At the same time the term on the minority partition Leader would remain the same. In the majority partition the term would move forward due to the triggering of elections. If commits were added to the Leader on the majority partition it's log would move forward and become the most uptodate log. Thus when the partition is remove the Leader of the minority partition will be forced to become a follower and it's log will get corrected. Having said this there may sometimes be a need to detect that a Leader has been isolated. Need to think about how that would be useful and how to model that without tampering with the raft implementation. |
| Comment by Moiz Raja [ 10/Nov/14 ] |
|
One way to handle this may be to add an additional state "MinorityLeader". The behavior of a MinorityLeader would the same as Leader however a state change would be an easier way for us to notify external components of this situation. It should be possible to transition between Leader <-> MinorityLeader. |
| Comment by Kamal Rameshan [ 20/Nov/14 ] |
|
https://git.opendaylight.org/gerrit/#/c/12799/ - merged To fix the tests a new gerrit has been added. |
| Comment by Moiz Raja [ 01/Dec/14 ] |
|
All merged to both master and stable/helium |