-
Bug
-
Resolution: Cannot Reproduce
-
High
-
None
-
None
-
None
Shard leader does not join back Cluster when it is isolated and rejoined
Steps to recreate:
1. Bring up 3 node cluster and make node 2 as Shard leader using "http://{{controller-ip}}:{{restconf-port}}/restconf/operations/cluster-admin:make-leader-local" API 2. Check if Node 2 is the leader for all shards and data-store-type. 3. Apply ACL on Node 1 and Node 3 to isolate Node 2. 4. Check if Node 1 or Node 3 is elected as Leader and Node 2 is "Isolated Leader" 5. When Active Leader marks Node 2 as terminated to resume Leader capability, remove the ACL applied on Node 1 and Node 3 Note - For Step 3, also tried applying ACL only on Node 2 to isolate Node 1 & 3. Same behavior.
Expected Behavior
Node 2 should join back the cluster as "Follower"
Actual Behavior
Node 2 remains as "Isolated Leader" and all 3 nodes there is "Quarantined address is still unreachable or has not been restarted" message which is seen for close to 50 minutes. After which Node 2 joins back the cluster and cluster operations are resumed.