[#CONTROLLER-1527] Entity Ownership cannot be reacquired after 3-nodes cluster recovers

[CONTROLLER-1527] Entity Ownership cannot be reacquired after 3-nodes cluster recovers Created: 21/Jun/16 Updated: 19/Oct/17 Resolved: 09/Sep/16
Status:	Resolved
Project:	controller
Component/s:	clustering
Affects Version/s:	Beryllium
Fix Version/s:	None

Type:

Bug

Reporter:

Vinh Nguyen

Assignee:

Unassigned

Resolution:

Cannot Reproduce

Votes:

Labels:

None

Remaining Estimate:

Not Specified

Time Spent:

Not Specified

Original Estimate:

Not Specified

Environment:

Operating System: All
Platform: All

External issue ID:

6092

Description

In a 3-node cluster when 2 of the nodes are shut down, the remain node got stuck with the ownership of the Entity after the entity is de-registered and registered.

Steps to reproduce,
1. Bring up a 3 node cluster
2. Set the managers of an OVS device to all 3 nodes in the cluster. This should result in a candidate registering for each node in the cluster.
3. Now let's say member-2 is the owner of the OVS connection entity
4. Kill the controller process for both member-1 and member-2.
5. Now the member-3 is owner of connection entity
6. Delete the managers of the OVS node
7. Re-set the managers of an OVS device to all 3 nodes in the cluster.
8. Restart the 2 nodes for member-1 and member-2
9. Delete and set manager of the OVS node.

Issue: there is no ownershipChanged event to the DeviceEntityOwnershipListener as result of Connection entity deregistered and re-registered

Comments

Comment by Vinh Nguyen [ 21/Jun/16 ]

At step 6, if the br-int is deleted at the OVS node, it is not created after step 9 when all 3 nodes up and running

Comment by Vinh Nguyen [ 31/Aug/16 ]

This is not a problem since ODL doesn't support two-node failures in a cluster of 3 ODL nodes. It means that any operations after that event are not supported

Comment by Vratko Polak [ 09/Sep/16 ]

Note that when nodes are explicitly marked as downed after step 4, the cluster should be able to transition into healthy state again after step 8.

See the 5 lines in a suite (related to plain old shard Leader, may not apply to entity ownership service):
https://git.opendaylight.org/gerrit/gitweb?p=integration/test.git;a=blob;f=csit/suites/controller/Clustering_Datastore/car_outage_corners.robot;h=8e10a1bc6fbc34e28f1a20da77d68f27089e24ef;hb=refs/heads/master#l44

Generated at Wed Feb 07 19:55:46 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.

[CONTROLLER-1527] Entity Ownership cannot be reacquired after 3-nodes cluster recovers Created: 21/Jun/16 Updated: 19/Oct/17 Resolved: 09/Sep/16