[NETVIRT-207] The member is not removed from the entity list, when we kill the karaf process. Created: 18/Oct/16  Updated: 09/Mar/18  Resolved: 02/Nov/16

Status: Resolved
Project: netvirt
Component/s: None
Affects Version/s: Boron
Fix Version/s: None

Type: Bug
Reporter: Arthi Bhattacharjee Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

External issue ID: 6959

 Description   

Setup:

  • Distribution Version: Boron RC3 Distribution.
  • Cluster setup.
  • One Control and two Compute nodes.
  • Openstack version: Mitaka.

Step to reproduce:

  • Install odl-ovsdb-openstack feature in all the ODLs
  • Check entity-owners list and identify the owner member.
  • Kill the karaf process of the owner. [[kill -9 <process>]]

Observation:

  • Ports (6640, 6653, 6633 and akka ) were released.
  • The owner is changed. The previous owner is still a candidate of the entity and it is not removed from the list.
    Waited for ample amount of time, still the member is not removed.

Note: The member was removed, when we logout from karaf.

Before killing the process
==========================
{"entity-owners":{"entity-type":[{"type":"ovsdb-southbound-provider","entity":[{"id":"/general-entity:entity[general-entity:name='ovsdb-southbound-provider']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"}]},{"type":"ovsdb-netvirt-provider","entity":[{"id":"/general-entity:entity[general-entity:name='ovsdb-netvirt-provider']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"}]},{"type":"org.opendaylight.mdsal.ServiceEntityType","entity":[{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:226669137800270']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:205192220308807']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:215885849431262']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:59578843938891']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:176904336768589']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:193652163050202']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"}]},{"type":"ovsdb","entity":[{"id":"/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/dc1f5e06-8125-4074-b903-b65ff5520827']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"},{"id":"/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/21f6c40c-f153-49ae-83ef-7e875b6c8f30']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"},{"id":"/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/5010eb43-fb53-4276-87c2-baad4b31e5f5']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-1"}]},{"type":"org.opendaylight.mdsal.AsyncServiceCloseEntityType","entity":[{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:226669137800270']","candidate":[

{"name":"member-1"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:205192220308807']","candidate":[

{"name":"member-1"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:215885849431262']","candidate":[

{"name":"member-1"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:59578843938891']","candidate":[

{"name":"member-1"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:176904336768589']","candidate":[

{"name":"member-1"}

],"owner":"member-1"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:193652163050202']","candidate":[

{"name":"member-1"}

],"owner":"member-1"}]}]}}[root@localhost log]#

After killing the process
=========================
{"entity-owners":{"entity-type":[{"type":"ovsdb-southbound-provider","entity":[{"id":"/general-entity:entity[general-entity:name='ovsdb-southbound-provider']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"}]},{"type":"ovsdb-netvirt-provider","entity":[{"id":"/general-entity:entity[general-entity:name='ovsdb-netvirt-provider']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"}]},{"type":"org.opendaylight.mdsal.ServiceEntityType","entity":[{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:226669137800270']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:205192220308807']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:215885849431262']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:59578843938891']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:176904336768589']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:193652163050202']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"}]},{"type":"ovsdb","entity":[{"id":"/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/dc1f5e06-8125-4074-b903-b65ff5520827']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"},{"id":"/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/21f6c40c-f153-49ae-83ef-7e875b6c8f30']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"},{"id":"/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/5010eb43-fb53-4276-87c2-baad4b31e5f5']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

,

{"name":"member-3"}

],"owner":"member-2"}]},{"type":"org.opendaylight.mdsal.AsyncServiceCloseEntityType","entity":[{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:226669137800270']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:205192220308807']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:215885849431262']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:59578843938891']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:176904336768589']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

],"owner":"member-2"},{"id":"/odl-general-entity:entity[odl-general-entity:name='openflow:193652163050202']","candidate":[

{"name":"member-1"}

,

{"name":"member-2"}

],"owner":"member-2"}]}]}}[root@localhost log]#

 Comments   
Comment by Vratko Polak [ 18/Oct/16 ]

I believe the described behavior is now expected. See comment [0] and the gerrit link listed there.

Is there a specific piece of documentation which needs to be updated, to make the expected behavior widely known?

[0] https://bugs.opendaylight.org/show_bug.cgi?id=6540#c15

Comment by Vratko Polak [ 18/Oct/16 ]

Oh, now I see my previous comments only apply for Singleton based services.
I am not sure whether netvirt uses Singleton or the raw Entity Ownership service.

Comment by Tom Pantelis [ 01/Nov/16 ]

(In reply to Vratko Polák from comment #1)
> I believe the described behavior is now expected. See comment [0] and the
> gerrit link listed there.
>
> Is there a specific piece of documentation which needs to be updated, to
> make the expected behavior widely known?
>
> [0] https://bugs.opendaylight.org/show_bug.cgi?id=6540#c15

Yes - this is expected. The candidate will remain until the node is restarted, in which case the candidate will either be removed or re-instated by the new incarnation. During that time, the "down" candidate is not eligible to become the entity owner.

The reasoning is all explained in the bug Vratko referenced, basically we don't know if the node process is actually down or isolated so we leave its candidates until proven otherwise.

Generated at Wed Feb 07 20:20:58 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.