[BGPCEP-540] ClusterSingletonService/EOS race condition Created: 02/Sep/16  Updated: 03/Mar/19  Resolved: 25/Oct/16

Status: Resolved
Project: bgpcep
Component/s: BGP
Affects Version/s: Bugzilla Migration
Fix Version/s: Bugzilla Migration

Type: Bug
Reporter: Milos Fabian Assignee: Milos Fabian
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: Text File karaf.log    
Issue Links:
Blocks
is blocked by MDSAL-195 ClusterSingletonService is not closed... Resolved
is blocked by MDSAL-197 Switch connection bounce results in w... Resolved
External issue ID: 6622

 Description   

Steps to reproduce:
1. start ODL Karaf and install bgp, netconf-connector, restconf features
2. try to reconfigure the default bgp-rib-impl instance via restconf (controller-config)
3. observe unexpected behavior:
-topology exporters are not restarted correctly (some might run, some stay closed)
-bgp-rib-impl is not initiated (the first reconfiguration attempt can work, more tries fails and the new instance is not initiated)

Looks like problem above are related to singleton-app service, probably a closing is not correct in our implementation.

2016-09-02 10:44:28,508 | INFO | entLoopGroup-8-2 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB Singleton Service ServiceGroupIdentifier

{value=Uri [_value=example-bgp-rib]-service-group}

registered
2016-09-02 10:44:28,508 | DEBUG | entLoopGroup-8-2 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | New Instance of ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 10:44:28,508 | DEBUG | entLoopGroup-8-2 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Initialization ClusterSingletonGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 10:44:28,509 | DEBUG | entLoopGroup-8-2 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | RegisterService method call for ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 10:44:28,511 | DEBUG | lt-dispatcher-27 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Ownership change DOMEntityOwnershipChange [entity=DOMEntity [type=org.opendaylight.mdsal.ServiceEntityType, id=/(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)entity/entity[

{(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)name=Uri [_value=example-bgp-rib]-service-group}

]], state=LOCAL_OWNERSHIP_LOST_NO_OWNER [wasOwner=true, isOwner=false, hasOwner=false], inJeopardy=false] for ClusterSingletonServiceGrou Uri [_value=example-bgp-rib]-service-group
2016-09-02 10:44:28,511 | DEBUG | lt-dispatcher-27 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | LostLeadership method for service Provider Uri [_value=example-bgp-rib]-service-group
2016-09-02 10:44:28,511 | DEBUG | lt-dispatcher-27 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Destroy service Instance Uri [_value=example-bgp-rib]-service-group is success
2016-09-02 10:44:28,611 | DEBUG | entLoopGroup-8-2 | BgpDeployerImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB instance created RIBImpl{}

 Comments   
Comment by Milos Fabian [ 02/Sep/16 ]

Expected:
Old instance is closed and new instance created, new instance become a new leader

2016-09-02 12:40:18,732 | DEBUG | on-dispatcher-43 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | UnregisterService method call for ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,733 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | Close RIB Singleton Service ServiceGroupIdentifier

{value=Uri [_value=example-bgp-rib]-service-group}
2016-09-02 12:40:18,746 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,751 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,756 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,762 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,766 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,767 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,770 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,778 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,781 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,783 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,785 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,787 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,790 | DEBUG | on-dispatcher-43 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Close method for service Provider Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,792 | DEBUG | on-dispatcher-43 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Destroy service Instance Uri [_value=example-bgp-rib]-service-group is success
2016-09-02 12:40:18,794 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:40:18,799 | INFO | on-dispatcher-43 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB Singleton Service ServiceGroupIdentifier{value=Uri [_value=example-bgp-rib]-service-group}

registered
2016-09-02 12:40:18,799 | DEBUG | on-dispatcher-43 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | New Instance of ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,799 | DEBUG | on-dispatcher-43 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Initialization ClusterSingletonGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,799 | DEBUG | on-dispatcher-43 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | RegisterService method call for ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,805 | DEBUG | lt-dispatcher-14 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Ownership change DOMEntityOwnershipChange [entity=DOMEntity [type=org.opendaylight.mdsal.ServiceEntityType, id=/(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)entity/entity[

{(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)name=Uri [_value=example-bgp-rib]-service-group}

]], state=LOCAL_OWNERSHIP_GRANTED [wasOwner=false, isOwner=true, hasOwner=true], inJeopardy=false] for ClusterSingletonServiceGrou Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,805 | DEBUG | lt-dispatcher-14 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | TryToTakeLeadership method for service Provider Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,817 | DEBUG | lt-dispatcher-14 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Ownership change DOMEntityOwnershipChange [entity=DOMEntity [type=org.opendaylight.mdsal.AsyncServiceCloseEntityType, id=/(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)entity/entity[

{(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)name=Uri [_value=example-bgp-rib]-service-group}

]], state=LOCAL_OWNERSHIP_GRANTED [wasOwner=false, isOwner=true, hasOwner=true], inJeopardy=false] for ClusterSingletonServiceGrou Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,818 | DEBUG | lt-dispatcher-14 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | TakeLeadership method for service Provider Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:40:18,818 | INFO | lt-dispatcher-14 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB Singleton Service ServiceGroupIdentifier

{value=Uri [_value=example-bgp-rib]-service-group}

instantiated

Comment by Milos Fabian [ 02/Sep/16 ]

Unexpected: old instance is closed, new created and registered. Then the new instance is closed, based on the received ownership change (LOCAL_OWNERSHIP_LOST_NO_OWNER) notification.

2016-09-02 12:45:33,980 | DEBUG | on-dispatcher-50 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | UnregisterService method call for ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:45:33,980 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | Close RIB Singleton Service ServiceGroupIdentifier

{value=Uri [_value=example-bgp-rib]-service-group}
2016-09-02 12:45:33,981 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:33,983 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:33,986 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:33,987 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:33,990 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,010 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,012 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,015 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,015 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,015 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,015 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,015 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,023 | DEBUG | on-dispatcher-50 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Close method for service Provider Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:45:34,023 | DEBUG | on-dispatcher-50 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Destroy service Instance Uri [_value=example-bgp-rib]-service-group is success
2016-09-02 12:45:34,023 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB KeyedInstanceIdentifier{targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib, path=[org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.BgpRib, org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.bgp.rib.rev130925.bgp.rib.Rib[key=RibKey [_id=Uri [_value=example-bgp-rib]]]]} closed successfully
2016-09-02 12:45:34,026 | INFO | on-dispatcher-50 | RIBImpl | 285 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.0.SNAPSHOT | RIB Singleton Service ServiceGroupIdentifier{value=Uri [_value=example-bgp-rib]-service-group}

registered
2016-09-02 12:45:34,026 | DEBUG | on-dispatcher-50 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | New Instance of ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:45:34,026 | DEBUG | on-dispatcher-50 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Initialization ClusterSingletonGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:45:34,026 | DEBUG | on-dispatcher-50 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | RegisterService method call for ClusterSingletonServiceGroup Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:45:34,027 | DEBUG | lt-dispatcher-29 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Ownership change DOMEntityOwnershipChange [entity=DOMEntity [type=org.opendaylight.mdsal.ServiceEntityType, id=/(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)entity/entity[

{(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)name=Uri [_value=example-bgp-rib]-service-group}

]], state=LOCAL_OWNERSHIP_LOST_NO_OWNER [wasOwner=true, isOwner=false, hasOwner=false], inJeopardy=false] for ClusterSingletonServiceGrou Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:45:34,027 | DEBUG | lt-dispatcher-29 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | LostLeadership method for service Provider Uri [_value=example-bgp-rib]-service-group
2016-09-02 12:45:34,027 | DEBUG | lt-dispatcher-29 | ClusterSingletonServiceGroupImpl | 162 - org.opendaylight.mdsal.singleton-dom-impl - 2.1.0.SNAPSHOT | Destroy service Instance Uri [_value=example-bgp-rib]-service-group is success

Comment by Milos Fabian [ 05/Sep/16 ]

Question addressed to mdsal: https://lists.opendaylight.org/pipermail/mdsal-dev/2016-September/000621.html

Comment by Peter Gubka [ 12/Sep/16 ]

The case when new instance of singleton is registered, but not initiated can be found in https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-1node-userfeatures-only-boron
Jobs 948-950. It happens in the same test, the last one.
See karaf log and look for the following log message: ROBOT MESSAGE: Starting test Odl Npaths Exa SendReceived

Comment by Vaclav Demcak [ 28/Sep/16 ]

I'd like to ask you for retesting described behavior with next patches:
MD-SAL : https://git.opendaylight.org/gerrit/#/c/46175/
Controller : https://git.opendaylight.org/gerrit/#/c/45638/

Comment by Milos Fabian [ 28/Sep/16 ]

(In reply to Vaclav Demcak from comment #5)
> I'd like to ask you for retesting described behavior with next patches:
> MD-SAL : https://git.opendaylight.org/gerrit/#/c/46175/
> Controller : https://git.opendaylight.org/gerrit/#/c/45638/

Thanks Vaso,
I did manual testing with those two patches included.
I can confirm, after a reconfiguration, the RIBImpl service is always getting initiated - working as expected.
However, this warning was observed in logs:

"2016-09-28 13:54:23,800 | WARN | lt-dispatcher-20 | ClusterSingletonServiceGroupImpl | 163 - org.opendaylight.mdsal.singleton-dom-impl - 2.2.0.SNAPSHOT | Unexpected lost doubleCandidate DOMEntity [type=org.opendaylight.mdsal.AsyncServiceCloseEntityType, id=/(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)entity/entity[

{(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)name=example-bgp-rib-service-group}

]] leadership. Close service instance example-bgp-rib-service-group"

Comment by Vaclav Demcak [ 30/Sep/16 ]

May I ask you for logs? Unexpected lost ownership for a GuardEntity is not look good.

Comment by Milos Fabian [ 03/Oct/16 ]

Attachment karaf.log has been added with description: karaf log

Comment by Milos Fabian [ 03/Oct/16 ]

(In reply to Vaclav Demcak from comment #7)
> May I ask you for logs? Unexpected lost ownership for a GuardEntity is not
> look good.

Please find attached log file - https://bugs.opendaylight.org/attachment.cgi?id=1259

Comment by Peter Gubka [ 06/Oct/16 ]

In the last few days the stability of userfeatures tests is better than usual. https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-userfeatures-only-carbon/ See jobs 175-177. But in the 177 it again occurred the case when singleton services are registered, but instantiated.

2016-10-05 22:11:49,614 | INFO | entLoopGroup-6-2 | RIBImpl | 302 - org.opendaylight.bgpcep.bgp-rib-impl - 0.7.0.SNAPSHOT | RIB Singleton Service ServiceGroupIdentifier

{value=example-bgp-rib-service-group} registered
2016-10-05 22:11:49,828 | INFO | entLoopGroup-6-2 | BgpPeer | 302 - org.opendaylight.bgpcep.bgp-rib-impl - 0.7.0.SNAPSHOT | Peer Singleton Service ServiceGroupIdentifier{value=example-bgp-rib-service-group}

registered
2016-10-05 22:11:50,330 | INFO | entLoopGroup-6-2 | AppPeer | 302 - org.opendaylight.bgpcep.bgp-rib-impl - 0.7.0.SNAPSHOT | Application Peer Singleton Service ServiceGroupIdentifier

{value=example-bgp-rib-service-group}

registered

This is just to remind that the problem is not solved in the master branch yet.

Comment by Martin Ciglan [ 17/Oct/16 ]

is this waiting for review or more work needs to be done? Many thanks.

Comment by Milos Fabian [ 17/Oct/16 ]

Hi Martin,
there are some "fixes" required on BGPCEP side. I am working on it right now.
AFAIK, also, https://git.opendaylight.org/gerrit/#/c/46761/ needs to be merged in order to get this issue fixed in Boron.

Comment by Martin Ciglan [ 17/Oct/16 ]

patch mentioned by you is bound to MDSAL-197
but that seems to be already resolved fixed already.

Could you update status please?

Comment by Milos Fabian [ 17/Oct/16 ]

moving to BGPCEP

Comment by Milos Fabian [ 17/Oct/16 ]

BGP stable/boron fix: https://git.opendaylight.org/gerrit/#/c/47018/

Comment by Milos Fabian [ 24/Oct/16 ]

master: https://git.opendaylight.org/gerrit/#/c/47416/

Generated at Wed Feb 07 19:13:23 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.