[OPNFLWPLUG-798] [GROUP RECONCILIATION] Group reconciliation fails during OVS restart

Duplicate — Mon, 27 Sep 2021 09:01:56 +0000

Test flow:

Setup with two compute nodes
To create one vm instance on each compute
Setting the floating ip for 2nd vm instance (allocated on d1)
Restart OVS in loop 10 times on d1, capture group table before and after the reset;

Issue: Missing groups after restart .
Аfter initial analysis looks there is a race between the external group installation (group table) before and after OVS reset.
The screenshots showing the issue and log are attached.
Note: happened twice out of 10 attempts.

These are suspicious prints I can see in the karaf log, however these errors appear after successful reboots as well. Should it be cleaned up ?
Do you see a difference between the unsuccessful reboot in (09:12:36) vs the successful ones (for example 09:12:40)?
In both cases it seems like the group is "installed" even though the port is not up according to the log message - is there any way to actually verify this? Any traces to enable?

2016-10-13 09:12:25,847 | ERROR | pool-45-thread-3 | FlowNodeReconciliationImpl | 342 - org.opendaylight.openflowplugin.applications.forwardingrules-manager - 0.4.0.SNAPSHOT | Not yet received the node-connector updated for 3 for the group with id GroupId [_value=210000]
2016-10-13 09:12:25,847 | INFO | n-dispatcher-333 | ControllerUpdateCommand | 293 - org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | Register ODL controllers : {} bridges detail : {}
2016-10-13 09:12:25,847 | TRACE | pool-45-thread-3 | FlowNodeReconciliationImpl | 342 - org.opendaylight.openflowplugin.applications.forwardingrules-manager - 0.4.0.SNAPSHOT | awaitGroups() completed: node=openflow:132572872974439
2016-10-13 09:12:25,847 | WARN | pool-45-thread-3 | FlowNodeReconciliationImpl | 342 - org.opendaylight.openflowplugin.applications.forwardingrules-manager - 0.4.0.SNAPSHOT | Installing the group for node connector 3
2016-10-13 09:12:25,847 | ERROR | pool-45-thread-3 | FlowNodeReconciliationImpl | 342 - org.opendaylight.openflowplugin.applications.forwardingrules-manager - 0.4.0.SNAPSHOT | Not yet received the node-connector updated for 3 for the group with id GroupId [_value=210002]
2016-10-13 09:12:25,848 | ERROR | pool-45-thread-3 | FlowNodeReconciliationImpl | 342 - org.opendaylight.openflowplugin.applications.forwardingrules-manager - 0.4.0.SNAPSHOT | These Groups are pointing to node-connectors that are not up yet [Group{getBuckets=Buckets{getBucket=[Bucket{getAction=[Action{getAction=GroupActionCase{getGroupAction=GroupAction{getGroupId=209999, augmentations={}}, augmentations={}}, getOrder=1, augmentations={}}], getBucketId=BucketId [_value=0], getWatchGroup=4294967295, getWatchPort=4294967295, getWeight=0, augmentations={}}, Bucket{getAction=[Action{getAction=OutputActionCase{getOutputAction=OutputAction{getMaxLength=0, getOutputNodeConnector=Uri [_value=3], augmentations={}}, augmentations={}}, getOrder=1, augmentations={}}, Action{getAction=SetFieldCase{getSetField=SetField{getTunnel=Tunnel{getTunnelId=5000, augmentations={}}, augmentations={}}, augmentations={}}, getOrder=0, augmentations={}}], getBucketId=BucketId [_value=1], getWatchGroup=4294967295, getWatchPort=4294967295, getWeight=0, augmentations={}}], augmentations={}}, getGroupId=GroupId [_value=210000], getGroupName=7a73adf3-f6b2-488e-b2b5-c3b7513fb9f0, getGroupType=GroupAll, augmentations={}}, Group{getBuckets=Buckets{getBucket=[Bucket{getAction=[Action{getAction=GroupActionCase{getGroupAction=GroupAction{getGroupId=210001, augmentations={}}, augmentations={}}, getOrder=1, augmentations={}}], getBucketId=BucketId [_value=0], getWatchGroup=4294967295, getWatchPort=4294967295, getWeight=0, augmentations={}}, Bucket{getAction=[Action{getAction=OutputActionCase{getOutputAction=OutputAction{getMaxLength=0, getOutputNodeConnector=Uri [_value=3], augmentations={}}, augmentations={}}, getOrder=1, augmentations={}}, Action{getAction=SetFieldCase{getSetField=SetField{getTunnel=Tunnel{getTunnelId=5001, augmentations={}}, augmentations={}}, augmentations={}}, getOrder=0, augmentations={}}], getBucketId=BucketId [_value=1], getWatchGroup=4294967295, getWatchPort=4294967295, getWeight=0, augmentations={}}], augmentations={}}, getGroupId=GroupId [_value=210002], getGroupName=62344e07-e250-45e9-a778-34608ea6889b, getGroupType=GroupAll, augmentations={}}]
2016-10-13 09:12:25,848 | ERROR | pool-45-thread-3 | FlowNodeReconciliationImpl | 342 - org.opendaylight.openflowplugin.applications.forwardingrules-manager - 0.4.0.SNAPSHOT | Installing the group GroupId [_value=210000] finally although the port is not up after checking for 5 times
2016-10-13 09:12:25,848 | ERROR | pool-45-thread-3 | FlowNodeReconciliationImpl | 342 - org.opendaylight.openflowplugin.applications.forwardingrules-manager - 0.4.0.SNAPSHOT | Installing the group GroupId [_value=210002] finally although the port is not up after checking for 5 times
2016-10-13 09:12:25,857 | ERROR | pool-47-thread-1 | SubnetOpDpnManager | 357 - org.opendaylight.netvirt.vpnmanager-impl - 0.4.0.SNAPSHOT | Cannot get, portOp for port 388b1e67-1f7d-4f1f-86b7-6ca596b88197 is not available in datastore

OpenDaylight JIRA

[OPNFLWPLUG-798] [GROUP RECONCILIATION] Group reconciliation fails during OVS restart