[NETVIRT-1041] Genius auto tunnel - tunnel not created when ODL stopped/wiped/started

Blocks — Mon, 25 Nov 2019 07:33:36 +0000

Setup: Devstack setup running stable/pike with master (oxygen). Two nodes, 1 control/compute, 1 compute.

Issue: Tunnel between compute nodes is not installed when ODL is restarted (stop, then start), but ONLY if ODL config is wiped (rm data/, journal, snapshots/) and tunnels are removed from OVS on compute nodes (if there from previous config)between ODL stop and start.

Steps to reproduce:
1. Configure genius auto-tz
2. Start ODL
3. Stack both nodes
4. Verify tunnel between compute nodes
5. Stop ODL
6. Manually remove OVS tunnel on compute nodes
7. Wipe ODL (rm data/ journal/ snapshots/)
8. Start ODL
9. install odl-netvirt-openstack
10. No tunnel in OVS on compute nodes. See this log [1]. Can't find the DPN in cache or MDSAL.
10. A little later(127 ms), the DPN is added to MDSAL. Looks like a small timing window. See log [2].

Originally, I though this issue could be reproduced just by stopping and starting ODL, but that is not the case. I could only see this issue when removing tunnels on compute node and wiping ODL config between ODL stop and ODL start

[1] 2017-12-13 00:14:31,473 | DEBUG | nPool-1-worker-0 | ItmInternalTunnelAddWorker | 244 - org.opendaylight.genius.itm-impl - 0.4.0.SNAPSHOT | No DPN in the mesh

[2] 2017-12-13 00:14:31,600 | DEBUG | eChangeHandler-0 | ElanOvsdbNodeListener | 327 - org.opendaylight.netvirt.elanmanager-impl - 0.6.0.SNAPSHOT | ElanOvsdbNodeLi
stener.add, new node detected Node{getNodeId=Uri [_value=ovsdb://uuid/02ca8a9e-06b0-4147-8cb4-38e77d882242], augmentations={interface org.opendaylight.yang.gen.v1.urn.ope
ndaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbNodeAugmentation=OvsdbNodeAugmentation{getConnectionInfo=ConnectionInfo{getLocalIp=IpAddress [_ipv4Address=Ipv4Address [
_value=10.8.125.250]], getLocalPort=PortNumber [_value=6640], getRemoteIp=IpAddress [_ipv4Address=Ipv4Address [_value=10.8.125.251]], getRemotePort=PortNumber [_value=402
18], augmentations={}}, getDatapathTypeEntry=[DatapathTypeEntry{getDatapathType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.Dat
apathTypeSystem, augmentations={}}, DatapathTypeEntry{getDatapathType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeN
etdev, augmentations={}}], getDbVersion=7.14.0, getInterfaceTypeEntry=[InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.
ns.yang.ovsdb.rev150105.InterfaceTypeSystem, augmentations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang
.ovsdb.rev150105.InterfaceTypeVxlan, augmentations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.r
ev150105.InterfaceTypeIpsecGre, augmentations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150
105.InterfaceTypeGeneve, augmentations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.Int
erfaceTypeGre, augmentations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceType
Lisp, augmentations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeInternal,
augmentations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypePatch, augment
ations={}}, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeStt, augmentations={}}
, InterfaceTypeEntry{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeTap, augmentations={}}], getMana
gedNodeEntry=[ManagedNodeEntry{getBridgeRef=OvsdbBridgeRef [_value=KeyedInstanceIdentifier

{targetType=interface [org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.ne|http://org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.ne/] twork.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.ope ndaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey [_topologyId=Uri [_value=ovsdb:1]]], org.opendayligh t.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey [_nodeId=Uri [_value=ovsdb://uuid/02ca8a9e-06b0-4147-8cb4-3 8e77d882242/bridge/br-ex]]]]}

], augmentations={}}, ManagedNodeEntry{getBridgeRef=OvsdbBridgeRef [_value=KeyedInstanceIdentifier

{targetType=interface org.opendaylight.yang .gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topolo gy.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey [_topologyId=Ur i [_value=ovsdb:1]]], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey [_nodeId=Uri [_value=o vsdb://uuid/02ca8a9e-06b0-4147-8cb4-38e77d882242/bridge/br-int]]]]}

], augmentations={}}], getManagerEntry=[ManagerEntry{getNumberOfConnections=1, getTarget=Uri [_value=tc
p:10.8.125.250:6640], isConnected=true, augmentations={}}], getOpenvswitchExternalIds=[OpenvswitchExternalIds{getExternalIdKey=odl_os_hostconfig_hostid, getExternalIdValu
e=control2, augmentations={}}, OpenvswitchExternalIds{getExternalIdKey=odl_os_hostconfig_config_odl_l2, getExternalIdValue={"allowed_network_types": ["local", "flat", "vl
an", "vxlan", "gre"], "bridge_mappings": {}, "datapath_type": "system", "supported_vnic_types": [{"vif_type": "ovs", "vnic_type": "normal", "vif_details": {"support_vhost
_user": false, "has_datapath_type_netdev": false, "uuid": "02ca8a9e-06b0-4147-8cb4-38e77d882242", "host_addresses": ["control2"]}}]}, augmentations={}}, OpenvswitchExtern
alIds{getExternalIdKey=hostname, getExternalIdValue=control2, augmentations={}}, OpenvswitchExternalIds{getExternalIdKey=system-id, getExternalIdValue=0697fb3a-40e3-4d15-
b055-9af0f2e34f1c, augmentations={}}], getOpenvswitchOtherConfigs=[OpenvswitchOtherConfigs{getOtherConfigKey=local_ip, getOtherConfigValue=192.168.254.251, augmentations=
{}}], getOvsVersion=2.6.1}}}

OpenDaylight JIRA

[NETVIRT-1041] Genius auto tunnel - tunnel not created when ODL stopped/wiped/started