Openstack rocky + ODL Fluorine SR1.

I noticed it includes mplsgre encap type but I can't see there is a gre tunnel there. I also noticed there are several flow items which have goto_table:44 action but table 44 doesn't exist at all.

$ sudo ovs-ofctl -Oopenflow13 dump-flows br-int | grep goto_table:44
cookie=0x8000022, duration=316419.349s, table=20, n_packets=0, n_bytes=0, priority=10,mpls,mpls_label=100026 actions=pop_mpls:0x0800,goto_table:44
cookie=0x8000003, duration=318799.571s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x30d42/0xfffffe,nw_dst=192.168.100.222 actions=write_metadata:0x30d42/0xfffffe,goto_table:44
cookie=0x8000003, duration=316419.354s, table=21, n_packets=99, n_bytes=9570, priority=42,ip,metadata=0x30d5e/0xfffffe,nw_dst=192.168.100.202 actions=write_metadata:0x30d5e/0xfffffe,goto_table:44
cookie=0x90186ba, duration=316419.351s, table=36, n_packets=0, n_bytes=0, priority=5,tun_id=0x186ba actions=goto_table:44
$ sudo ovs-ofctl -Oopenflow13 dump-flows br-int | grep table=44
$

$ curl -u admin:admin http://192.168.0.5:8181/restconf/config/odl-fib:fibEntries/ | python3 -mjson.tool
{
"fibEntries": {
"vrfTables": [
{
"routeDistinguisher": "24ee0046-b656-4632-94bb-3dc44ea26a45",
"vrfEntry": [
{
"destPrefix": "192.168.100.202/32",
"origin": "s",
"encap-type": "mplsgre",
"route-paths": [

],
"parent-vpn-rd": "24ee0046-b656-4632-94bb-3dc44ea26a45"
}
]
},
{
"routeDistinguisher": "4f217a15-f0db-4dd1-b6e1-57e08a7d74c4",
"vrfEntry": [

,

,
{
"destPrefix": "192.168.100.215/32",
"mac": "fa:16:3e:3c:d7:cc",
"origin": "s",
"encap-type": "mplsgre",
"route-paths": [

],
"parent-vpn-rd": "24ee0046-b656-4632-94bb-3dc44ea26a45"
}
]
},
{
"routeDistinguisher": "16d6b03e-9476-4648-8fdd-8dfa43061a55",
"vrfEntry": [
{
"destPrefix": "10.15.1.3/32",
"origin": "l",
"encap-type": "mplsgre",
"route-paths": [

],
"gateway_mac_address": "fa:16:3e:05:0d:55"
},
{
"destPrefix": "10.16.2.8/32",
"origin": "l",
"encap-type": "mplsgre",
"route-paths": [

],
"gateway_mac_address": "fa:16:3e:db:1c:0b"
},
{
"destPrefix": "10.15.1.1/32",
"origin": "l",
"route-paths": [

],
"ip-address": "10.15.1.1/32",
"uuid": "16d6b03e-9476-4648-8fdd-8dfa43061a55",
"mac-address": "fa:16:3e:05:0d:55"
},
{
"destPrefix": "10.15.1.2/32",
"origin": "l",
"encap-type": "mplsgre",
"route-paths": [

],
"gateway_mac_address": "fa:16:3e:05:0d:55"
},
{
"destPrefix": "10.16.2.1/32",
"origin": "l",
"route-paths": [

],
"ip-address": "10.16.2.1/32",
"uuid": "16d6b03e-9476-4648-8fdd-8dfa43061a55",
"mac-address": "fa:16:3e:db:1c:0b"
},
{
"destPrefix": "10.16.2.2/32",
"origin": "l",
"encap-type": "mplsgre",
"route-paths": [

],
"gateway_mac_address": "fa:16:3e:db:1c:0b"
},
{
"destPrefix": "10.15.1.5/32",
"origin": "l",
"encap-type": "mplsgre",
"route-paths": [

],
"gateway_mac_address": "fa:16:3e:05:0d:55"
},
{
"destPrefix": "10.16.2.12/32",
"origin": "l",
"encap-type": "mplsgre",
"route-paths": [

],
"gateway_mac_address": "fa:16:3e:db:1c:0b"
}
]
},
{
"routeDistinguisher": "931e9554-9062-4cc0-858a-6dbabbc6411a",
"vrfEntry": [

]
}
]
}
}
$

We observed this regression in OPNFV Fuel too, after bumping from Fluorine SR0 to SR1, using both Openstack Queens and Rocky.

What's interesting is that we only see it on virtual deployments (where all nodes are virtual machines - we have a dedicated ODL VM, as well as dedicated compute VMs), but not on baremetal (where compute nodes are baremetal, yet ODL runs in HA mode inside 3 VMs). I started a CI job without HA on baremetal to see whether the differentiator is HA-mode or the node type (virtual vs baremetal) ...

Also, restarting the ODL server seems to fix the issue for us.

LE: This is odd. With Rocky, all ODL deploymenets have this issue. With Queens, a simple ODL-L3 scenario works fine, but not with SFC or BGPVPN (I doubt the features themselves are related to the issue, but both install additional packages - tacker or quagga - adding some delay till we actually try to use the FIPs; so maybe this issue is time-sensitive and was not caught by CI without long-term stability testing?).

[1] https://build.opnfv.org/ci/view/fuel/job/functest-fuel-virtual-daily-gambia/624/console

Hi, Chetan

Can you help check this one? In this issue, I used real external network, but 192.168.100.1 can't be learnt, on the contrary, it learned another IP of neutron ext router.

It's vagrant virtual box network issue, this issue disappeared after used change: vbox.customize ['modifyvm', :id, '--nicpromisc3', 'allow-all']

So marked it as "invalid"