Operating System: All
Platform: All

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-snat-conntrack-carbon/122/log.html.gz#s1-s7-t2-k31

Update on NETVIRT-811.

Vivek and I analyzed the CSIT logs, and it looks like the Flow Cleanup phase for External Network Test is not cleaning up all the flows.

From the below link:
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-snat-conntrack-carbon/122/log.html.gz#s1-s7

it looks like NAT related flows are not getting cleaned up.

Secifically regarding NETVIRT-811, a stale NAT conntrack flow is present in the table-36 entry, that is failing ping in its reply path, i.e,

cookie=0x8000006, duration=203.299s, table=36, n_packets=5, n_bytes=331, priority=10,ip,tun_id=0x186a9 actions=set_field:0x30d52->metadata,ct(table=46,zone=5003,nat) IN COMPUTE_1 is the stale flow, which is not allowing the packet to reach cookie=0x90186a9, duration=144.439s, table=36, n_packets=0, n_bytes=0, priority=5,tun_id=0x186a9 actions=group:150025.

@Aswin,
Could you please help us in finding out why the conntrack flow was not cleaned up properly?

Thanks,
Kiran

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-upstream-stateful-snat-conntrack-carbon/82/log.html.gz#s1-s7-t2-k31

I tried multiple times locally(in one node and two node setup). But did see an instance where this flow is not getting cleaned up. Also this failure does not happen last 15 run. Is it safe for us to assume this may have got fixed?

(In reply to Aswin Suryanarayanan from comment #4)
> I tried multiple times locally(in one node and two node setup). But did see
> an instance where this flow is not getting cleaned up. Also this failure
> does not happen last 15 run. Is it safe for us to assume this may have got
> fixed?

yeah, lets close this and I'll re-open if it shows back up. CSIT is
passing pretty reliably now so I think a lot of the sporadic bugs
(like this) have been fixed.