Operating System: All
Platform: All

Attachment flows.txt has been added with description: dump flows

(In reply to balakrishnan k from comment #0)
> Created attachment 1777 [details]
> dump flows
>
> environment details:
> One ODL controller + One control node + 2 compute node.
> ODL version: Carbon
> https://nexus.opendaylight.org/content/repositories/autorelease-1779/org/
> opendaylight/integration/distribution-karaf/0.6.0-Carbon/distribution-karaf-
> 0.6.0-Carbon.tar.gz
>
> openstack Version:Ocata
>
> issue scenario:
> Communication between two VM instance on different tenant network failed
> when using default SG.
> steps to reproduce the issue:
> 1.create network1 (10.0.0.0/24)
> 2.create network2 (20.0.0.0/24)
> 3.create 2VM's using network1 (VM1:10.0.0.3,VM2:10.0.0.4)
> 4.create 2VM's using network2 (VM3:20.0.0.3,VM4:20.0.0.4)
> 5.create router (R1) and attach interface(network1, network2) to the
> router.
> 6.Now open VM1 console try to ping VM3.
>
> step 6 failed, VM1 able to ping network2 dhcp (20.0.0.2) but unable to
> ping VM3 and VM4.
> also VM1 unable to ping 10.0.0.1
>
> Note: if we explicitly add icmp ingress rule in "defaultSG" ping to all
> VM's are working as expected.
>
> icmp ingress flow added in OVS:
> cookie=0x6900000, duration=1231.003s, table=243, n_packets=3, n_bytes=294,
> priority=1006,ct_state=+new+trk,icmp,reg6=0x300/0xfffff00
> actions=ct(commit,zone=5002),resubmit(,220)
>
> default SG flow:
> cookie=0x6900000, duration=1657.956s, table=243, n_packets=0, n_bytes=0,
> priority=1000,ct_state=+new+trk,ip,reg6=0x300/0xfffff00,metadata=0x2/
> 0xfffffe actions=ct(commit,zone=5002),resubmit(,220)
>
> my guess ping to other network doesn't matching "metadata=0x2/0xfffffe" ,
> where as explicitly added icmp rule doesn't contain metadata in the flow
> ping was working.

I think this scenario is already there in our CSIT jobs, and not failing.
BUT, we are not using Ocata yet. That must be a requirement for hitting
this bug. Agree?

Hi Jamo,
I think the problem seems to be CSIT job not using Default SG.
we are creating job with Default SG in sandbox we will come back with result.

(In reply to balakrishnan k from comment #2)
> Hi Jamo,
> I think the problem seems to be CSIT job not using Default SG.
> we are creating job with Default SG in sandbox we will come back with result.

I'm a little confused now. You mean if CSIT uses default SG then we'll start
having failures?

(In reply to Jamo Luhrsen from comment #3)
> (In reply to balakrishnan k from comment #2)
> > Hi Jamo,
> > I think the problem seems to be CSIT job not using Default SG.
> > we are creating job with Default SG in sandbox we will come back with result.
>
> I'm a little confused now. You mean if CSIT uses default SG then we'll start
> having failures?

yes Jamo,

In the below sandbox run, we have modified tests to use default SG.
Only one rule to allow tcp is added to default, the ping between vm's connected in differnt subnets fails with router.

https://logs.opendaylight.org/sandbox/jenkins091/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-carbon/23/archives/log.html.gz

Analysed the pipeline for the failure the problem not only occurs in Default SG.
it also occurs user create SG rule with remote SG option.
below are the observation:

net1 ELAN_ID = 0x157d000000/0xffff000000
net2 ELAN_ID = 0x157e000000/0xffff000000

VM1:10.0.0.8(net1) initiate ping to VM2:20.0.0.3(net2)
VM1(compute1) and VM2(compute2) are created in different compute nodes.

compute1 pipeline flow:

table 0:
cookie=0x8000000, duration=138584.111s, table=0, n_packets=3001, n_bytes=286378, priority=4,in_port=4,vlan_tci=0x0000/0x1fff actions=write_metadata:0x20000000000/0xffffff0000000001,goto_table:17

table 17:
cookie=0x6900000, duration=138584.111s, table=17, n_packets=3001, n_bytes=286378, priority=10,metadata=0x20000000000/0xffffff0000000000 actions=write_metadata:0x8000021772000000/0xfffffffffffffffe,goto_table:211

table :211
cookie=0x6900000, duration=138584.114s, table=211, n_packets=2826, n_bytes=276770, priority=61010,ip,dl_src=fa:16:3e:50:18:9c,nw_src=10.0.0.10 actions=ct(table=212,zone=6002)

table 212:
cookie=0x6900000, duration=138584.113s, table=212, n_packets=2496, n_bytes=244056, priority=0 actions=goto_table:213

table 213
cookie=0x6900000, duration=138584.113s, table=213, n_packets=1, n_bytes=98,priority=3902,ct_state=+new+trk,ip,metadata=0x20000000000/0xfffff0000000000 actions=ct(commit,zone=6002),resubmit(,17)

table 17
cookie=0x8000001, duration=138584.111s, table=17, n_packets=2822, n_bytes=272406, priority=10,metadata=0x8000020000000000/0xffffff0000000000 actions=load:0x19e10->NXM_NX_REG3[0..24],write_metadata:0x9000020000033c20/0xfffffffffffffffe,goto_table:19

table 19:
cookie=0x8000009, duration=138584.113s, table=19, n_packets=2267, n_bytes=222166, priority=20,metadata=0x33c20/0xfffffe,dl_dst=fa:16:3e:17:53:6a actions=goto_table:21

table 21 :
cookie=0x8000003, duration=138584.113s, table=21, n_packets=2262, n_bytes=221676, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=20.0.0.11 actions=set_field:0x19e18->tun_id,set_field:fa:16:3e:68:23:05->eth_dst,load:0x900->NXM_NX_REG6[],resubmit(,220)

table 220:
cookie=0x8000230, duration=138584.113s, table=220, n_packets=2904, n_bytes=275920, priority=6,reg6=0x900 actions=load:0x90000900->NXM_NX_REG6[],write_metadata:0/0xfffffffffe,goto_table:230
table 230:
cookie=0x8000230, duration=138584.112s, table=230, n_packets=3304, n_bytes=308034, priority=0 actions=resubmit(,220)

table 220:
cookie=0x8000007, duration=138584.113s, table=220, n_packets=2904, n_bytes=275920, priority=9,reg6=0x90000900 actions=output:3

compute2 :

table 0:

cookie=0x8000001, duration=84529.272s, table=0, n_packets=325699, n_bytes=23220100, priority=5,in_port=3 actions=write_metadata:0x70000000001/0xfffff0000000001,goto_table:36

table 36:

cookie=0x9019e18, duration=84529.273s, table=36, n_packets=2262, n_bytes=221676, priority=5,tun_id=0x19e18 actions=group:155003

group: group_id=155003,type=all,bucket=actions=set_field:fa:16:3e:bc:8e:10->eth_src,set_field:fa:16:3e:68:23:05->eth_dst,load:0xb00->NXM_NX_REG6[],resubmit(,220)

table 220:

cookie=0x6900000, duration=84529.274s, table=220, n_packets=2357, n_bytes=232120, priority=6,reg6=0xb00 actions=load:0x90000b00->NXM_NX_REG6[],write_metadata:0x1773000000/0xfffffffffe,goto_table:241

table 241:

cookie=0x6900000, duration=84529.273s, table=241, n_packets=2333, n_bytes=228720, priority=61010,ip,dl_dst=fa:16:3e:68:23:05,nw_dst=20.0.0.11 actions=ct(table=242,zone=6003)

table 242:

cookie=0x6900000, duration=84529.273s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1773000000/0xffff000000,nw_src=20.0.0.11 actions=write_metadata:0x7d2/0xfffffe,goto_table:243
cookie=0x6900000, duration=84529.273s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1772000000/0xffff000000,nw_src=10.0.0.10 actions=write_metadata:0x7d2/0xfffffe,goto_table:243
cookie=0x6900000, duration=84529.273s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1772000000/0xffff000000,ipv6_src=fe80::f816:3eff:fe50:189c actions=write_metadata:0x7d2/0xfffffe,goto_table:243
cookie=0x6900000, duration=84529.273s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1773000000/0xffff000000,ipv6_src=fe80::f816:3eff:fe68:2305 actions=write_metadata:0x7d2/0xfffffe,goto_table:243
cookie=0x6900000, duration=84529.273s, table=242, n_packets=2552, n_bytes=253089, priority=0 actions=goto_table:243

table 243:
cookie=0x6900001, duration=84529.273s, table=243, n_packets=2258, n_bytes=221284, priority=50,ct_state=+new+trk,reg6=0xb00/0xfffff00 actions=drop

when the packets goes through pipeline table 36 on compute2 group action is applied.
and reg6 value is loaded based on reg value ELAN_tag net2 is loaded in metadata and the packets forwarded to egressACL tables.
now the packet contains "metadata=ELAN_ID(net1), ip_src/dst=VM2_IP(net2)".

but the match condition for remote SG created on table 242 contains "metadata=ELAN_ID(net1), ip_src/dst=VM2_IP(net1)"
and "metadata=ELAN_ID(net2), ip_src/dst=VM2_IP(net2)"
so, if we use router in with Remote SG the packets getting dropped in table 243.

when the router involved the remote SG match in table 212 and 242 should contain
1.metadata=ELAN_ID(net1), ip_src/dst=VM2_IP(net1)
2.metadata=ELAN_ID(net2), ip_src/dst=VM2_IP(net2)
3.metadata=ELAN_ID(net1), ip_src/dst=VM2_IP(net2)
4.metadata=ELAN_ID(net2), ip_src/dst=VM2_IP(net1)
to make communication work between two different tenant in case of remote SG.

Hi Aswin et al,

I tend to believe that the better way to fix this is to have the VPN-ID written into a free register (regZ) in Table 36 itself before sending to VPN Group (ie., 155003 in the NETVIRT-657 explanation).

The Table 220 will retain that regZ PLUS will continue to write ELAN-Tag of the destination VM into metadata as usual and then additional flow rules will be installed in Table 242 that matches on the VPN-ID in (regZ + nw-src), since a given IP-Address can appear only once in a VPNInstance.

More specifically:

cookie=0x9019e18, duration=84529.273s, table=36, n_packets=2262, n_bytes=221676, priority=5,tun_id=0x19e18 actions= load:<vpn-id>/0x0000ffffff->NXM_NX_REGZ[],group:155003

cookie=0x6900000, duration=84529.274s, table=220, n_packets=2357, n_bytes=232120, priority=6,reg6=0xb00 actions=load:0x90000b00->NXM_NX_REG6[],write_metadata:0x1773000000/0xffff000000,goto_table:241

cookie=0x6900000, duration=84529.273s, table=242, n_packets=0, n_bytes=0, priority=50,ip,regZ=<vpn-id>/0x0000ffffff,nw_src=20.0.0.11 actions=write_metadata:0x7d2/0xfffffe,goto_table:243

But I would like to understand the implications of this proposal from ACL + GENIUS folks.

–
Thanks,

Vivek

From: netvirt-dev-bounces@lists.opendaylight.org netvirt-dev-bounces@lists.opendaylight.org On Behalf Of N Vivekanandan
Sent: Monday, May 22, 2017 8:31 PM
To: Sam Hague <shague@redhat.com>; Aswin Suryanarayanan <asuryana@redhat.com>
Cc: netvirt-dev@lists.opendaylight.org
Subject: Re: [netvirt-dev] Reg Bug:8398

Hi Aswin,

“It seems to be a flaw in the approach of populating the elan ID in the dispatcher. At the egress side(241-243) ACL service tables expects elan id based on the n/w src , but we are populating it based it on the interface elan, that is the destination elan. Vice-verse at ingress side”

I believe the dispatcher always used the metadata to set the destination ELAN Tag of the destination VMs LPort even before this ‘remote indirection of ACL’ feature came in.

On the destination hypervisor, how do we know without ambiguity what is the ELAN-Tag of a given Source IPs when the packet has come in over a tunnel?

It looks to me that a design issue for routed packets in the ‘remote acl indirection feature’ that might have been overlooked 

–
Thanks,

Vivek

From: Sam Hague shague@redhat.com
Sent: Monday, May 22, 2017 7:33 PM
To: Aswin Suryanarayanan <asuryana@redhat.com>; N Vivekanandan <n.vivekanandan@ericsson.com>
Cc: Balakrishnan Karuppasamy <balakrishnan.ka@hcl.com>; Dayavanti Gopal Kamath <dayavanti.gopal.kamath@ericsson.com>; Faseela K <faseela.k@ericsson.com>; netvirt-dev@lists.opendaylight.org; Venkatrangan G - ERS, HCL Tech <venkatrangang@hcl.com>
Subject: Re: Reg Bug:8398

Adding Vivek.
Vivek, Faseela,
any thoughts on Aswin's questions? Seems like a nasty bug we should fix.
Thanks, Sam

On Mon, May 22, 2017 at 9:44 AM, Aswin Suryanarayanan <asuryana@redhat.com> wrote:
really + Daya +Faseela,
Thanks
Aswin

On Mon, May 22, 2017 at 7:13 PM, Aswin Suryanarayanan <asuryana@redhat.com> wrote:
+Daya, Faseela
Hi Bala,
it is a good find. It seems to be a flaw in the approach of populating the elan ID in the dispatcher. At the egress side(241-243) ACL service tables expects elan id based on the n/w src , but we are populating it based it on the interface elan, that is the destination elan. Vice-verse at ingress side With same n/w it worked as the elan id is same. The approach of populating all the combinations does not solve the issue I think as the intend is to allow the traffic from the expected elan id IP combination.
Daya/Faseela,
Do you have any suggestion in retrieving/populating the source elan id at the egress table for traffic from another vm.
Thanks
Aswin

On Mon, May 22, 2017 at 12:11 PM, Balakrishnan Karuppasamy <balakrishnan.ka@hcl.com> wrote:
Hi Aswin,
Thanks for the documents.

We have analyzed further and updated the bug report
https://bugs.opendaylight.org/show_bug.cgi?id=8398

This issue occurs not only in Default SG.
All remote SG cases will hit the same issue.
When the packets goes through pipeline table 36 group action applied for
Router scenario. Here reg6 value is changed based on the dst address.
Then table 220 matching this reg6 value to add the ELAN_ID in metadata.

So, the packets from table 220 to ACL table 242 contains “metadata=ELAN_ID(net1), ip_src/dst=VM_IP(net2)”

but the match condition for remote SG created on table 242 contains "metadata=ELAN_ID(net1), ip_src/dst=VM2_IP(net1)"
and "metadata=ELAN_ID(net2), ip_src/dst=VM_IP(net2)"
so, if we use router in with Remote SG the packets getting dropped in table 243.

when the router involved the remote SG match in table 212 and 242 should contain below combinaion
1.metadata=ELAN_ID(net1), ip_src/dst=VM_IP(net1)
2.metadata=ELAN_ID(net2), ip_src/dst=VM_IP(net2)
3.metadata=ELAN_ID(net1), ip_src/dst=VM_IP(net2)
4.metadata=ELAN_ID(net2), ip_src/dst=VM_IP(net1)
to make communication work between two different tenant in case of remote SG.
Please check the Bugzilla for detailed explanation.

Can we follow the above approach to fix the issue?
Let us know if you have any queries.

Regards,
Bala

From: Aswin Suryanarayanan asuryana@redhat.com
Sent: Friday, May 12, 2017 8:16 PM
To: Balakrishnan Karuppasamy
Cc: Sam Hague; netvirt-dev@lists.opendaylight.org; Venkatrangan G - ERS, HCL Tech
Subject: Re: Reg Bug:8398

On Fri, May 12, 2017 at 7:04 PM, Balakrishnan Karuppasamy <balakrishnan.ka@hcl.com> wrote:
Hi All,
We are working on the Bug[1].
From the initial analysis found matadata mismatch causing the issue.
If we remove default SG and add Custom SG with the rule any protocol.
communication between two tenant network working as expected.

[1]https://bugs.opendaylight.org/show_bug.cgi?id=8398

Is there any specific reason for using metadata match in ingress/egress table only for Default SG?
If yes , Router scenario is handled using Default SG?

The metadata match is used for remote SG. You can refer [1] for details. This was an optimization added in carbon. So this could be potentially a bug in the remote SG implementation.

[1]https://git.opendaylight.org/gerrit/#/c/51462/

We want to understand the metadata computation of new netvirt pipeline .
Every table in the pipeline changing the metadata for specific reason.
Is there any document available ?

In new netvirt we have the concept of dispatcher table and based on the metadata it is send to each services from dispatcher table . Once the service process the packet it is submitted back with new metadata and based on the metadata the packet will be send to a new service until it exits the pipeline through the egress dispatcher. The high level pipeline diagram can be seen in [2]. The bits in the metadata used for remote SG optimization is different from the one used by dispatcher.

[2]https://docs.google.com/presentation/d/15h4ZjPxblI5Pz9VWIYnzfyRcQrXYxA1uUoqJsgA53KM/edit#slide=id.p

Sample flow :
Default SG(Any):
cookie=0x6900000, duration=6817.703s, table=243, n_packets=0, n_bytes=0, priority=3900,ct_state=+new+trk,ip,reg6=0xb00/0xfffff00,metadata=0x7d0/0xfffffe actions=ct(commit,zone=6003),resubmit(,220)

Custom SG(Any):
cookie=0x6900000, duration=475.577s, table=243, n_packets=1, n_bytes=98, priority=3904,ct_state=+new+trk,ip,reg6=0xb00/0xfffff00 actions=ct(commit,zone=6003),resubmit(,220)

Regards,
Bala

We can workaround this bug by using remote ip prefix rules. The default SG allows traffic between VM's that are in different network but are in default SG. But due this bug it stops working. The below rules shall be added to default SG to allow traffic between vm in default SG. The 0.0.0.0/0 ip prefix shall be replaced with n/w address and mask to allow the traffic between vm in that n/w.

neutron security-group-rule-create --direction ingress --protocol tcp --remote-ip-prefix 0.0.0.0/0 default
neutron security-group-rule-create --direction ingress --protocol icmp --remote-ip-prefix 0.0.0.0/0 default
neutron security-group-rule-create --direction ingress --protocol udp --remote-ip-prefix 0.0.0.0/0 default

The same workaround shall be used in Custom SG ,if traffic needs to be allowed across network. That is remote ip-prefix needs to be used instead of remote-SG.

Logs when tested with patch 10 of
https://git.opendaylight.org/gerrit/#/c/57705

Attachment karaf.tar.gz has been added with description: Logs

Log of tests with PS 12:

Hi KiranN,

I tried another run after the fix and could see the following issues:

1. Table 220 is not filling the vpnId in metadata, while Table 17 is writing metadata correctly.
The service-bindings for 220 show vpnId correctly, but it is not enforced in flows.
2. I donot see the legacy elan-Tag matching rules in Table 212 and Table 242, are we overwriting them with vpnId too?
3. Table 212 and Table 242 both are matching correctly on vpnId as metadata in the flow rules.
4. I tried deletion of the VM itself, and after that I could see ELAN flow coming up , while we should have no flows lingering in Table 17.
5. Next I tried to create a new VM on the router, and this time around Table 17 alone was OK. Table 220, 212 and 242 all had only ELAN-Tag matches.
6. I did not try the delete router lifecycle yet (with VM inside it), as we have to persevere to fix above problem.

Can you please check out the above points 1 and 2?

The karaf logs are here:

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13 | grep table=0
cookie=0x8000000, duration=725.662s, table=0, n_packets=8, n_bytes=648, priority=4,in_port=1,vlan_tci=0x0000/0x1fff actions=write_metadata:0x10000000000/0xffffff0000000001,goto_table:17
cookie=0x8000000, duration=655.359s, table=0, n_packets=9, n_bytes=1455, priority=4,in_port=2,vlan_tci=0x0000/0x1fff actions=write_metadata:0x20000000000/0xffffff0000000001,goto_table:17
stack@u14-bgpvpn-Newton:~/devstack$

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13 | grep table=17
cookie=0x8040000, duration=128.900s, table=17, n_packets=6, n_bytes=946, priority=10,metadata=0x9000020000000000/0xffffff0000000000 actions=load:0x2->NXM_NX_REG1[0..19],load:0x1389->NXM_NX_REG7[0..15],write_metadata:0xa000021389000000/0xfffffffffffffffe,goto_table:48
cookie=0x8000001, duration=6.875s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x10000000000/0xffffff0000000000 actions=load:0x186a0->NXM_NX_REG3[0..24],write_metadata:0x9000010000030d40/0xfffffffffffffffe,goto_table:19
cookie=0x8040000, duration=6.873s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x9000010000000000/0xffffff0000000000 actions=load:0x1->NXM_NX_REG1[0..19],load:0x1389->NXM_NX_REG7[0..15],write_metadata:0xa000011389000000/0xfffffffffffffffe,goto_table:48
cookie=0x6900000, duration=6.814s, table=17, n_packets=8, n_bytes=1126, priority=10,metadata=0x20000000000/0xffffff0000000000 actions=write_metadata:0x8000020000030d40/0xfffffffffffffffe,goto_table:211
cookie=0x8000001, duration=6.811s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x8000020000000000/0xffffff0000000000 actions=load:0x186a0->NXM_NX_REG3[0..24],write_metadata:0x9000020000030d40/0xfffffffffffffffe,goto_table:19
cookie=0x8000000, duration=316.155s, table=17, n_packets=0, n_bytes=0, priority=0,metadata=0x8000000000000000/0xf000000000000000 actions=write_metadata:0x9000000000000000/0xf000000000000000,goto_table:80

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13 | grep table=220
cookie=0x8000007, duration=212.272s, table=220, n_packets=0, n_bytes=0, priority=9,reg6=0x100 actions=output:1
cookie=0x6900000, duration=141.995s, table=220, n_packets=0, n_bytes=0, priority=6,reg6=0x200 actions=load:0x90000200->NXM_NX_REG6[],write_metadata:0x1389000000/0xfffffffffe,goto_table:241
cookie=0x8000007, duration=141.995s, table=220, n_packets=0, n_bytes=0, priority=9,reg6=0x90000200 actions=output:2
stack@u14-bgpvpn-Newton:~/devstack$
stack@u14-bgpvpn-Newton:~/devstack$

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13 | grep table=212
cookie=0x6900000, duration=184.067s, table=211, n_packets=1, n_bytes=90, priority=61010,ipv6,dl_src=fa:16:3e:c8:aa:0b,ipv6_src=fe80::f816:3eff:fec8:aa0b actions=ct(table=212,zone=5001)
cookie=0x6900000, duration=184.061s, table=211, n_packets=0, n_bytes=0, priority=61010,ip,dl_src=fa:16:3e:c8:aa:0b,nw_src=10.1.1.3 actions=ct(table=212,zone=5001)
cookie=0x6900000, duration=62.186s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x30d40/0xfffffe,ipv6_dst=fe80::f816:3eff:fec8:aa0b actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=62.161s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x30d40/0xfffffe,nw_dst=10.1.1.3 actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=370.602s, table=212, n_packets=1, n_bytes=90, priority=0 actions=goto_table:213
stack@u14-bgpvpn-Newton:~/devstack$
stack@u14-bgpvpn-Newton:~/devstack$
stack@u14-bgpvpn-Newton:~/devstack$
stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13 | grep table=242
cookie=0x6900000, duration=225.952s, table=241, n_packets=0, n_bytes=0, priority=61010,ipv6,dl_dst=fa:16:3e:c8:aa:0b,ipv6_dst=fe80::f816:3eff:fec8:aa0b actions=ct(table=242,zone=5001)
cookie=0x6900000, duration=225.945s, table=241, n_packets=0, n_bytes=0, priority=61010,ip,dl_dst=fa:16:3e:c8:aa:0b,nw_dst=10.1.1.3 actions=ct(table=242,zone=5001)
cookie=0x6900000, duration=104.024s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x30d40/0xfffffe,ipv6_src=fe80::f816:3eff:fec8:aa0b actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=103.995s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x30d40/0xfffffe,nw_src=10.1.1.3 actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=412.401s, table=242, n_packets=0, n_bytes=0, priority=0 actions=goto_table:243

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl show br-int -OOpenflow13
OFPT_FEATURES_REPLY (OF1.3) (xid=0x2): dpid:0000869c6e77bb4f
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS QUEUE_STATS
OFPST_PORT_DESC reply (OF1.3) (xid=0x3):
1(tap9e952013-8c): addr:00:00:00:00:0e:00
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
2(tap0899ac09-18): addr:fe:16:3e:c8:aa:0b
config: 0
state: 0
current: 10MB-FD COPPER
speed: 10 Mbps now, 0 Mbps max
LOCAL(br-int): addr:86:9c:6e:77:bb:4f
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (OF1.3) (xid=0x5): frags=normal miss_send_len=0
stack@u14-bgpvpn-Newton:~/devstack$
stack@u14-bgpvpn-Newton:~/devstack$ neutron port-list
------------------------------------------------------------------------------------------------------------------------------------------+

------------------------------------------------------------------------------------------------------------------------------------------+

------------------------------------------------------------------------------------------------------------------------------------------+
stack@u14-bgpvpn-Newton:~/devstack$

SService Bindings:

{
"service-bindings": {
"services-info": [
{
"interface-name": "9e952013-8c69-4ad1-b17f-f61f96c7cf75",
"service-mode": "interface-service-bindings:service-mode-ingress",
"bound-services": [
{
"service-priority": 9,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 9,
"flow-cookie": 134479872,
"instruction": [
{
"order": 1,
"write-metadata":

},
{
"order": 2,
"apply-actions": {
"action": [
{
"order": 1,
"nx-reg-load": {
"dst":

,
"value": 5001
}
},
{
"order": 0,
"nx-reg-load": {
"dst":

,
"value": 1
}
}
]
}
},
{
"order": 3,
"go-to-table":

}
],
"service-name": "elan.7ed08cd2-fd50-409f-91cc-a015a0922728.9e952013-8c69-4ad1-b17f-f61f96c7cf75"
},
{
"service-priority": 8,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 10,
"flow-cookie": 134217729,
"instruction": [
{
"order": 1,
"apply-actions": {
"action": [
{
"order": 0,
"nx-reg-load": {
"dst":

,
"value": 100000
}
}
]
}
},
{
"order": 2,
"write-metadata":

},
{
"order": 3,
"go-to-table":

}
],
"service-name": "vpn.e989a71e-29be-4bd1-9936-20c45edcde3a.9e952013-8c69-4ad1-b17f-f61f96c7cf75"
}
]
},
{
"interface-name": "0899ac09-185b-40e9-a135-49c8c6f21f1e",
"service-mode": "interface-service-bindings:service-mode-ingress",
"bound-services": [
{
"service-priority": 2,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 11,
"flow-cookie": 110100480,
"instruction": [
{
"order": 1,
"write-metadata":

},
{
"order": 2,
"go-to-table":

}
],
"service-name": "acl.egressacl.0899ac09-185b-40e9-a135-49c8c6f21f1e"
},
{
"service-priority": 9,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 9,
"flow-cookie": 134479872,
"instruction": [
{
"order": 1,
"write-metadata":

},
{
"order": 2,
"apply-actions": {
"action": [
{
"order": 1,
"nx-reg-load": {
"dst":

,
"value": 5001
}
},
{
"order": 0,
"nx-reg-load": {
"dst":

,
"value": 2
}
}
]
}
},
{
"order": 3,
"go-to-table":

}
],
"service-name": "elan.7ed08cd2-fd50-409f-91cc-a015a0922728.0899ac09-185b-40e9-a135-49c8c6f21f1e"
},
{
"service-priority": 8,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 10,
"flow-cookie": 134217729,
"instruction": [
{
"order": 1,
"apply-actions": {
"action": [
{
"order": 0,
"nx-reg-load": {
"dst":

,
"value": 100000
}
}
]
}
},
{
"order": 2,
"write-metadata":

},
{
"order": 3,
"go-to-table":

}
],
"service-name": "vpn.e989a71e-29be-4bd1-9936-20c45edcde3a.0899ac09-185b-40e9-a135-49c8c6f21f1e"
}
]
},
{
"interface-name": "0899ac09-185b-40e9-a135-49c8c6f21f1e",
"service-mode": "interface-service-bindings:service-mode-egress",
"bound-services": [
{
"service-priority": 6,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 1,
"flow-cookie": 110100480,
"instruction": [
{
"order": 1,
"write-metadata":

},
{
"order": 2,
"go-to-table":

}
],
"service-name": "acl.ingressacl.0899ac09-185b-40e9-a135-49c8c6f21f1e"
},
{
"service-priority": 9,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 9,
"flow-cookie": 134217735,
"instruction": [
{
"order": 0,
"apply-actions": {
"action": [
{
"order": 0,
"output-action":

}
]
}
}
],
"service-name": "default.0899ac09-185b-40e9-a135-49c8c6f21f1e"
}
]
},
{
"interface-name": "9e952013-8c69-4ad1-b17f-f61f96c7cf75",
"service-mode": "interface-service-bindings:service-mode-egress",
"bound-services": [
{
"service-priority": 9,
"service-type": "interface-service-bindings:service-type-flow-based",
"flow-priority": 9,
"flow-cookie": 134217735,
"instruction": [
{
"order": 0,
"apply-actions": {
"action": [
{
"order": 0,
"output-action":

}
]
}
}
],
"service-name": "default.9e952013-8c69-4ad1-b17f-f61f96c7cf75"
}
]
}
]
}
}
Full flow dumps on router creation with one VM inside:

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x8000000, duration=1027.744s, table=0, n_packets=8, n_bytes=648, priority=4,in_port=1,vlan_tci=0x0000/0x1fff actions=write_metadata:0x10000000000/0xffffff0000000001,goto_table:17
cookie=0x8000000, duration=957.441s, table=0, n_packets=9, n_bytes=1455, priority=4,in_port=2,vlan_tci=0x0000/0x1fff actions=write_metadata:0x20000000000/0xffffff0000000001,goto_table:17
cookie=0x8040000, duration=957.299s, table=17, n_packets=6, n_bytes=946, priority=10,metadata=0x9000020000000000/0xffffff0000000000 actions=load:0x2->NXM_NX_REG1[0..19],load:0x1389->NXM_NX_REG7[0..15],write_metadata:0xa000021389000000/0xfffffffffffffffe,goto_table:48
cookie=0x8000001, duration=835.274s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x10000000000/0xffffff0000000000 actions=load:0x186a0->NXM_NX_REG3[0..24],write_metadata:0x9000010000030d40/0xfffffffffffffffe,goto_table:19
cookie=0x8040000, duration=835.272s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x9000010000000000/0xffffff0000000000 actions=load:0x1->NXM_NX_REG1[0..19],load:0x1389->NXM_NX_REG7[0..15],write_metadata:0xa000011389000000/0xfffffffffffffffe,goto_table:48
cookie=0x6900000, duration=835.213s, table=17, n_packets=9, n_bytes=1455, priority=10,metadata=0x20000000000/0xffffff0000000000 actions=write_metadata:0x8000020000030d40/0xfffffffffffffffe,goto_table:211
cookie=0x8000001, duration=835.210s, table=17, n_packets=1, n_bytes=329, priority=10,metadata=0x8000020000000000/0xffffff0000000000 actions=load:0x186a0->NXM_NX_REG3[0..24],write_metadata:0x9000020000030d40/0xfffffffffffffffe,goto_table:19
cookie=0x8000000, duration=1144.554s, table=17, n_packets=0, n_bytes=0, priority=0,metadata=0x8000000000000000/0xf000000000000000 actions=write_metadata:0x9000000000000000/0xf000000000000000,goto_table:80
cookie=0x6800000, duration=1143.658s, table=18, n_packets=0, n_bytes=0, priority=0 actions=goto_table:38
cookie=0x1080000, duration=1142.237s, table=19, n_packets=0, n_bytes=0, priority=100,arp,arp_op=1 actions=group:5000
cookie=0x1080000, duration=1142.237s, table=19, n_packets=0, n_bytes=0, priority=100,arp,arp_op=2 actions=CONTROLLER:65535
cookie=0x8000009, duration=835.253s, table=19, n_packets=0, n_bytes=0, priority=20,metadata=0x30d40/0xfffffe,dl_dst=fa:16:3e:3e:5d:27 actions=goto_table:21
cookie=0x1080000, duration=1142.238s, table=19, n_packets=1, n_bytes=329, priority=0 actions=drop
cookie=0x1030000, duration=1142.238s, table=20, n_packets=0, n_bytes=0, priority=0 actions=goto_table:80
cookie=0x8000003, duration=835.253s, table=21, n_packets=0, n_bytes=0, priority=42,icmp,metadata=0x30d40/0xfffffe,nw_dst=10.1.1.1,icmp_type=8,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:3e:5d:27>eth_src,move:NXM_OF_IP_SRC[]>NXM_OF_IP_DST[],set_field:10.1.1.1>ip_src,set_field:0->icmp_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=834.672s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x30d40/0xfffffe,nw_dst=10.1.1.2 actions=group:150001
cookie=0x8000003, duration=834.672s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x30d40/0xfffffe,nw_dst=10.1.1.3 actions=group:150000
cookie=0x8000004, duration=1142.238s, table=22, n_packets=0, n_bytes=0, priority=0 actions=CONTROLLER:65535
cookie=0x1080000, duration=1142.238s, table=23, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x9001389, duration=1026.576s, table=36, n_packets=0, n_bytes=0, priority=5,tun_id=0x5f actions=write_metadata:0x1389000000/0xfffffffff000000,goto_table:51
cookie=0x9051389, duration=1027.699s, table=38, n_packets=0, n_bytes=0, priority=5,tun_id=0x5f actions=write_metadata:0x1389000000/0xfffffffff000000,goto_table:51
cookie=0x4000000, duration=1143.659s, table=45, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x8500000, duration=1143.659s, table=48, n_packets=11, n_bytes=1336, priority=0 actions=drop
cookie=0x8051389, duration=1026.576s, table=50, n_packets=0, n_bytes=0, priority=20,metadata=0x11389000000/0xfffffffff000000,dl_src=fa:16:3e:76:04:c7 actions=goto_table:51
cookie=0x8051389, duration=957.319s, table=50, n_packets=0, n_bytes=0, priority=20,metadata=0x21389000000/0xfffffffff000000,dl_src=fa:16:3e:c8:aa:0b actions=goto_table:51
cookie=0x8050000, duration=1143.659s, table=50, n_packets=0, n_bytes=0, priority=0 actions=CONTROLLER:65535,goto_table:51
cookie=0x8050001, duration=1142.238s, table=50, n_packets=0, n_bytes=0, priority=10 actions=goto_table:51
cookie=0x8031389, duration=1026.576s, table=51, n_packets=0, n_bytes=0, priority=20,metadata=0x1389000000/0xffff000000,dl_dst=fa:16:3e:76:04:c7 actions=load:0x100->NXM_NX_REG6[],resubmit(,220)
cookie=0x8031389, duration=957.319s, table=51, n_packets=0, n_bytes=0, priority=20,metadata=0x1389000000/0xffff000000,dl_dst=fa:16:3e:c8:aa:0b actions=load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x8030000, duration=1142.238s, table=51, n_packets=0, n_bytes=0, priority=15,dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop
cookie=0x8030000, duration=1142.238s, table=51, n_packets=0, n_bytes=0, priority=0 actions=goto_table:52
cookie=0x8701389, duration=1026.576s, table=52, n_packets=0, n_bytes=0, priority=5,metadata=0x1389000000/0xffff000001 actions=write_actions(group:210002)
cookie=0x8701389, duration=1026.576s, table=52, n_packets=0, n_bytes=0, priority=5,metadata=0x1389000001/0xffff000001 actions=write_actions(group:210001)
cookie=0x8800001, duration=1026.576s, table=55, n_packets=0, n_bytes=0, priority=10,tun_id=0x1,metadata=0x10000000000/0xfffff0000000000 actions=drop
cookie=0x8800002, duration=957.319s, table=55, n_packets=0, n_bytes=0, priority=10,tun_id=0x2,metadata=0x20000000000/0xfffff0000000000 actions=drop
cookie=0x8800001, duration=1026.576s, table=55, n_packets=0, n_bytes=0, priority=9,tun_id=0x1 actions=load:0x100->NXM_NX_REG6[],resubmit(,220)
cookie=0x8800002, duration=957.319s, table=55, n_packets=0, n_bytes=0, priority=9,tun_id=0x2 actions=load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x6800000, duration=1143.659s, table=60, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x1030000, duration=1142.238s, table=80, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x12230104, duration=835.253s, table=81, n_packets=0, n_bytes=0, priority=100,arp,metadata=0x9000020000030d40/0xffffff0000fffffe,arp_tpa=10.1.1.1,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:3e:5d:27>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e3e5d27->NXM_NX_ARP_SHA[],load:0xa010101->NXM_OF_ARP_SPA[],load:0->NXM_OF_IN_PORT[],load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x12230103, duration=835.253s, table=81, n_packets=0, n_bytes=0, priority=100,arp,metadata=0x9000010000030d40/0xffffff0000fffffe,arp_tpa=10.1.1.1,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:3e:5d:27>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e3e5d27->NXM_NX_ARP_SHA[],load:0xa010101->NXM_OF_ARP_SPA[],load:0->NXM_OF_IN_PORT[],load:0x100->NXM_NX_REG6[],resubmit(,220)
cookie=0x8220000, duration=1142.238s, table=81, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x4000001, duration=1143.659s, table=90, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x6900000, duration=957.165s, table=211, n_packets=3, n_bytes=987, priority=63010,udp,metadata=0x20000000000/0xfffff0000000000,tp_src=68,tp_dst=67 actions=resubmit(,17)
cookie=0x6900000, duration=957.160s, table=211, n_packets=0, n_bytes=0, priority=63010,udp6,metadata=0x20000000000/0xfffff0000000000,tp_src=546,tp_dst=547 actions=resubmit(,17)
cookie=0x6900000, duration=957.155s, table=211, n_packets=0, n_bytes=0, priority=63010,udp,metadata=0x20000000000/0xfffff0000000000,tp_src=67,tp_dst=68 actions=drop
cookie=0x6900000, duration=957.150s, table=211, n_packets=0, n_bytes=0, priority=63010,udp6,metadata=0x20000000000/0xfffff0000000000,tp_src=547,tp_dst=546 actions=drop
cookie=0x6900000, duration=957.146s, table=211, n_packets=0, n_bytes=0, priority=63020,icmp6,metadata=0x20000000000/0xfffff0000000000,icmp_type=134,icmp_code=0 actions=drop
cookie=0x6900000, duration=957.143s, table=211, n_packets=3, n_bytes=210, priority=63010,icmp6,metadata=0x20000000000/0xfffff0000000000,icmp_type=133,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=957.140s, table=211, n_packets=1, n_bytes=78, priority=63010,icmp6,metadata=0x20000000000/0xfffff0000000000,icmp_type=135,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=957.136s, table=211, n_packets=0, n_bytes=0, priority=63010,icmp6,metadata=0x20000000000/0xfffff0000000000,icmp_type=136,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=957.127s, table=211, n_packets=0, n_bytes=0, priority=63010,arp,metadata=0x20000000000/0xfffff0000000000,arp_sha=fa:16:3e:c8:aa:0b actions=resubmit(,17)
cookie=0x6900000, duration=957.124s, table=211, n_packets=1, n_bytes=90, priority=61010,ipv6,dl_src=fa:16:3e:c8:aa:0b,ipv6_src=fe80::f816:3eff:fec8:aa0b actions=ct(table=212,zone=5001)
cookie=0x6900000, duration=957.118s, table=211, n_packets=0, n_bytes=0, priority=61010,ip,dl_src=fa:16:3e:c8:aa:0b,nw_src=10.1.1.3 actions=ct(table=212,zone=5001)
cookie=0x6900000, duration=1143.659s, table=211, n_packets=1, n_bytes=90, priority=0 actions=drop
cookie=0x6900000, duration=835.243s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x30d40/0xfffffe,ipv6_dst=fe80::f816:3eff:fec8:aa0b actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=835.218s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x30d40/0xfffffe,nw_dst=10.1.1.3 actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=1143.659s, table=212, n_packets=1, n_bytes=90, priority=0 actions=goto_table:213
cookie=0x6900000, duration=1143.659s, table=213, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x6900000, duration=1142.238s, table=213, n_packets=1, n_bytes=90, priority=62020 actions=drop
cookie=0x6900001, duration=957.108s, table=213, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,metadata=0x20000000000/0xfffff0000000000 actions=drop
cookie=0x6900000, duration=957.104s, table=213, n_packets=0, n_bytes=0, priority=1000,ct_state=+new+trk,ip,metadata=0x20000000000/0xfffff0000000000 actions=ct(commit,zone=5001),resubmit(,17)
cookie=0x6900000, duration=957.099s, table=213, n_packets=0, n_bytes=0, priority=1001,ct_state=+new+trk,ipv6,metadata=0x20000000000/0xfffff0000000000 actions=ct(commit,zone=5001),resubmit(,17)
cookie=0x6900001, duration=957.113s, table=213, n_packets=0, n_bytes=0, priority=50,ct_state=+new+trk,metadata=0x20000000000/0xfffff0000000000 actions=drop
cookie=0x8000007, duration=1027.630s, table=220, n_packets=0, n_bytes=0, priority=9,reg6=0x100 actions=output:1
cookie=0x6900000, duration=957.353s, table=220, n_packets=0, n_bytes=0, priority=6,reg6=0x200 actions=load:0x90000200->NXM_NX_REG6[],write_metadata:0x1389000000/0xfffffffffe,goto_table:241
cookie=0x8000007, duration=957.353s, table=220, n_packets=0, n_bytes=0, priority=9,reg6=0x90000200 actions=output:2
cookie=0x8000230, duration=1143.659s, table=230, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x8000231, duration=1143.659s, table=231, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x6900000, duration=957.330s, table=241, n_packets=0, n_bytes=0, priority=63010,udp,reg6=0x200/0xfffff00,tp_src=67,tp_dst=68 actions=resubmit(,220)
cookie=0x6900000, duration=957.319s, table=241, n_packets=0, n_bytes=0, priority=63010,udp6,reg6=0x200/0xfffff00,tp_src=547,tp_dst=546 actions=resubmit(,220)
cookie=0x6900000, duration=957.300s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x200/0xfffff00,icmp_type=130,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=957.300s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x200/0xfffff00,icmp_type=135,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=957.289s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x200/0xfffff00,icmp_type=136,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=957.289s, table=241, n_packets=0, n_bytes=0, priority=63010,arp,reg6=0x200/0xfffff00 actions=resubmit(,220)
cookie=0x6900000, duration=957.210s, table=241, n_packets=0, n_bytes=0, priority=61010,ipv6,dl_dst=fa:16:3e:c8:aa:0b,ipv6_dst=fe80::f816:3eff:fec8:aa0b actions=ct(table=242,zone=5001)
cookie=0x6900000, duration=957.203s, table=241, n_packets=0, n_bytes=0, priority=61010,ip,dl_dst=fa:16:3e:c8:aa:0b,nw_dst=10.1.1.3 actions=ct(table=242,zone=5001)
cookie=0x6900000, duration=1143.659s, table=241, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x6900000, duration=835.282s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x30d40/0xfffffe,ipv6_src=fe80::f816:3eff:fec8:aa0b actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=835.253s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x30d40/0xfffffe,nw_src=10.1.1.3 actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=1143.659s, table=242, n_packets=0, n_bytes=0, priority=0 actions=goto_table:243
cookie=0x6900000, duration=1143.659s, table=243, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x6900000, duration=1142.238s, table=243, n_packets=0, n_bytes=0, priority=62020 actions=drop
cookie=0x6900001, duration=957.182s, table=243, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,reg6=0x200/0xfffff00 actions=drop
cookie=0x6900001, duration=957.183s, table=243, n_packets=0, n_bytes=0, priority=50,ct_state=+new+trk,reg6=0x200/0xfffff00 actions=drop

When the VPN (i.e., router) is retained and VM alone is deleted:

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13 | grep table=17
cookie=0x8000001, duration=1185.230s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x10000000000/0xffffff0000000000 actions=load:0x186a0->NXM_NX_REG3[0..24],write_metadata:0x9000010000030d40/0xfffffffffffffffe,goto_table:19
cookie=0x8040000, duration=1185.228s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x9000010000000000/0xffffff0000000000 actions=load:0x1->NXM_NX_REG1[0..19],load:0x1389->NXM_NX_REG7[0..15],write_metadata:0xa000011389000000/0xfffffffffffffffe,goto_table:48
cookie=0x6900000, duration=2.819s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x20000000000/0xffffff0000000000 actions=write_metadata:0x8000021389000000/0xfffffffffffffffe,goto_table:211
cookie=0x8000001, duration=2.819s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x8000020000000000/0xffffff0000000000 actions=load:0x186a0->NXM_NX_REG3[0..24],write_metadata:0x9000020000030d40/0xfffffffffffffffe,goto_table:19
cookie=0x8000000, duration=1494.510s, table=17, n_packets=0, n_bytes=0, priority=0,metadata=0x8000000000000000/0xf000000000000000 actions=write_metadata:0x9000000000000000/0xf000000000000000,goto_table:80
stack@u14-bgpvpn-Newton:~/devstack$

When on the VPN(i.e, router), a brand new VM is created (same in_port=2 and LportTag 2000 was assigned by Ifmgr) new-port-id: 47d76985-84c6-42b7-891b-ef0e4556d11d

stack@u14-bgpvpn-Newton:~/devstack$ sudo ovs-ofctl dump-flows br-int -OOpenflow13 | grep table=17
cookie=0x8000001, duration=1533.815s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x10000000000/0xffffff0000000000 actions=load:0x186a0->NXM_NX_REG3[0..24],write_metadata:0x9000010000030d40/0xfffffffffffffffe,goto_table:19
cookie=0x8040000, duration=1533.813s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x90000...

I tested the patches looks like still facing the same issue.
unable to communicate between two VM's using remote SG.
created TWO VM's VM1(10.0.0.9) one compute host1 VM2(20.0.0.11) on compute host2 attached router interface between two tenant network.

dump flows:
compute host1:
[root@ocata-compute1 ~]# sudo ovs-ofctl -O OpenFlow13 dump-flows br-int
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x8000001, duration=920.124s, table=0, n_packets=3485, n_bytes=259235, priority=5,in_port=2 actions=write_metadata:0x90000000001/0xfffff0000000001,goto_table:36
cookie=0x8000001, duration=920.124s, table=0, n_packets=0, n_bytes=0, priority=5,in_port=3 actions=write_metadata:0xa0000000001/0xfffff0000000001,goto_table:36
cookie=0x8000000, duration=682.935s, table=0, n_packets=117, n_bytes=11536, priority=4,in_port=4,vlan_tci=0x0000/0x1fff actions=write_metadata:0x30000000000/0xffffff0000000001,goto_table:17
cookie=0x8000001, duration=920.124s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x60000000000/0xffffff0000000000 actions=load:0x19e10->NXM_NX_REG3[0..24],write_metadata:0x9000060000033c20/0xfffffffffffffffe,goto_table:19
cookie=0x6900000, duration=679.540s, table=17, n_packets=117, n_bytes=11536, priority=10,metadata=0x30000000000/0xffffff0000000000 actions=write_metadata:0x8000031770000000/0xfffffffffffffffe,goto_table:211
cookie=0x8040000, duration=679.517s, table=17, n_packets=110, n_bytes=10858, priority=10,metadata=0x9000030000000000/0xffffff0000000000 actions=load:0x3->NXM_NX_REG1[0..19],load:0x1770->NXM_NX_REG7[0..15],write_metadata:0xa000031770000000/0xfffffffffffffffe,goto_table:48
cookie=0x8000001, duration=679.517s, table=17, n_packets=116, n_bytes=11446, priority=10,metadata=0x8000030000000000/0xffffff0000000000 actions=load:0x19e10->NXM_NX_REG3[0..24],write_metadata:0x9000030000033c20/0xfffffffffffffffe,goto_table:19
cookie=0x8000000, duration=920.124s, table=17, n_packets=0, n_bytes=0, priority=0,metadata=0x8000000000000000/0xf000000000000000 actions=write_metadata:0x9000000000000000/0xf000000000000000,goto_table:80
cookie=0x6800000, duration=920.125s, table=18, n_packets=0, n_bytes=0, priority=0 actions=goto_table:38
cookie=0x1080000, duration=920.125s, table=19, n_packets=0, n_bytes=0, priority=100,arp,arp_op=2 actions=CONTROLLER:65535,resubmit(,17)
cookie=0x1080000, duration=920.125s, table=19, n_packets=5, n_bytes=210, priority=100,arp,arp_op=1 actions=group:5000
cookie=0x8000009, duration=682.200s, table=19, n_packets=6, n_bytes=588, priority=20,metadata=0x33c20/0xfffffe,dl_dst=fa:16:3e:41:61:2f actions=goto_table:21
cookie=0x8000009, duration=681.210s, table=19, n_packets=0, n_bytes=0, priority=20,metadata=0x33c20/0xfffffe,dl_dst=fa:16:3e:53:6e:14 actions=goto_table:21
cookie=0x1080000, duration=920.125s, table=19, n_packets=215, n_bytes=21458, priority=0 actions=resubmit(,17)
cookie=0x1030000, duration=920.125s, table=20, n_packets=0, n_bytes=0, priority=0 actions=goto_table:80
cookie=0x8000003, duration=682.200s, table=21, n_packets=6, n_bytes=588, priority=42,icmp,metadata=0x33c20/0xfffffe,nw_dst=20.0.0.1,icmp_type=8,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:41:61:2f>eth_src,move:NXM_OF_IP_SRC[]>NXM_OF_IP_DST[],set_field:20.0.0.1>ip_src,set_field:0->icmp_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=681.216s, table=21, n_packets=0, n_bytes=0, priority=42,icmp,metadata=0x33c20/0xfffffe,nw_dst=10.0.0.1,icmp_type=8,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:53:6e:14>eth_src,move:NXM_OF_IP_SRC[]>NXM_OF_IP_DST[],set_field:10.0.0.1>ip_src,set_field:0->icmp_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=681.763s, table=21, n_packets=6, n_bytes=588, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=20.0.0.11 actions=group:155002
cookie=0x8000003, duration=680.740s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=20.0.0.2 actions=set_field:0x2d->tun_id,set_field:fa:16:3e:ff:2b:c3->eth_dst,load:0x900->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=680.740s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=10.0.0.2 actions=set_field:0x4c->tun_id,set_field:fa:16:3e:0e:c0:4b->eth_dst,load:0x900->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=680.740s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=10.0.0.9 actions=set_field:0x4c->tun_id,set_field:fa:16:3e:16:21:cc->eth_dst,load:0xa00->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000004, duration=920.125s, table=22, n_packets=0, n_bytes=0, priority=0 actions=CONTROLLER:65535
cookie=0x1080000, duration=920.125s, table=23, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x9001770, duration=679.649s, table=36, n_packets=150, n_bytes=15766, priority=5,tun_id=0x2d actions=write_metadata:0x1770000000/0xfffffffff000000,goto_table:51
cookie=0x9051770, duration=682.847s, table=38, n_packets=0, n_bytes=0, priority=5,tun_id=0x2d actions=write_metadata:0x1770000000/0xfffffffff000000,goto_table:51
cookie=0x4000000, duration=920.125s, table=45, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x8500000, duration=920.125s, table=48, n_packets=220, n_bytes=21668, priority=0 actions=resubmit(,49),resubmit(,50)
cookie=0x8051770, duration=679.649s, table=50, n_packets=110, n_bytes=10858, priority=20,metadata=0x31770000000/0xfffffffff000000,dl_src=fa:16:3e:13:d9:3c actions=goto_table:51
cookie=0x8050001, duration=920.124s, table=50, n_packets=0, n_bytes=0, priority=10,reg4=0x1 actions=goto_table:51
cookie=0x8050000, duration=920.124s, table=50, n_packets=0, n_bytes=0, priority=0 actions=CONTROLLER:65535,learn(table=49,hard_timeout=10,priority=0,cookie=0x8600000,NXM_OF_ETH_SRC[],NXM_NX_REG1[0..19],load:0x1->NXM_NX_REG4[0..7]),goto_table:51
cookie=0x8031770, duration=679.649s, table=51, n_packets=101, n_bytes=9720, priority=20,metadata=0x1770000000/0xffff000000,dl_dst=fa:16:3e:ff:2b:c3 actions=set_field:0x2d->tun_id,load:0x900->NXM_NX_REG6[],resubmit(,220)
cookie=0x8031770, duration=679.649s, table=51, n_packets=74, n_bytes=7842, priority=20,metadata=0x1770000000/0xffff000000,dl_dst=fa:16:3e:13:d9:3c actions=load:0x300->NXM_NX_REG6[],resubmit(,220)
cookie=0x8030000, duration=920.124s, table=51, n_packets=0, n_bytes=0, priority=15,dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop
cookie=0x8030000, duration=920.124s, table=51, n_packets=18, n_bytes=2228, priority=0 actions=goto_table:52
cookie=0x8701770, duration=679.649s, table=52, n_packets=9, n_bytes=1138, priority=5,metadata=0x1770000000/0xffff000001 actions=write_actions(group:212000)
cookie=0x8701770, duration=679.649s, table=52, n_packets=0, n_bytes=0, priority=5,metadata=0x1770000001/0xffff000001 actions=write_actions(group:211999)
cookie=0x8800003, duration=679.649s, table=55, n_packets=9, n_bytes=1138, priority=10,tun_id=0x3,metadata=0x30000000000/0xfffff0000000000 actions=drop
cookie=0x8800003, duration=679.649s, table=55, n_packets=0, n_bytes=0, priority=9,tun_id=0x3 actions=load:0x300->NXM_NX_REG6[],resubmit(,220)
cookie=0x6800000, duration=920.124s, table=60, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x1030000, duration=920.124s, table=80, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x1c220005, duration=682.398s, table=81, n_packets=1, n_bytes=42, priority=100,arp,metadata=0x9000030000033c20/0xffffff0000fffffe,arp_tpa=20.0.0.1,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:41:61:2f>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e41612f->NXM_NX_ARP_SHA[],load:0x14000001->NXM_OF_ARP_SPA[],load:0->NXM_OF_IN_PORT[],load:0x300->NXM_NX_REG6[],resubmit(,220)
cookie=0x8220000, duration=920.124s, table=81, n_packets=3, n_bytes=126, priority=0 actions=drop
cookie=0x4000001, duration=920.125s, table=90, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x6900000, duration=678.929s, table=211, n_packets=2, n_bytes=676, priority=63010,udp,metadata=0x30000000000/0xfffff0000000000,tp_src=68,tp_dst=67 actions=resubmit(,17)
cookie=0x6900000, duration=678.907s, table=211, n_packets=0, n_bytes=0, priority=63010,udp6,metadata=0x30000000000/0xfffff0000000000,tp_src=546,tp_dst=547 actions=resubmit(,17)
cookie=0x6900000, duration=678.881s, table=211, n_packets=0, n_bytes=0, priority=63010,udp,metadata=0x30000000000/0xfffff0000000000,tp_src=67,tp_dst=68 actions=drop
cookie=0x6900000, duration=678.852s, table=211, n_packets=0, n_bytes=0, priority=63010,udp6,metadata=0x30000000000/0xfffff0000000000,tp_src=547,tp_dst=546 actions=drop
cookie=0x6900000, duration=678.827s, table=211, n_packets=0, n_bytes=0, priority=63020,icmp6,metadata=0x30000000000/0xfffff0000000000,icmp_type=134,icmp_code=0 actions=drop
cookie=0x6900000, duration=678.805s, table=211, n_packets=3, n_bytes=210, priority=63010,icmp6,metadata=0x30000000000/0xfffff0000000000,icmp_type=133,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=678.779s, table=211, n_packets=1, n_bytes=78, priority=63010,icmp6,metadata=0x30000000000/0xfffff0000000000,icmp_type=135,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=678.701s, table=211, n_packets=0, n_bytes=0, priority=63010,icmp6,metadata=0x30000000000/0xfffff0000000000,icmp_type=136,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=678.679s, table=211, n_packets=2, n_bytes=84, priority=63010,arp,metadata=0x30000000000/0xfffff0000000000,arp_sha=fa:16:3e:13:d9:3c actions=resubmit(,17)
cookie=0x6900000, duration=678.643s, table=211, n_packets=107, n_bytes=10308, priority=61010,ip,dl_src=fa:16:3e:13:d9:3c,nw_src=20.0.0.11 actions=ct(table=212,zone=6000)
cookie=0x6900000, duration=678.606s, table=211, n_packets=1, n_bytes=90, priority=61010,ipv6,dl_src=fa:16:3e:13:d9:3c,ipv6_src=fe80::f816:3eff:fe13:d93c actions=ct(table=212,zone=6000)
cookie=0x6900000, duration=920.125s, table=211, n_packets=3, n_bytes=270, priority=0 actions=drop
cookie=0x6900000, duration=678.445s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1770000000/0xffff000000,nw_dst=20.0.0.11 actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=678.291s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1771000000/0xffff000000,nw_dst=10.0.0.9 actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=678.399s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1770000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fe13:d93c actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=678.302s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1771000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fe16:21cc actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=920.125s, table=212, n_packets=215, n_bytes=20706, priority=0 actions=goto_table:213
cookie=0x6900000, duration=920.125s, table=213, n_packets=158, n_bytes=16004, priority=62020,ct_state=-new+est-rel-inv+trk actions=resubmit(,17)
cookie=0x6900000, duration=920.125s, table=213, n_packets=0, n_bytes=0, priority=62020,ct_state=-new-est+rel-inv+trk actions=resubmit(,17)
cookie=0x6900001, duration=678.516s, table=213, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,metadata=0x30000000000/0xfffff0000000000 actions=drop
cookie=0x6900000, duration=678.490s, table=213, n_packets=1, n_bytes=90, priority=1000,ct_state=+new+trk,ipv6,metadata=0x30000000000/0xfffff0000000000 actions=ct(commit,zone=6000),resubmit(,17)
cookie=0x6900000, duration=678.464s, table=213, n_packets=28, n_bytes=2306, priority=1001,ct_state=+new+trk,ip,metadata=0x30000000000/0xfffff0000000000 actions=ct(commit,zone=6000),resubmit(,17)
cookie=0x6900001, duration=678.546s, table=213, n_packets=0, n_bytes=0, priority=50,ct_state=+new+trk,metadata=0x30000000000/0xfffff0000000000 actions=drop
cookie=0x6900000, duration=920.125s, table=213, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x8000230, duration=920.125s, table=220, n_packets=0, n_bytes=0, priority=6,reg6=0xa00 actions=load:0x90000a00->NXM_NX_REG6[],write_metadata:0/0xfffffffffe,goto_table:230
cookie=0x8000007, duration=920.125s, table=220, n_packets=220, n_bytes=21668, priority=9,reg6=0x90000900 actions=output:2
cookie=0x8000230, duration=920.125s, table=220, n_packets=220, n_bytes=21668, priority=6,reg6=0x900 actions=load:0x90000900->NXM_NX_REG6[],write_metadata:0/0xfffffffffe,goto_table:230
cookie=0x8000007, duration=920.125s, table=220, n_packets=0, n_bytes=0, priority=9,reg6=0x90000a00 actions=output:3
cookie=0x8000007, duration=682.220s, table=220, n_packets=75, n_bytes=7884, priority=9,reg6=0x90000300 actions=output:4
cookie=0x6900000, duration=682.220s, table=220, n_packets=81, n_bytes=8472, priority=6,reg6=0x300 actions=load:0x90000300->NXM_NX_REG6[],write_metadata:0x1770000000/0xfffffffffe,goto_table:241
cookie=0x8000230, duration=920.125s, table=230, n_packets=220, n_bytes=21668, priority=0 actions=resubmit(,220)
cookie=0x8000231, duration=920.125s, table=231, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,220)
cookie=0x6900000, duration=682.849s, table=241, n_packets=2, n_bytes=756, priority=63010,udp,reg6=0x300/0xfffff00,tp_src=67,tp_dst=68 actions=resubmit(,220)
cookie=0x6900000, duration=682.200s, table=241, n_packets=0, n_bytes=0, priority=63010,udp6,reg6=0x300/0xfffff00,tp_src=547,tp_dst=546 actions=resubmit(,220)
cookie=0x6900000, duration=681.064s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x300/0xfffff00,icmp_type=130,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=680.785s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x300/0xfffff00,icmp_type=135,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=680.714s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x300/0xfffff00,icmp_type=136,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=680.204s, table=241, n_packets=2, n_bytes=84, priority=63010,arp,reg6=0x300/0xfffff00 actions=resubmit(,220)
cookie=0x6900000, duration=680.070s, table=241, n_packets=77, n_bytes=7632, priority=61010,ip,dl_dst=fa:16:3e:13:d9:3c,nw_dst=20.0.0.11 actions=ct(table=242,zone=6000)
cookie=0x6900000, duration=679.850s, table=241, n_packets=0, n_bytes=0, priority=61010,ipv6,dl_dst=fa:16:3e:13:d9:3c,ipv6_dst=fe80::f816:3eff:fe13:d93c actions=ct(table=242,zone=6000)
cookie=0x6900000, duration=920.125s, table=241, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x6900000, duration=679.240s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1770000000/0xffff000000,nw_src=20.0.0.11 actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=679.018s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1771000000/0xffff000000,nw_src=10.0.0.9 actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=679.220s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1770000000/0xffff000000,ipv6_src=fe80::f816:3eff:fe13:d93c actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=679.049s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1771000000/0xffff000000,ipv6_src=fe80::f816:3eff:fe16:21cc actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=920.124s, table=242, n_packets=154, n_bytes=15263, priority=0 actions=goto_table:243
cookie=0x6900000, duration=920.124s, table=243, n_packets=142, n_bytes=14087, priority=62020,ct_state=-new+est-rel-inv+trk actions=resubmit(,220)
cookie=0x6900000, duration=920.124s, table=243, n_packets=0, n_bytes=0, priority=62020,ct_state=-new-est+rel-inv+trk actions=resubmit(,220)
cookie=0x6900001, duration=679.705s, table=243, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,reg6=0x300/0xfffff00 actions=drop
cookie=0x6900000, duration=679.678s, table=243, n_packets=0, n_bytes=0, priority=1000,ct_state=+new+trk,ip,reg6=0x300/0xfffff00,metadata=0x2/0xfffffe actions=ct(commit,zone=6000),resubmit(,220)
cookie=0x6900000, duration=679.628s, table=243, n_packets=0, n_bytes=0, priority=1001,ct_state=+new+trk,ipv6,reg6=0x300/0xfffff00,metadata=0x2/0xfffffe actions=ct(commit,zone=6000),resubmit(,220)
cookie=0x6900001, duration=679.808s, table=243, n_packets=6, n_bytes=588, priority=50,ct_state=+new+trk,reg6=0x300/0xfffff00 actions=drop
cookie=0x6900000, duration=920.125s, table=243, n_packets=0, n_bytes=0, priority=0 actions=drop

compute host2:
[root@ocata-compute2 ~]# sudo ovs-ofctl -O OpenFlow13 dump-flows br-int
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x8000001, duration=2008.846s, table=0, n_packets=2854, n_bytes=213297, priority=5,in_port=2 actions=write_metadata:0x40000000001/0xfffff0000000001,goto_table:36
cookie=0x8000001, duration=1854.994s, table=0, n_packets=2488, n_bytes=181645, priority=5,in_port=3 actions=write_metadata:0x70000000001/0xfffff0000000001,goto_table:36
cookie=0x8000000, duration=209.219s, table=0, n_packets=117, n_bytes=11536, priority=4,in_port=4,vlan_tci=0x0000/0x1fff actions=write_metadata:0x60000000000/0xffffff0000000001,goto_table:17
cookie=0x8000001, duration=498.582s, table=17, n_packets=0, n_bytes=0, priority=10,metadata=0x30000000000/0xffffff0000000000 actions=load:0x19e10->NXM_NX_REG3[0..24],write_metadata:0x9000030000033c20/0xfffffffffffffffe,goto_table:19
cookie=0x6900000, duration=206.774s, table=17, n_packets=117, n_bytes=11536, priority=10,metadata=0x60000000000/0xffffff0000000000 actions=write_metadata:0x8000061771000000/0xfffffffffffffffe,goto_table:211
cookie=0x8040000, duration=206.746s, table=17, n_packets=110, n_bytes=10858, priority=10,metadata=0x9000060000000000/0xffffff0000000000 actions=load:0x6->NXM_NX_REG1[0..19],load:0x1771->NXM_NX_REG7[0..15],write_metadata:0xa000061771000000/0xfffffffffffffffe,goto_table:48
cookie=0x8000001, duration=206.746s, table=17, n_packets=116, n_bytes=11446, priority=10,metadata=0x8000060000000000/0xffffff0000000000 actions=load:0x19e10->NXM_NX_REG3[0..24],write_metadata:0x9000060000033c20/0xfffffffffffffffe,goto_table:19
cookie=0x8000000, duration=2532.708s, table=17, n_packets=0, n_bytes=0, priority=0,metadata=0x8000000000000000/0xf000000000000000 actions=write_metadata:0x9000000000000000/0xf000000000000000,goto_table:80
cookie=0x6800000, duration=2532.708s, table=18, n_packets=0, n_bytes=0, priority=0 actions=goto_table:38
cookie=0x1080000, duration=2532.707s, table=19, n_packets=14, n_bytes=588, priority=100,arp,arp_op=1 actions=group:5000
cookie=0x1080000, duration=2532.707s, table=19, n_packets=1, n_bytes=42, priority=100,arp,arp_op=2 actions=CONTROLLER:65535,resubmit(,17)
cookie=0x8000009, duration=208.505s, table=19, n_packets=0, n_bytes=0, priority=20,metadata=0x33c20/0xfffffe,dl_dst=fa:16:3e:41:61:2f actions=goto_table:21
cookie=0x8000009, duration=208.090s, table=19, n_packets=6, n_bytes=588, priority=20,metadata=0x33c20/0xfffffe,dl_dst=fa:16:3e:53:6e:14 actions=goto_table:21
cookie=0x1080000, duration=2532.707s, table=19, n_packets=218, n_bytes=21744, priority=0 actions=resubmit(,17)
cookie=0x1030000, duration=2532.707s, table=20, n_packets=0, n_bytes=0, priority=0 actions=goto_table:80
cookie=0x8000003, duration=208.505s, table=21, n_packets=0, n_bytes=0, priority=42,icmp,metadata=0x33c20/0xfffffe,nw_dst=20.0.0.1,icmp_type=8,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:41:61:2f>eth_src,move:NXM_OF_IP_SRC[]>NXM_OF_IP_DST[],set_field:20.0.0.1>ip_src,set_field:0->icmp_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=208.090s, table=21, n_packets=6, n_bytes=588, priority=42,icmp,metadata=0x33c20/0xfffffe,nw_dst=10.0.0.1,icmp_type=8,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:53:6e:14>eth_src,move:NXM_OF_IP_SRC[]>NXM_OF_IP_DST[],set_field:10.0.0.1>ip_src,set_field:0->icmp_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=208.090s, table=21, n_packets=6, n_bytes=588, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=10.0.0.9 actions=group:155003
cookie=0x8000003, duration=207.150s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=20.0.0.2 actions=set_field:0x2d->tun_id,set_field:fa:16:3e:ff:2b:c3->eth_dst,load:0x400->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=207.150s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=10.0.0.2 actions=set_field:0x4c->tun_id,set_field:fa:16:3e:0e:c0:4b->eth_dst,load:0x400->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=49.973s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x33c20/0xfffffe,nw_dst=20.0.0.11 actions=set_field:0x2d->tun_id,set_field:fa:16:3e:13:d9:3c->eth_dst,load:0x700->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000004, duration=2532.707s, table=22, n_packets=0, n_bytes=0, priority=0 actions=CONTROLLER:65535
cookie=0x1080000, duration=2532.707s, table=23, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x9001771, duration=206.836s, table=36, n_packets=154, n_bytes=16156, priority=5,tun_id=0x4c actions=write_metadata:0x1771000000/0xfffffffff000000,goto_table:51
cookie=0x9051771, duration=209.112s, table=38, n_packets=0, n_bytes=0, priority=5,tun_id=0x4c actions=write_metadata:0x1771000000/0xfffffffff000000,goto_table:51
cookie=0x4000000, duration=2532.708s, table=45, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x8500000, duration=2532.708s, table=48, n_packets=233, n_bytes=22374, priority=0 actions=resubmit(,49),resubmit(,50)
cookie=0x8051771, duration=206.836s, table=50, n_packets=110, n_bytes=10858, priority=20,metadata=0x61771000000/0xfffffffff000000,dl_src=fa:16:3e:16:21:cc actions=goto_table:51
cookie=0x8050001, duration=2532.708s, table=50, n_packets=0, n_bytes=0, priority=10,reg4=0x1 actions=goto_table:51
cookie=0x8050000, duration=2532.708s, table=50, n_packets=0, n_bytes=0, priority=0 actions=CONTROLLER:65535,learn(table=49,hard_timeout=10,priority=0,cookie=0x8600000,NXM_OF_ETH_SRC[],NXM_NX_REG1[0..19],load:0x1->NXM_NX_REG4[0..7]),goto_table:51
cookie=0x8031771, duration=206.836s, table=51, n_packets=74, n_bytes=7840, priority=20,metadata=0x1771000000/0xffff000000,dl_dst=fa:16:3e:16:21:cc actions=load:0x600->NXM_NX_REG6[],resubmit(,220)
cookie=0x8031771, duration=206.836s, table=51, n_packets=101, n_bytes=9720, priority=20,metadata=0x1771000000/0xffff000000,dl_dst=fa:16:3e:0e:c0:4b actions=set_field:0x4c->tun_id,load:0x400->NXM_NX_REG6[],resubmit(,220)
cookie=0x8030000, duration=2532.708s, table=51, n_packets=0, n_bytes=0, priority=15,dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop
cookie=0x8030000, duration=2532.708s, table=51, n_packets=28, n_bytes=2696, priority=0 actions=goto_table:52
cookie=0x8701771, duration=206.836s, table=52, n_packets=0, n_bytes=0, priority=5,metadata=0x1771000001/0xffff000001 actions=write_actions(group:212001)
cookie=0x8701771, duration=206.836s, table=52, n_packets=9, n_bytes=1138, priority=5,metadata=0x1771000000/0xffff000001 actions=write_actions(group:212002)
cookie=0x8800006, duration=206.836s, table=55, n_packets=9, n_bytes=1138, priority=10,tun_id=0x6,metadata=0x60000000000/0xfffff0000000000 actions=drop
cookie=0x8800006, duration=206.836s, table=55, n_packets=0, n_bytes=0, priority=9,tun_id=0x6 actions=load:0x600->NXM_NX_REG6[],resubmit(,220)
cookie=0x6800000, duration=2532.708s, table=60, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x1030000, duration=2532.707s, table=80, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x12220008, duration=208.561s, table=81, n_packets=1, n_bytes=42, priority=100,arp,metadata=0x9000060000033c20/0xffffff0000fffffe,arp_tpa=10.0.0.1,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:53:6e:14>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e536e14->NXM_NX_ARP_SHA[],load:0xa000001->NXM_OF_ARP_SPA[],load:0->NXM_OF_IN_PORT[],load:0x600->NXM_NX_REG6[],resubmit(,220)
cookie=0x8220000, duration=2532.707s, table=81, n_packets=3, n_bytes=126, priority=0 actions=drop
cookie=0x4000001, duration=2532.708s, table=90, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17)
cookie=0x6900000, duration=206.735s, table=211, n_packets=2, n_bytes=676, priority=63010,udp,metadata=0x60000000000/0xfffff0000000000,tp_src=68,tp_dst=67 actions=resubmit(,17)
cookie=0x6900000, duration=206.684s, table=211, n_packets=0, n_bytes=0, priority=63010,udp6,metadata=0x60000000000/0xfffff0000000000,tp_src=546,tp_dst=547 actions=resubmit(,17)
cookie=0x6900000, duration=206.645s, table=211, n_packets=0, n_bytes=0, priority=63010,udp,metadata=0x60000000000/0xfffff0000000000,tp_src=67,tp_dst=68 actions=drop
cookie=0x6900000, duration=206.605s, table=211, n_packets=0, n_bytes=0, priority=63010,udp6,metadata=0x60000000000/0xfffff0000000000,tp_src=547,tp_dst=546 actions=drop
cookie=0x6900000, duration=206.585s, table=211, n_packets=0, n_bytes=0, priority=63020,icmp6,metadata=0x60000000000/0xfffff0000000000,icmp_type=134,icmp_code=0 actions=drop
cookie=0x6900000, duration=206.539s, table=211, n_packets=3, n_bytes=210, priority=63010,icmp6,metadata=0x60000000000/0xfffff0000000000,icmp_type=133,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=206.507s, table=211, n_packets=1, n_bytes=78, priority=63010,icmp6,metadata=0x60000000000/0xfffff0000000000,icmp_type=135,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=206.467s, table=211, n_packets=0, n_bytes=0, priority=63010,icmp6,metadata=0x60000000000/0xfffff0000000000,icmp_type=136,icmp_code=0 actions=resubmit(,17)
cookie=0x6900000, duration=206.397s, table=211, n_packets=2, n_bytes=84, priority=63010,arp,metadata=0x60000000000/0xfffff0000000000,arp_sha=fa:16:3e:16:21:cc actions=resubmit(,17)
cookie=0x6900000, duration=206.319s, table=211, n_packets=1, n_bytes=90, priority=61010,ipv6,dl_src=fa:16:3e:16:21:cc,ipv6_src=fe80::f816:3eff:fe16:21cc actions=ct(table=212,zone=6001)
cookie=0x6900000, duration=206.265s, table=211, n_packets=107, n_bytes=10308, priority=61010,ip,dl_src=fa:16:3e:16:21:cc,nw_src=10.0.0.9 actions=ct(table=212,zone=6001)
cookie=0x6900000, duration=2532.708s, table=211, n_packets=2, n_bytes=180, priority=0 actions=drop
cookie=0x6900000, duration=205.705s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1771000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fe16:21cc actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=46.735s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1770000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fe13:d93c actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=205.651s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1771000000/0xffff000000,nw_dst=10.0.0.9 actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=46.787s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1770000000/0xffff000000,nw_dst=20.0.0.11 actions=write_metadata:0x2/0xfffffe,goto_table:213
cookie=0x6900000, duration=2532.708s, table=212, n_packets=404, n_bytes=39220, priority=0 actions=goto_table:213
cookie=0x6900000, duration=2532.708s, table=213, n_packets=160, n_bytes=16200, priority=62020,ct_state=-new+est-rel-inv+trk actions=resubmit(,17)
cookie=0x6900000, duration=2532.708s, table=213, n_packets=0, n_bytes=0, priority=62020,ct_state=-new-est+rel-inv+trk actions=resubmit(,17)
cookie=0x6900001, duration=206.007s, table=213, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,metadata=0x60000000000/0xfffff0000000000 actions=drop
cookie=0x6900000, duration=205.948s, table=213, n_packets=1, n_bytes=90, priority=1000,ct_state=+new+trk,ipv6,metadata=0x60000000000/0xfffff0000000000 actions=ct(commit,zone=6001),resubmit(,17)
cookie=0x6900000, duration=205.797s, table=213, n_packets=28, n_bytes=2306, priority=1001,ct_state=+new+trk,ip,metadata=0x60000000000/0xfffff0000000000 actions=ct(commit,zone=6001),resubmit(,17)
cookie=0x6900001, duration=206.058s, table=213, n_packets=0, n_bytes=0, priority=50,ct_state=+new+trk,metadata=0x60000000000/0xfffff0000000000 actions=drop
cookie=0x6900000, duration=2532.708s, table=213, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x8000230, duration=2007.365s, table=220, n_packets=235, n_bytes=22570, priority=6,reg6=0x400 actions=load:0x90000400->NXM_NX_REG6[],write_metadata:0/0xfffffffffe,goto_table:230
cookie=0x8000007, duration=2007.365s, table=220, n_packets=235, n_bytes=22570, priority=9,reg6=0x90000400 actions=output:2
cookie=0x8000230, duration=1853.715s, table=220, n_packets=8, n_bytes=784, priority=6,reg6=0x700 actions=load:0x90000700->NXM_NX_REG6[],write_metadata:0/0xfffffffffe,goto_table:230
cookie=0x8000007, duration=1853.715s, table=220, n_packets=8, n_bytes=784, priority=9,reg6=0x90000700 actions=output:3
cookie=0x8000007, duration=207.304s, table=220, n_packets=75, n_bytes=7882, priority=9,reg6=0x90000600 actions=output:4
cookie=0x6900000, duration=207.304s, table=220, n_packets=81, n_bytes=8470, priority=6,reg6=0x600 actions=load:0x90000600->NXM_NX_REG6[],write_metadata:0x1771000000/0xfffffffffe,goto_table:241
cookie=0x8000230, duration=2532.708s, table=230, n_packets=243, n_bytes=23354, priority=0 actions=resubmit(,220)
cookie=0x8000231, duration=2532.708s, table=231, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,220)
cookie=0x6900000, duration=208.533s, table=241, n_packets=2, n_bytes=755, priority=63010,udp,reg6=0x600/0xfffff00,tp_src=67,tp_dst=68 actions=resubmit(,220)
cookie=0x6900000, duration=207.284s, table=241, n_packets=0, n_bytes=0, priority=63010,udp6,reg6=0x600/0xfffff00,tp_src=547,tp_dst=546 actions=resubmit(,220)
cookie=0x6900000, duration=207.137s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x600/0xfffff00,icmp_type=130,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=207.101s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x600/0xfffff00,icmp_type=135,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=207.078s, table=241, n_packets=0, n_bytes=0, priority=63010,icmp6,reg6=0x600/0xfffff00,icmp_type=136,icmp_code=0 actions=resubmit(,220)
cookie=0x6900000, duration=207.049s, table=241, n_packets=2, n_bytes=84, priority=63010,arp,reg6=0x600/0xfffff00 actions=resubmit(,220)
cookie=0x6900000, duration=206.989s, table=241, n_packets=0, n_bytes=0, priority=61010,ipv6,dl_dst=fa:16:3e:16:21:cc,ipv6_dst=fe80::f816:3eff:fe16:21cc actions=ct(table=242,zone=6001)
cookie=0x6900000, duration=206.966s, table=241, n_packets=77, n_bytes=7631, priority=61010,ip,dl_dst=fa:16:3e:16:21:cc,nw_dst=10.0.0.9 actions=ct(table=242,zone=6001)
cookie=0x6900000, duration=2532.708s, table=241, n_packets=0, n_bytes=0, priority=0 actions=drop
cookie=0x6900000, duration=205.917s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x33c20/0xfffffe,ipv6_src=fe80::f816:3eff:fe16:21cc actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=205.841s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x33c20/0xfffffe,nw_src=10.0.0.9 actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=47.596s, table=242, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1770000000/0xffff000000,nw_src=20.0.0.11 actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=47.550s, table=242, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1770000000/0xffff000000,ipv6_src=fe80::f816:3eff:fe13:d93c actions=write_metadata:0x2/0xfffffe,goto_table:243
cookie=0x6900000, duration=2532.708s, table=242, n_packets=334, n_bytes=32902, priority=0 actions=goto_table:243
cookie=0x6900000, duration=2532.708s, table=243, n_packets=146, n_bytes=14478, priority=62020,ct_state=-new+est-rel-inv+trk actions=resubmit(,220)
cookie=0x6900000, duration=2532.708s, table=243, n_packets=0, n_bytes=0, priority=62020,ct_state=-new-est+rel-inv+trk actions=resubmit(,220)
cookie=0x6900001, duration=206.856s, table=243, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,reg6=0x600/0xfffff00 actions=drop
cookie=0x6900000, duration=47.774s, table=243, n_packets=0, n_bytes=0, priority=1000,ct_state=+new+trk,ip,reg6=0x600/0xfffff00,metadata=0x2/0xfffffe actions=ct(commit,zone=6001),resubmit(,220)
cookie=0x6900000, duration=47.634s, table=243, n_packets=0, n_bytes=0, priority=1001,ct_state=+new+trk,ipv6,reg6=0x600/0xfffff00,metadata=0x2/0xfffffe actions=ct(commit,zone...

Please hold on until we give you a go ahead to test out the patches. The patches are still WIP.

Vivek,
We had a blue jeans session now and was able to reproduce the issue.
Thanks
Aswin

On Tue, Jun 27, 2017 at 7:29 PM, N Vivekanandan <n.vivekanandan@ericsson.com> wrote:
Hi Shashidhar,

Aswin, can you please share detailed steps to Shashidhar on your failure.

And Shashi, I kindly request you to run these two tests, as it’s a very frequent
use-case with VPN (All ports only the same default SG):

Test 1 (3 VM required in total):
a. Create two networks N1 and N2
b. Create a subnet A for network N1 and spin off a VM-A1.
c. Create a subnet B for network N2 and spin off a VM -B1
d. Create a router R1 and attach a subnet A and subnet B to the router.
e. Ping from VM-A1 to VM-B1 successfully
f. Create a BGPVPN1 from Openstack
g. Associate the router R1 to the BGPVPN1
h. Again Ping from VM-A1 to VM-B1 successfully.
i. Spin off one more VM-B2 on subnet B.
j. Ping VM-A1 to VM-B2 successfully.
k. Ping VM-B1 to VM-B2 successfully.
l. Disassociate the router R1 from BGPVPN1.
m. Ping VM-A1 to VM-B2 successfully
n. Ping VM-B1 to VM-B2 successfully.
o. Associate the router R1 back into BGPVPN1.
p. Ping VM-A1 to VM-B2 successfully
q. Ping VM-B1 to VM-B2 successfully.
r. Delete the BGPVPN1.
s. Ping VM-A1 to VM-B2 successfully.
t. Ping from VM-B1 to VM-B2 successfully.
u. Remove subnet A from the router R1.
v. Ping VM-A1 to VM-B2 should fail.
w. Ping from VM-B1 to VM-B2 successfully.

Test 2 (3 VM required in total):
a. Create two networks N1 and N2
b. Create a subnet A for network N1 and spin off a VM-A1.
c. Create a subnet B for network N2 and spin off a VM -B1
d. Create a BGPVPN1 from Openstack
e. Associate the network N1 to the BGPVPN1
f. Associate the network N2 to the BGPVPN1
g. Ping from VM-A1 to VM-B1 successfully.
h. Spin off one more VM-B2 on subnet B.
i. Ping VM-A1 to VM-B2 successfully.
j. Ping VM-B1 to VM-B2 successfully.
k. Disassociate the network N1 from BGPVPN1.
l. Ping VM-A1 to VM-B2 should fail.
m. Ping VM-B1 to VM-B2 successfully.
n. Associate the network N1 back into BGPVPN1.
o. Ping VM-A1 to VM-B2 successfully.
p. Ping VM-B1 to VM-B2 successfully.
q. Delete the bgpvpn BGPVPN1
r. Ping VM-A1 to VM-B2 should fail.
s. Ping VM-B1 to VM-B2 successfully.

–
Thanks,

Vivek

From: Shashidhar R shashidharr@altencalsoftlabs.com
Sent: Tuesday, June 27, 2017 7:03 PM
To: 'Aswin Suryanarayanan' <asuryana@redhat.com>
Cc: N Vivekanandan <n.vivekanandan@ericsson.com>; Faseela K <faseela.k@ericsson.com>; Somashekar Byrappa <somashekar.b@altencalsoftlabs.com>; Kiran N Upadhyaya <kiran.n.upadhyaya@ericsson.com>; Ravi Sundareswaran <ravi.sundareswaran@ericsson.com>; 'Sam Hague' <shague@redhat.com>
Subject: RE: Carbon ACL blocker NETVIRT-657

Hi Aswin,

I could not reproduce issue mentioned by you. I will continue to test further tomorrow.

Meantime, below are pending items with this commit:
(a) Fix UT failures
(b) Change logic to update metadata of bound service rather than doing unbind and bind service

Thanks,
Shashidhar

From: Aswin Suryanarayanan asuryana@redhat.com
Sent: 26 June 2017 19:19
To: Shashidhar R <shashidharr@altencalsoftlabs.com>
Cc: N Vivekanandan <n.vivekanandan@ericsson.com>; Faseela K <faseela.k@ericsson.com>; Somashekar Byrappa <somashekar.b@altencalsoftlabs.com>; Kiran N Upadhyaya <kiran.n.upadhyaya@ericsson.com>; Ravi Sundareswaran <ravi.sundareswaran@ericsson.com>; Sam Hague <shague@redhat.com>
Subject: Re: Carbon ACL blocker NETVIRT-657

+Sam

Shashi,
I have run the sandbox against latest patch. The last one seems to be failing badly, but when I test locally I see issues only when a thrid vm with a remote group is deleted. It affects traffic between remaining two. Otherwise all the issues seems to be fixed, have started a new build now

[1]https://jenkins.opendaylight.org/sandbox/job/netvirt-csit-1node-openstack-newton-nodl-v2-jamo-upstream-stateful-carbon/2/robot/netvirt-1node-openstack.txt/Connectivity/04%20Security%20Group%20Tests/
Thanks
Aswin

On Fri, Jun 23, 2017 at 7:55 PM, Shashidhar R <shashidharr@altencalsoftlabs.com> wrote:
Hi Aswin,

Latest patch pushed today fixes issues observed by you.
My changes are in https://git.opendaylight.org/gerrit/#/c/58412/

Thanks,
Shashidhar

From: Aswin Suryanarayanan asuryana@redhat.com
Sent: 20 June 2017 20:29
To: Shashidhar R <shashidharr@altencalsoftlabs.com>
Cc: N Vivekanandan <n.vivekanandan@ericsson.com>; Faseela K <faseela.k@ericsson.com>; Somashekar Byrappa <somashekar.b@altencalsoftlabs.com>; Kiran N Upadhyaya <kiran.n.upadhyaya@ericsson.com>; Ravi Sundareswaran <ravi.sundareswaran@ericsson.com>
Subject: Re: Carbon ACL blocker NETVIRT-657

Shashi,
I have added the test case [1]to cover the issues addressed as a part of the patch. I have started the sandbox for the same. When tested locally I faced connectivity issues when an second SG is removed from the vms. The router interface delete test case also failed. May be due to the former.

[1]https://git.opendaylight.org/gerrit/#/c/59161/
[2]https://jenkins.opendaylight.org/sandbox/job/netvirt-csit-1node-openstack-newton-jamo-upstream-stateful-carbon/7/

On Tue, Jun 20, 2017 at 12:17 PM, Shashidhar R <shashidharr@altencalsoftlabs.com> wrote:
Hi Vivek,

https://git.opendaylight.org/gerrit/#/c/57705/ is required; I have created my review as dependent review on it.

I have also added reviewers.

Thanks,
Shashidhar

From: N Vivekanandan n.vivekanandan@ericsson.com
Sent: 19 June 2017 19:50
To: Shashidhar Raja <shashidharr@altencalsoftlabs.com>; Faseela K <faseela.k@ericsson.com>; 'Aswin Suryanarayanan' <asuryana@redhat.com>; Somashekar Byrappa <somashekar.b@altencalsoftlabs.com>
Cc: Kiran N Upadhyaya <kiran.n.upadhyaya@ericsson.com>; Ravi Sundareswaran <ravi.sundareswaran@ericsson.com>
Subject: RE: Carbon ACL blocker NETVIRT-657

Please add us as reviewers !

And does that mean https://git.opendaylight.org/gerrit/#/c/57705/
is not required anymore?

Vivek

From: Shashidhar R shashidharr@altencalsoftlabs.com
Sent: Monday, June 19, 2017 7:36 PM
To: N Vivekanandan <n.vivekanandan@ericsson.com>; Faseela K <faseela.k@ericsson.com>; 'Aswin Suryanarayanan' <asuryana@redhat.com>; Somashekar Byrappa <somashekar.b@altencalsoftlabs.com>
Cc: Kiran N Upadhyaya <kiran.n.upadhyaya@ericsson.com>; Ravi Sundareswaran <ravi.sundareswaran@ericsson.com>
Subject: RE: Carbon ACL blocker NETVIRT-657

Hi Vivek,

My changes are in https://git.opendaylight.org/gerrit/#/c/58412/ and latest PS was pushed last Friday.

Thanks,
Shashidhar

Resolved in Carbon,

[1] https://git.opendaylight.org/gerrit/#/c/58412/
[2]https://git.opendaylight.org/gerrit/#/c/57705/

Waiting for review in Master

[1]https://git.opendaylight.org/gerrit/#/c/57742/

Patches in master.
https://git.opendaylight.org/gerrit/#/c/58412/
https://git.opendaylight.org/gerrit/#/c/57742/

https://git.opendaylight.org/gerrit/#/c/59814/

The issue still exists, doesn't seem to be resolved:
Tested with the latest distro, pfb for the link:

https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/distribution-karaf/0.6.1-SNAPSHOT/distribution-karaf-0.6.1-20170705.175423-560.tar.gz

Trying to ping from 10.0.0.5 to 20.0.0.4 VM's.The flows in the table 212 the destination doesnot match :

[root@compute2-packstack ~]# sudo ovs-ofctl -O OpenFlow13 dump-flows br-int table=212
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x6900000, duration=90.537s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x138a000000/0xffff000000,nw_dst=20.0.0.4 actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=90.044s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x138a000000/0xffff000000,nw_dst=20.0.0.9 actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=90.044s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x138a000000/0xffff000000,nw_dst=20.0.0.7 actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=90.044s, table=212, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x1389000000/0xffff000000,nw_dst=10.0.0.5 actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=90.537s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x138a000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fe32:b02f actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=90.044s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x138a000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fe71:eecb actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=90.044s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x138a000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fe5f:853d actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=90.044s, table=212, n_packets=0, n_bytes=0, priority=50,ipv6,metadata=0x1389000000/0xffff000000,ipv6_dst=fe80::f816:3eff:fedb:1e43 actions=write_metadata:0x8/0xfffffe,goto_table:213
cookie=0x6900000, duration=1880.867s, table=212, n_packets=710, n_bytes=68920, priority=0 actions=goto_table:213

Also all the packets are dropped in the table 243 :

[root@compute2-packstack ~]# sudo ovs-ofctl -O OpenFlow13 dump-flows br-int table=243
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x6900000, duration=1912.379s, table=243, n_packets=0, n_bytes=0, priority=62020,ct_state=-new-est+rel-inv+trk actions=resubmit(,220)
cookie=0x6900000, duration=1912.379s, table=243, n_packets=179, n_bytes=17712, priority=62020,ct_state=-new+est-rel-inv+trk actions=resubmit(,220)
cookie=0x6900001, duration=1710.977s, table=243, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,reg6=0xa00/0xfffff00 actions=drop
cookie=0x6900001, duration=1527.571s, table=243, n_packets=0, n_bytes=0, priority=62015,ct_state=+inv+trk,reg6=0xd00/0xfffff00 actions=drop
cookie=0x6900000, duration=122.780s, table=243, n_packets=0, n_bytes=0, priority=1003,ct_state=+new+trk,ip,reg6=0xd00/0xfffff00,metadata=0x8/0xfffffe actions=ct(commit,zone=5002),resubmit(,220)
cookie=0x6900000, duration=122.765s, table=243, n_packets=0, n_bytes=0, priority=1005,ct_state=+new+trk,ipv6,reg6=0xd00/0xfffff00,metadata=0x8/0xfffffe actions=ct(commit,zone=5002),resubmit(,220)
cookie=0x6900000, duration=122.764s, table=243, n_packets=0, n_bytes=0, priority=1000,ct_state=+new+trk,ip,reg6=0xa00/0xfffff00,metadata=0x8/0xfffffe actions=ct(commit,zone=5001),resubmit(,220)
cookie=0x6900000, duration=122.708s, table=243, n_packets=0, n_bytes=0, priority=1004,ct_state=+new+trk,ipv6,reg6=0xa00/0xfffff00,metadata=0x8/0xfffffe actions=ct(commit,zone=5001),resubmit(,220)
cookie=0x6900001, duration=1710.977s, table=243, n_packets=0, n_bytes=0, priority=50,ct_state=+new+trk,reg6=0xa00/0xfffff00 actions=drop
cookie=0x6900001, duration=1527.602s, table=243, n_packets=77, n_bytes=7258, priority=50,ct_state=+new+trk,reg6=0xd00/0xfffff00 actions=drop
cookie=0x6900000, duration=1912.379s, table=243, n_packets=0, n_bytes=0, priority=0 actions=drop

Detailed flows available in the attachment.

Attachment bug8398_dumpflows.txt has been added with description: atatching the dump flows

Retested the ping is working between the vm's but the ping from vm to router interface is failing. the ping to the same network router interface and the other network interface also fails

Router ping fails due to [1] this requires a fix in Ovs which is currently being worked on.

[1]https://bugs.opendaylight.org/show_bug.cgi?id=7545

Thanks Aswin.

Yogalakshmi,

Let us track ping failure to router-interfaces(same or different subnet) with Stateful-SG in the other NETVIRT-430.

Since PINGs between VMs across subnets with default SG in Stateful Mode is working fine as per the original reason for this bug, we are good with the fix.

Vivek

Has this blocker bug been verified as fixed in Carbon SR1 Build 20170711?

HI Balakrishnan/Yogalakshmi,

We are moving this bug to 'WORKSFORME' which is 'VERIFIED' state based on your runs of build 20170705.

Please get back if you find any issues with carbon SR1 build 20170711.

Vivek