https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/NETVIRT-659 netvirt <p>Setup Used:</p> <p>1. 3 node ODL cluster + HA proxy<br/> 2. new netvirt (carbon)<br/> 3. Packstack with ocata.</p> <p>Steps followed for testing:</p> <p>Scenario 1:<br/> ------------<br/> 1.Created Network.<br/> 2. Created 2 VM's<br/> 3.Created SG1 with TCP ALL rule.<br/> 4. Created SG2 without any rule.<br/> 5.Applied SG1 to both the VM's.<br/> 6.Observed TCP communication is working for Ingress and Egress as well.<br/> 7.Removed SG1 rule and Applied SG2 to both Vm's during TCP communication.</p> <p>Observations:<br/> ---------------<br/> 1) Observed TCP communication still working in same session for Ingress and Egress as well.</p> <p>2) Observed TCP communication is does not working with a new session for Ingress and Egress as well.</p> <p>Scenario 2:<br/> -----------<br/> 1.Created Network.<br/> 2. Created 2 VM's<br/> 3.Created SG1 with TCP ALL rule.<br/> 4.Applied SG1 to both the VM's.<br/> 5.Observed TCP communication is working for Ingress and Egress as well.<br/> 6.Removed TCP ALL rule from both the VM's during TCP communication.</p> <p>Observations:<br/> ---------------<br/> 1) Observed TCP communication still working in same session for Ingress and Egress as well.</p> <p>2) Observed TCP communication is does not working with a new session for Ingress and Egress as well.</p> <p>Operating System: All<br/> Platform: All</p> NETVIRT-659

TCP communication is working, even remove the TCP SG rule from the VM instance.

Bug Medium Resolved Done Vinoth B Hari Prasidh Mon, 8 May 2017 12:41:28 +0000 Thu, 3 May 2018 14:37:00 +0000 Thu, 5 Apr 2018 20:46:47 +0000 Carbon General 0 15 <p>While evaluating the bug, we have found an another issue. <br/> Ref: <a href="https://bugs.opendaylight.org/show_bug.cgi?id=8553" class="external-link" target="_blank" rel="nofollow noopener">https://bugs.opendaylight.org/show_bug.cgi?id=8553</a></p> <p>Note:</p> <p>I've tested the same scenario with Any SG rule (TCP/ICMP/UDP) and observed the issue.</p> <p>This is since conntrack entries are not deleted when a SG rule is deleted. We need to find a way to prevent the traffic corresponding the deleted rules not hitting +est. One way to workaround this is add a higher priority drop rule to match the traffic corresponding to deleted rule with a timeout.</p> <p>Observation - 1:</p> <p>Created 2 VMs with ICMP ALL rule.. When ping operation begin between 2 VMs , the below conntrack entry has been created,</p> <p> <span class="error">&#91;1&#93;</span> <span class="error">&#91;root@ocata-compute1 ~&#93;</span># conntrack -L | grep icmp<br/> conntrack v1.4.3 (conntrack-tools): 38 flow entries have been shown.<br/> icmp 1 19 src=20.0.0.5 dst=20.0.0.12 type=8 code=0 id=31489 src=20.0.0.12 dst=20.0.0.5 type=0 code=0 id=31489 mark=0 zone=6002 use=1</p> <p>This entry will be available until we stopped the ping communication.</p> <p>When we remove the ICMP rule from the SG, Ongoing communication is not getting disturbed because the respective conntrack entry <span class="error">&#91;1&#93;</span> is still exists and make the communication to be successful.</p> <p>Observation - 2:</p> <p>Executed the same test case in pure openstack and observered that ongoing communication has been stopped and respective conntrack rule also removed, when we remove the rule.</p> <p>As mentioned earlier we could add a higher priority rule to drop the traffic. The delete action will overwrite the commit flow with a drop flow of higher priority with a timeout. The value for timeout for each protocol can be selected from <span class="error">&#91;1&#93;</span>.</p> <p><span class="error">&#91;1&#93;</span>https://github.com/opendaylight/netvirt/blob/master/vpnservice/aclservice/impl/src/main/yang/aclservice-config.yang</p> <p>Observation - 3:</p> <p>Setup : pure openstack with OVS firewall,</p> <p>1. Created 1 VM with ICMP ALL ingress rule and it installs the below specified flows in the flow table.</p> <p> <span class="error">&#91;1&#93;</span> cookie=0xa99821b3245d1aa6, duration=95.226s, table=82, n_packets=23, n_bytes=2254, reset_counts priority=70,ct_state=+est-rel-rpl,icmp,reg5=0x4,dl_dst=fa:16:3e:7c:07:75 actions=pop_vlan,output:4 </p> <p> <span class="error">&#91;2&#93;</span> cookie=0xa99821b3245d1aa6, duration=95.225s, table=82, n_packets=1, n_bytes=98, reset_counts priority=70,ct_state=+new-est,icmp,reg5=0x4,dl_dst=fa:16:3e:7c:07:75 actions=ct(commit,zone=NXM_NX_REG6<span class="error">&#91;0..15&#93;</span>),pop_vlan,output:4</p> <p> <span class="error">&#91;1&#93;</span> Kernal conntrack entry flow<br/> <span class="error">&#91;2&#93;</span> SG flow</p> <p>2. Pinged from DHCP to VM and observed that the first packet is going through the SG flow and further packets are hitting the kernal conntrack entry flow.</p> <p>3. When we delete the ICMP ingress rule, the on going communication has been stopped because the respective conntrack kernal entry flow is also getting deleted along with the SG flow.</p> <p>But in ODL, since we have only one global Kernal conntrack entry flow for all the VMs, The ongoing communication is still succeeded even though we removed the SG rule from VM.</p> <p>(In reply to Vinoth B from comment #6)<br/> &gt; Observation - 3:<br/> &gt; <br/> &gt; Setup : pure openstack with OVS firewall,<br/> &gt; <br/> &gt; 1. Created 1 VM with ICMP ALL ingress rule and it installs the below<br/> &gt; specified flows in the flow table.<br/> &gt; <br/> &gt; <span class="error">&#91;1&#93;</span> cookie=0xa99821b3245d1aa6, duration=95.226s, table=82,<br/> &gt; n_packets=23, n_bytes=2254, reset_counts<br/> &gt; priority=70,ct_state=+est-rel-rpl,icmp,reg5=0x4,dl_dst=fa:16:3e:7c:07:75<br/> &gt; actions=pop_vlan,output:4 <br/> &gt; <br/> &gt; <span class="error">&#91;2&#93;</span> cookie=0xa99821b3245d1aa6, duration=95.225s, table=82,<br/> &gt; n_packets=1, n_bytes=98, reset_counts<br/> &gt; priority=70,ct_state=+new-est,icmp,reg5=0x4,dl_dst=fa:16:3e:7c:07:75<br/> &gt; actions=ct(commit,zone=NXM_NX_REG6<span class="error">&#91;0..15&#93;</span>),pop_vlan,output:4<br/> &gt; <br/> &gt; <span class="error">&#91;1&#93;</span> Kernal conntrack entry flow<br/> &gt; <span class="error">&#91;2&#93;</span> SG flow<br/> &gt; <br/> &gt; 2. Pinged from DHCP to VM and observed that the first packet is going<br/> &gt; through the SG flow and further packets are hitting the kernal conntrack<br/> &gt; entry flow.<br/> &gt; <br/> &gt; 3. When we delete the ICMP ingress rule, the on going communication has been<br/> &gt; stopped because the respective conntrack kernal entry flow is also getting<br/> &gt; deleted along with the SG flow.<br/> &gt; <br/> &gt; But in ODL, since we have only one global Kernal conntrack entry flow for<br/> &gt; all the VMs, The ongoing communication is still succeeded even though we<br/> &gt; removed the SG rule from VM.</p> <p>Shall i proceed the way how OVS firewall handles this issue? I mean adding the conntrack-Kernel entry per rule instead of common for all the VMs.</p> <p>Adding a +est rule per vm defeat the purpose of using connection tracking in my opinion. An established packet need to traverse through all the rules which is added per vm before we have a hit. </p> <p>I would suggest adding a drop rule on deletion would be better solution. A similar proposal implemented in OVN<span class="error">&#91;1&#93;</span>,<span class="error">&#91;2&#93;</span>.</p> <p><span class="error">&#91;1&#93;</span>https://github.com/openvswitch/ovs/commit/cc58e1f2cc414b844c36d21a9a39e4f3383e75e4<br/> <span class="error">&#91;2&#93;</span>https://mail.openvswitch.org/pipermail/ovs-dev/2016-February/065716.html</p> <p>Aswin,</p> <p>The DROP flow fix is fine. </p> <p>Fix Explanation :</p> <p> When we delete a specific SG rule from the VM, netvirt should add the high priority drop rule with the exact match condition with idle timeout. This drop flow will stops the existing communication.</p> <p> Similarly, When we add a same rule again, netvirt should remove the added drop flow to make the existing stopped communication to be resumed. </p> <p>We have to look on below problem before starting the implementation.<br/> Consider we are associating two ALL ICMP Ingress rule to one VM. So, we could see two ICMP ingress flow with different priority for the same VM. When we delete one ICMP ingress rule from the VM, then we will add one drop rule as per the fix above, which make the existing communication to break irrespective of exact ICMP ingress flow still exists in the flow table.<br/> So before adding the drop rule, we have to check whether the VM associated with similar rule or not. </p> <p>Apart from this, this approach is fine. Shall | start the implementation for this fix?</p> <p>I have pushed a draft patch to fix this issue.</p> <p>Netvirt:<br/> <a href="https://git.opendaylight.org/gerrit/#/c/60077/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/60077/</a></p> <p>Fix provided:</p> <p>Added a DROP rule with idle_timeout(5 secs) when we delete or remove the SG from the VM. If we add the same rule again to the VM, Netvirt will delete the added DROP rule and make the existing communication to be resume.</p> <p>Limitations :</p> <p>1) When we add the two similar rules to the VMs (SG1 with ALL ICMP ingress, SG2 with ALL ICMP ingress), there are two ALL ICMP ingress flow will be exists in the flow table with different priority. while removing SG1 from VM, as per the above fix, the DROP rule will be added which stops the existing ICMP communication irrespective of there is another similar rule (SG2) is still exist in the VM.</p> <p>2) After establishing SSH communication between VMs, TCP doesn't send packets continuously. So the added DROP flow is getting removed from the flow table after reaching the idle timeout 5 secs. Observed the same behavior also for SCP communication</p> <p>The draft is not accessible. Please publish the patch or add reviewers.</p> <p>Since the fix having some limitation, I am working on the new approach which will resolve all the limitation.</p> <p>The new approach involves pipeline changes, So I have pushed the spec patch.</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/60580/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/60580/</a></p> <p><a href="https://git.opendaylight.org/gerrit/66788" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/66788</a></p> Development External issue ID 8400 External issue URL Rank 0|i01t5z: