https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/VTN-121 vtn <p>The following CSIT for Beryllium VTN Manager occasional failed to test flow filter functions.<br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/</a></p> <p>For example, there was one failure on the build #81.<br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/81/robot/report/log.html" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/81/robot/report/log.html</a><br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/81/robot/report/log.html#s1-s1-s3-t19-k1" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/81/robot/report/log.html#s1-s1-s3-t19-k1</a></p> <p>For example, there was one failure on the build #82.<br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/82/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/82/</a><br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/82/robot/report/log.html#s1-s2-s4-t19-k1" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/82/robot/report/log.html#s1-s2-s4-t19-k1</a></p> <p>The CSIT configures flow filters, and sends packets which match the flow filters.<br/> After that, the CSIT checks if flow entries for the flow filters are correctly created.<br/> However, the CSIT found that the flow entries were not created.</p> <p>Operating System: All<br/> Platform: All</p> VTN-121

CSIT for VTN Manager failed to test flow filter

Bug Resolved Done Karthik Sivasamy Hideyuki Tai Wed, 17 Feb 2016 17:56:22 +0000 Tue, 22 Mar 2016 09:57:57 +0000 Tue, 22 Mar 2016 09:57:57 +0000 unspecified VTN Manager 0 2 <p>It seems to me that there were other applications which installed flow entries during the tests.</p> <p>The following page shows flow entries during failed test case.<br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/81/robot/report/log.html#s1-s1-s3-t19-k1-k20-k1-k1-k3-k1-k1" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/81/robot/report/log.html#s1-s1-s3-t19-k1-k20-k1-k1-k3-k1-k1</a><br/> However, it seems to me that these flow entries are different from what VTN Manager installs.<br/> I guess other application installed these flow entries.<br/> I guess that is related to the failure (<a href="https://jira.opendaylight.org/browse/VTN-121" title="CSIT for VTN Manager failed to test flow filter" class="issue-link" data-issue-key="VTN-121"><del>VTN-121</del></a>).</p> <p>For your information, the following page shows flow entries while this test case worked well. <br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/82/robot/report/log.html#s1-s1-s3-t19-k1-k1-k1-k1-k2" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/82/robot/report/log.html#s1-s1-s3-t19-k1-k1-k1-k1-k2</a></p> <p>Reading the karaf.log on the CSIT, I've noticed that SFC OFL2 Renderer and NetVirt was installed. These are applications which install flow entries into OpenFlow switches.</p> <p>And, I've found the following patch which, I guess, would fix the issue.<br/> <a href="https://git.opendaylight.org/gerrit/#/c/34780/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/34780/</a></p> <p>Let's wait for the merge of the patch, and see if the same failure happens after the merge.</p> <p>The error have not occurred in the CSIT for beryllium since the following patch is merged into the beryllium branch of the distribution group.<br/> <a href="https://git.opendaylight.org/gerrit/#/c/34780/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/34780/</a></p> <p>However, Similar failures happened in the CSIT for the master branch.<br/> <a href="https://jenkins.opendaylight.org/releng/job/vtn-csit-1node-manager-only-boron/133/robot/report/log.html#s1-s1-s3" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/job/vtn-csit-1node-manager-only-boron/133/robot/report/log.html#s1-s1-s3</a></p> <p>FYI, I'm waiting the following patch is merged.<br/> <a href="https://git.opendaylight.org/gerrit/#/c/35352/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/35352/</a><br/> (The master branch of the integration/distribution.git.)</p> <p>While i test in sandbox with latest Beryllium code. I find two test case failed in sandbox. </p> <p>There was failure in flowfilter and below is the link.<br/> <a href="https://jenkins.opendaylight.org/sandbox/job/vtn-csit-1node-manager-all-beryllium/2/robot/report/log.html#s1-s1-s3-t32" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/sandbox/job/vtn-csit-1node-manager-all-beryllium/2/robot/report/log.html#s1-s1-s3-t32</a></p> <p><a href="https://jenkins.opendaylight.org/sandbox/job/vtn-csit-1node-manager-all-beryllium/2/robot/report/log.html#s1-s2-s4-t23" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/sandbox/job/vtn-csit-1node-manager-all-beryllium/2/robot/report/log.html#s1-s2-s4-t23</a></p> <p>Analysis of the CSIT flow-filter test case failures</p> <p>The CSIT flow filter test case was failing in the CSIT setup for inet-src-action/inet-dst-action. </p> <p>Test scenarios Failed<br/> • VTN Flow filter<br/> • VBridge flow filter.</p> <p>Test scenarios Passed <br/> • VBridge Interface flow filter.</p> <p>Order of Test scenarios execution <br/> • VTN Flow filter<br/> • vBridge Flow filter<br/> • vBridge Interface flow filter.</p> <p>Initial analysis of the test case<br/> • Incorrect inputs values were provided to the inet-ser/inet-dst parameters<br/> • The ping was expected to be success as the input parameters were incorrect.<br/> • Dump flow analysis verification was incorrect as the parameters were incorrect.</p> <p>Changes done to the test case<br/> • Input parameters were modified to expected values for inet-ser/inet-dst parameters<br/> • The ping verification was modified for failure as the src/dst ip address were modified.<br/> • Dump flow analysis verification was modified to verify the correct set action in the edge port switch.</p> <p>Topology used<br/> h1,h2 &lt;--<del>&gt; s2 &lt;</del>-<del>&gt; s1 &lt;</del>-<del>&gt; s3 &lt;</del>--&gt; h3,h4</p> <p>Following are the steps followed in the CSIT test script.</p> <p>1. Create Tenant1<br/> 2. Create vBridge1<br/> 3. Create interfaces(if1 and if2) in vBridge1<br/> 4. Create portmapping for if1 and if2.<br/> 5. Ping h1 and h3 <br/> 6. Verify Ping success<br/> 7. Create vBridge2,<br/> 8. Create interfaces(if3and if4) in Vbridge 2,<br/> 9. Create portmapping for if3 and if4.<br/> 10. Ping h3 and h4 <br/> 11. Verify ping success<br/> 12. Add flow condition to match h1, h3 ip address<br/> 13. Add VTN flowfilter to set flow action (vtn-set-inet-src-action/ vtn-set-inet-dst-action)<br/> 14. Ping h1 and h3<br/> 15. Verify ping failure<br/> 16. Read the flow entries in mininet switch s3 to verify actions=mod_nw_src is set or not.<br/> 17. Add VBR flowfilter to set flow action (vtn-set-inet-src-action/ vtn-set-inet-dst-action)<br/> 18. Ping h1 and h3<br/> 19. Verify ping failure<br/> 20. Read the flow entries in mininet switch s3 to verify actions=mod_nw_src is set or not.<br/> 21. Add VBRIf flowfilter in If1 to set flow action (vtn-set-inet-src-action/ vtn-set-inet-dst-action)<br/> 22. Ping h1 and h3<br/> 23. Verify ping failure<br/> 24. Read the flow entries in mininet switch s3 to verify actions=mod_nw_src is set or not.<br/> 25. Delete flow condition<br/> 26. Delete vtn1</p> <p>In the above steps step 16 and step 20 failed but step 24 is success.<br/> After the above changes were done, It was observed that steps 16 and step 20 were still failing as before. The issue was the correct flows were not available in the switch when the flows were dumped in mininet switch. So the flow verification test case failed.<br/> To understand the issues, Karaf log was analyzed for VTN Manager module while the above steps were executed.<br/> Karaf log file name - all_flowfilter.log</p> <p>VTN flow filter - Test steps 13 – 16.</p> <p>a) Flow entry install request is issued in line 1521 at time stamp 2016-03-04 10:15:30,965 <br/> b) Flow entries remove request has be issued in line 1531 at time stamp 2016-03-04 10:15:31,034</p> <p>VBR Flow filter – Test steps 17 – 20<br/> a) Flow entry install request is issued in line 1905 at time stamp 2016-03-04 10:15:53,201<br/> b) Flow entries remove request has be issued in line 1934 at time stamp 2016-03-04 10:15:56,781</p> <p>In both the above cases its observed that the flow entries are removed after the below log message<br/> Ignore packet from internal node connector: openflow:2:3</p> <p>VBR Flow filter – Test steps 21 – 24<br/> a) Flow entry install request is issued in line 2112 at time stamp 2016-03-04 10:16:16,010<br/> b) Flow entries remove request has be issued in line 1934 at time stamp 2016-03-04 10:16:28,130</p> <p>In this case the flow verification is success as the flows are removed only during the removal of the vtn configuration during step 25, 26.</p> <p>From our analysis, the flows for VTN Flow filter and VBridge flowfilter are removed in VTN Manager before the test can verify the flows.</p> <p>Tried to verify this behavior by manually creating VTN, VBridge and VBridge IF flow filter. But not able to reproduce the same behavior. </p> <p>Manual test – vtn flow filter<br/> Manually Tested creating vtn flowfilter with the above scenario. <br/> Configured flow filters, and send packets which match the flow filters.<br/> After that, checked if flow entries for the flow filters are correctly created in mininet.<br/> As expected Flow entry created in Switch s3.</p> <p>In manualflow.log</p> <p>1) Created VTN, VBR, IF1 and IF2, portmapping IF1 and IF2. <br/> 2) Added Flowconditions and created VTN flowfilter with actions inetdrc and inetdst.<br/> 3) As expected Ping fails between hosts h1 and h3.<br/> 4) Checked dumpflows flow actions listed in mininet dumpflows.<br/> 5) In manualflow.log Line No: :2112, Flow entries has been created at time stamp 2016-03-07 04:42:27,685 <br/> 6) Flow entries is not deleted unless we manually remove flows. </p> <p>Request VTN Manager teams support to analyze the logs and inform if there is any issue in the CSIT test procedure. </p> <p>Attached log file ff_karaf.log</p> <p>Attachment ff_karaf.log has been added with description: Karaf log enabled TRACE</p> <p>The problem still happened in recent builds.</p> <p>For example:<br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-only-beryllium/216/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-only-beryllium/216/</a><br/> (Mar 6, 2016)</p> <p>The following CSIT for Beryllium to verify flowentries are working fine in below CSIT Beryllium jobs.<br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-only-beryllium/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-only-beryllium/</a><br/> <a href="https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-manager-all-beryllium/</a></p> <p>From 11/03/2016 we have checked the above CSIT jobs and there is no failures occurred which is related to Verify flowentries.</p> <p>The issue is fixed after the following patch is merged.<br/> <a href="https://git.opendaylight.org/gerrit/#/c/36039/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/36039/</a></p> Development External issue ID 5372 External issue URL Rank 0|i02tpr: