https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/OPNFLWPLUG-860 OpenFlowPlugin <p>Steps to Reproduce :<br/> ====================<br/> 1. Get Inventory Follower and Leader information</p> <p>2. Start Mininet Connect To Follower Node1</p> <p>3. Add Bulk Flow ( 1000 flows ) From Follower Node1</p> <p>4. Get Bulk Flows and Verify In Cluster config DS and verify all flows are installed</p> <p>5. Verify Default flows in switches when connected to Follower node. Here it shows zero flows in all 15 switches </p> <p>6. Verify Flows In Switch when connected to Follower Node . </p> <p>7. Flows will get installed when connected to Follower node.</p> <p>8. Verify Flows in switch when connected to Leader Node . </p> <p>9. Issue observed here . Flows will not get installed when connected to Leader Node.</p> <p>10. Issue is inconsistent . Flows will get installed into few switches and other switches will have zero flows.</p> <p>Attached failed log</p> <p>Operating System: All<br/> Platform: All</p> OPNFLWPLUG-860

Flows are not installed into switches when connected to Leader Node in cluster

Bug Resolved Done Luis Gomez Naveen kumar Thatipalli Wed, 1 Mar 2017 15:54:00 +0000 Mon, 27 Sep 2021 09:02:00 +0000 Tue, 21 Mar 2017 14:20:14 +0000 General 0 4 <p>Attachment flows_not_installed_connected_to_Leader_Node_SF142.mht has been added with description: Log_file for flows not installed into ovs switches when connected to leader node</p> <p>Verified with minimal flows .<br/> 500 flows with 30 switches . Total = 15000 flows.<br/> Four switches with zero flows and remaining switches each with 500 flows installed.</p> <p>This actually reminds me: <a href="https://bugs.opendaylight.org/show_bug.cgi?id=7770" class="external-link" target="_blank" rel="nofollow noopener">https://bugs.opendaylight.org/show_bug.cgi?id=7770</a> so maybe they are related. Also rising importance to critical as it really is.</p> <p>Actually we can track this issue in this test case:</p> <p><a href="https://jenkins.opendaylight.org/releng/view/CSIT-3node/job/openflowplugin-csit-3node-periodic-bulkomatic-clustering-daily-only-carbon/199/robot/openflowplugin-bulkomatic-clustering-daily.txt/Cluster%20Reconcilliation%20Multi%20DPN/Verify%20Flows%20In%20Switch%20Connected%20To%20Leader/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/CSIT-3node/job/openflowplugin-csit-3node-periodic-bulkomatic-clustering-daily-only-carbon/199/robot/openflowplugin-bulkomatic-clustering-daily.txt/Cluster%20Reconcilliation%20Multi%20DPN/Verify%20Flows%20In%20Switch%20Connected%20To%20Leader/</a></p> <p>It looks like the failure is often but not consistently happening.</p> <p>This issue can be reproduced by just adding 1 flow in follower and then connect 1 switch to leader of inventory config. It is similar but not the same as <a href="https://jira.opendaylight.org/browse/OPNFLWPLUG-853" title="Only 1 cluster instance is able to push table miss flow after initial switch connection" class="issue-link" data-issue-key="OPNFLWPLUG-853"><del>OPNFLWPLUG-853</del></a>. Also this issue is only in Carbon while 7770 is in both Carbon and Boron.</p> <p>This issue seems very inconsistent, when i tried today in my local cluster setup I could not really reproduce <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/sad.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Hi Luis,<br/> Reproduced the issue with less number of flows.</p> <p>15 switches <br/> Per switch : 1000 flows<br/> Total : 15000 flows.</p> <p>Given below log file :<br/> <a href="https://jenkins.opendaylight.org/sandbox/job/openflowplugin-csit-3node-periodic-bulkomatic-clustering-daily-only-boron/9/robot/report/log.html" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/sandbox/job/openflowplugin-csit-3node-periodic-bulkomatic-clustering-daily-only-boron/9/robot/report/log.html</a></p> <p>Some of the switches were populated with zero flows when connected to Leader Node.<br/> Log snippet:<br/> ============</p> <p>${flows} = <span class="error">&#91;u&#39;0&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;0&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;, u&#39;1000&#39;&#93;</span> <br/> Start / End / Elapsed: 20170309 10:07:27.749 / 20170309 10:07:27.750 / 00:00:00.001 <br/> 10:07:27.750 FAIL Keyword 'MininetKeywords.Check Flows In Mininet' failed after retrying for 1 minute 40 seconds. The last error was: 13000.0 != 15000.0</p> <p>Right, I tried with 15 switches and 15K flows and it reproduces consistently. It is important to note that with small amount of switches (3 switches) and flows (1K flows) the issue does not show up which means this is kind of perf/scale related issue.</p> <p>OK, after doing more test this issue also happens with single node so raising priority to blocker because this is an important regression. Also consider this bug could be the same as 7915.</p> <p>Also you can use the odl_tester.py in int/test repo to generate the flows as alterative of bulkomatic:</p> <p>python ./odl_tester.py --threads 5 --flows 10000 --fpr 100 --no-delete</p> <p>Reproduce is as simple as:</p> <p>1) Start 1 controller<br/> 2) Start mininet: sudo mn --controller 'remote,ip=192.168.0.1,port=6633' --topo linear,15<br/> 3) Push 10K flows: python ./odl_tester.py --threads 5 --flows 10000 --fpr 100 --no-delete<br/> 4) Stop and start mininet<br/> 5) check flows in switches: mininet&gt; dpctl dump-aggregate -O OpenFlow13</p> <p>2 additional notes:</p> <ul class="alternate" type="square"> <li>The issue does not show when I enable FRS as you mention.</li> <li>The regression was introduced after Boron SR2 and is present on both Boron and master branches.</li> </ul> <p>Anil's fix for FRM was merged, so can this one be closed now?</p> Development External issue ID 7884 External issue URL Rank 0|i033rb: