OpenDaylight JIRA

OpenDaylight JIRA https://jira.opendaylight.org This file is an XML representation of an issue en-us

8.20.10

820010

22-06-2022

[VTN-43] VTN Manager failed to forward packets occasionally. https://jira.opendaylight.org/browse/VTN-43 vtn <p>VTN Manager failed to forward packets occasionally.</p> <p>Due to this issue, ping tests of CSIT conducted by Integration Group fails occasionally.</p> <p> <a href="https://jenkins.opendaylight.org/integration/job/integration-master-csit-karaf-vtn-only/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/integration/job/integration-master-csit-karaf-vtn-only/</a><br/> <a href="https://jenkins.opendaylight.org/integration/job/integration-master-csit-karaf-vtn-all/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/integration/job/integration-master-csit-karaf-vtn-all/</a></p> <p>The issues has been discussed in the ML.<br/> <a href="https://lists.opendaylight.org/pipermail/vtn-dev/2014-September/000625.html" class="external-link" target="_blank" rel="nofollow noopener">https://lists.opendaylight.org/pipermail/vtn-dev/2014-September/000625.html</a></p> <p>Operating System: All<br/> Platform: All</p> VTN-43

VTN Manager failed to forward packets occasionally.

Bug Resolved Done Unassigned Hideyuki Tai Mon, 29 Sep 2014 01:23:52 +0000 Thu, 3 May 2018 15:45:10 +0000 Thu, 2 Oct 2014 00:45:56 +0000 unspecified VTN Manager 0 6 <p><span class="error">[How to reproduce the bug]</span></p> <p>The following procedure does NOT always reproduce the issue.<br/> To reproduce the issue, you have to do the procedure several times.</p> <p>Step 1. Run Karaf.</p> <p>Step 2. Install VNT Manager features.</p> <p> feature:install odl-vtn-manager-all odl-adsal-compatibility-all odl-openflowplugin-all</p> <p>Step 3. Run Mininet.</p> <p> sudo mn --controller=remote,ip=HOST --topo tree,2</p> <p>Step 4. Create a virtual network.</p> <p> I'm attaching the script for it.</p> <p> Or you can get the script from the following URL.<br/> <a href="https://gist.github.com/anonymous/b0ae9794fc97da5eb4c0" class="external-link" target="_blank" rel="nofollow noopener">https://gist.github.com/anonymous/b0ae9794fc97da5eb4c0</a></p> <p>Step 5. Try to send ping packets.</p> <p> h1 ping h3</p> <p> The above ping packets should be forwarded.<br/> But, due to the bug, it fails occasionally.</p> <p>If you don't face the issue, stop the Mininet and Karaf, and restart the above step 1, 3, and 5.<br/> (You don't have to do step 2 and 4.)</p> <p>Attachment vtn_test_script.sh.sh has been added with description: Script to create a virtual network. (See comment 1)</p> <p>As Shigeru said in the following mail, it seems that this issue is caused by asynchronous notification of data change event.</p> <p><a href="https://lists.opendaylight.org/pipermail/vtn-dev/2014-September/000639.html" class="external-link" target="_blank" rel="nofollow noopener">https://lists.opendaylight.org/pipermail/vtn-dev/2014-September/000639.html</a></p> <p>It seems that the OpenFlow plugin notifies MD-SAL of events of switches in the correct order.<br/> However, MD-SAL processes these notification in different threads.<br/> Because the kernel thread scheduler does not guarantee execution order of these threads, the order of these notification for MD-SAL application is unstable.</p> <p>Does it mean that this issue is caused by the OpenFlow plugin or MD-SAL?</p> <p>Hi Ed,</p> <p>It seems to me that VTN Neutron bundle has a bad impact in your setup.</p> <p>Therefore, could you install VTN features as follows instead of odl-vtn-manager-all?</p> <p>feature:install odl-vtn-manager-java-api odl-vtn-manager-northbound<br/> feature:install odl-adsal-compatibility-all odl-openflowplugin-all</p> <p>It does not install odl-vtn-manager-neutron feature.</p> <p>So tonight I tested this bug more in detail and it looks like the root cause = <a href="https://jira.opendaylight.org/browse/CONTROLLER-904" title="Randomly missing nodes in Inventory" class="issue-link" data-issue-key="CONTROLLER-904"><del>CONTROLLER-904</del></a> = Missing node connectors in AD-SAL Switch manager.</p> <p>BR/Luis</p> <p>The details of he test:</p> <p>VTN gets the node connector s2-eth2 when node registers:</p> <p>2014-09-30 08:16:28,835 | INFO | notification-113 | VTNManagerImpl | 218 - org.opendaylight.vtn.manager.implementation - 0.2.0.SNAPSHOT | default: addPort: New port: port=OF|2@OF|00:00:00:00:00:00:00:02, prop=PortProperty<span class="error">[name=s2-eth2,cost=1000,enabled]</span></p> <p>However adsal switch manager does not show this port:<br/> {<br/> "nodeConnectorProperties": [<br/> {<br/> "properties": {<br/> "name": </p> { "value": "s2" } <p>,<br/> "state": </p> { "value": 1 } <p>,<br/> "config": </p> { "value": 1 } <p> },<br/> "nodeconnector": {<br/> "node": </p> { "id": "00:00:00:00:00:00:00:02", "type": "OF" } <p>,<br/> "id": "0",<br/> "type": "SW"<br/> }<br/> },<br/> {<br/> "properties": {<br/> "name": </p> { "value": "s2-eth1" } <p>,<br/> "state": </p> { "value": 1 } <p>,<br/> "config": </p> { "value": 1 } <p>,<br/> "bandwidth": </p> { "value": 10000000000 } <p> },<br/> "nodeconnector": {<br/> "node": </p> { "id": "00:00:00:00:00:00:00:02", "type": "OF" } <p>,<br/> "id": "1",<br/> "type": "OF"<br/> }<br/> },<br/> {<br/> "properties": {<br/> "name": </p> { "value": "s2-eth3" } <p>,<br/> "state": </p> { "value": 1 } <p>,<br/> "config": </p> { "value": 1 } <p>,<br/> "bandwidth": </p> { "value": 10000000000 } <p> },<br/> "nodeconnector": {<br/> "node": </p> { "id": "00:00:00:00:00:00:00:02", "type": "OF" } <p>,<br/> "id": "3",<br/> "type": "OF"<br/> }<br/> }<br/> ]<br/> }</p> <p>This makes VInterface if3 to go DOWN:</p> <p>2014-09-30 08:16:35,241 | INFO | Thread: default | VTNManagerImpl | 218 - org.opendaylight.vtn.manager.implementation - 0.2.0.SNAPSHOT | default:vBridge-IF:Tenant1.vBridge2.if3: Virtual interface added: VInterface<span class="error">[name=if3,enabled,state=UNKNOWN,entityState=UNKNOWN]</span></p> <p>2014-09-30 08:16:35,294 | INFO | Thread: default | VTNManagerImpl | 218 - org.opendaylight.vtn.manager.implementation - 0.2.0.SNAPSHOT | default:vBridge-IF:Tenant1.vBridge2.if3: Port mapping added: PortMap[config=PortMapConfig[node=OF|00:00:00:00:00:00:00:02,port=SwitchPort<span class="error">[name=s2-eth2]</span>,vlan=0]]</p> <p>2014-09-30 08:16:35,294 | INFO | Thread: default | VTNManagerImpl | 218 - org.opendaylight.vtn.manager.implementation - 0.2.0.SNAPSHOT | default:vBridge-IF:Tenant1.vBridge2.if3: Virtual interface changed: VInterface<span class="error">[name=if3,enabled,state=DOWN,entityState=UNKNOWN]</span></p> <p>And finally ping fails in the test:</p> <p>Ping h2 to h4 :: Ping h2 to h4, verify no packet loss | FAIL |<br/> 'PING 10.0.0.4 (10.0.0.4) 56(84) bytes of data.<br/> From 10.0.0.2 icmp_seq=1 Destination Host Unreachable<br/> From 10.0.0.2 icmp_seq=2 Destination Host Unreachable<br/> From 10.0.0.2 icmp_seq=3 Destination Host Unreachable</p> <p>— 10.0.0.4 ping statistics —<br/> 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2015ms</p> <p>BR/Luis</p> <p>Fix here:</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/11705" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/11705</a></p> <p>So far it has succeeded 8 out of 8 tries for me <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>(In reply to Ed Warnicke from comment #7)<br/> > Fix here:<br/> > <br/> > <a href="https://git.opendaylight.org/gerrit/#/c/11705" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/11705</a><br/> > <br/> > <br/> > So far it has succeeded 8 out of 8 tries for me <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Hi Ed,</p> <p>Thank you for submitting the patch.</p> <p>I've tested the patch, and it has succeeded 10 out 10 tries in my setup.<br/> I think this patch fixes the bug.</p> <p>Per Hideyuki's comments, this is fixed for him.</p> Development External issue ID 2107 External issue URL Rank 0|i02t8f: