OpenDaylight JIRA

OpenDaylight JIRA https://jira.opendaylight.org This file is an XML representation of an issue en-us

8.20.10

820010

22-06-2022

[CONTROLLER-232] Packet loss on first time ping test https://jira.opendaylight.org/browse/CONTROLLER-232 controller <p>After setting up mininet with a normal topology like tree,2. We do the ping test and first time, there is always some packet loss:</p> <p>mininet@mininet-vm:~$ sudo mn --controller=remote,ip=192.168.60.180 --topo tree,2</p> <p>mininet> pingall</p> <ul> <li> <ul> <li> <ul> <li>Ping: testing ping reachability<br/> h1 -> X h3 h4 <br/> h2 -> h1 h3 h4 <br/> h3 -> h1 h2 X <br/> h4 -> h1 h2 h3 </li> <li>Results: 16% dropped (10/12 received)<br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 h3 h4 <br/> h2 -> h1 h3 h4 <br/> h3 -> h1 h2 h4 <br/> h4 -> h1 h2 h3 </li> <li>Results: 0% dropped (12/12 received)</li> </ul> </li> </ul> </li> </ul> <p>Operating System: All<br/> Platform: All</p> CONTROLLER-232

Packet loss on first time ping test

Bug Resolved Done Luis Gomez Luis Gomez Tue, 25 Mar 2014 17:13:01 +0000 Tue, 25 Jul 2023 08:23:17 +0000 Tue, 5 May 2015 15:17:09 +0000 adsal 0 6 <p>Should be fixed in the stable release.</p> <p>Hello, Can we have an update on this activity? Thanks!</p> <p>From: Flavio Fernandes <ffernand@redhat.com><br/> Subject: Re: ODL <a href="https://jira.opendaylight.org/browse/CONTROLLER-232" title="Packet loss on first time ping test" class="issue-link" data-issue-key="CONTROLLER-232"><del>CONTROLLER-232</del></a><br/> Date: April 28, 2014 at 6:29:02 PM EDT<br/> To: Luis Gomez <ecelgp@gmail.com></p> <p>Hi Luis,</p> <p>I spent most of today trying to reproduce the problem using the vms you pointed me to, and<br/> the latest integration-base build. I failed miserably (see below). :/ What I really would like to see<br/> is the log file when it does not work. Can you please set the 3 logs to trace and reproduce the<br/> problem and send them to me?</p> <p>W/out those, I’m pretty unsure on whether my assumption for the fix is going to do anything.</p> <p>Sorry,</p> <p>— flavio</p> <p>mininet@mininet-vm:~/controller-base/opendaylight\> more version.properties <br/> org.opendaylight.controller.version = 0.1<br/> org.opendaylight.controller.build.scm.version = 61bad4207cc91cd14d8d38a255ad6549c20ff54e<br/> org.opendaylight.controller.build.user = jenkins-controller<br/> org.opendaylight.controller.build.workspace = **********<br/> org.opendaylight.controller.build.timestamp = 1398697205857<br/> org.opendaylight.controller.build.machine = **********</p> <p>—</p> <p>mininet@mininet-vm:~/controller-base/opendaylight\> ./run.sh -of13<br/> setLogLevel org.opendaylight.controller.samples.simpleforwarding.internal TRACE<br/> setLogLevel org.opendaylight.controller.hosttracker.internal TRACE</p> <p> setLogLevel org.opendaylight.controller.routing TRACE</p> <p>...</p> <p>mininet@mininet-vm:~\> ~/integration/vm/scripts/start_mininet_of13.sh</p> <ul> <li> <ul> <li> <ul> <li>Creating network</li> <li>Adding controller</li> <li>Adding hosts:<br/> h1 h2 h3 h4 </li> <li>Adding switches:<br/> have protcol params!<br/> s1 have protcol params!<br/> s2 have protcol params!<br/> s3 </li> <li>Adding links:<br/> (h1, s2) (h2, s2) (h3, s3) (h4, s3) (s1, s2) (s1, s3) </li> <li>Configuring hosts<br/> h1 h2 h3 h4 </li> <li>Starting controller</li> <li>Starting 3 switches<br/> s1 OVSswitch opts: protocols=OpenFlow13<br/> s2 OVSswitch opts: protocols=OpenFlow13<br/> s3 OVSswitch opts: protocols=OpenFlow13</li> </ul> </li> </ul> </li> </ul> <ul> <li> <ul> <li> <ul> <li>Starting CLI:<br/> mininet> <br/> mininet> <br/> mininet> <br/> mininet> <br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 h3 h4 <br/> h2 -> h1 h3 h4 <br/> h3 -> h1 h2 h4 <br/> h4 -> h1 h2 h3 </li> <li>Results: 0% dropped (12/12 received)<br/> mininet> </li> </ul> </li> </ul> </li> </ul> <p>On Apr 28, 2014, at 11:57 AM, Luis Gomez <ecelgp@gmail.com> wrote:</p> <p>Hi Flavio,</p> <p>Yes I use same VM to run controller + mininet and I use latest base edition. With this setup I see same issue with old 1.0 plugin and new 1.3 plugin.</p> <p>Now after your mail I have tried with different setup: controller VM + mininet VM and you are right I do not see the problem with old OF1.0 plugin (at least with tree,2 and tree,3 topologies) but still I see it with OF1.3 plugin.</p> <p>BR/Luis</p> <p>On Apr 28, 2014, at 5:32 AM, Flavio Fernandes <ffernand@redhat.com> wrote:</p> <p>Hello Luis!</p> <p>Thanks for the kind words. The way I have been testing so far has been by building/running ODL straight out<br/> of my laptop: Mac OS X. I tend to use the latest and greatest 1.2-snapshot codebase.</p> <p>Also, I have mininet running from a virtual machine. I downloaded mininet VM from the mininet’s <br/> website: mininet-2.1.0-130919-ubuntu-13.04-server-amd64-ovf. To run the VM, I use vmware fusion. <br/> I gave the VM 1 Gb and 1 processor. Using that setup, I was able to reproduce the original issues. What I think happened is that the race is now slightly tighter and the difference in our setups is enough to run differently.</p> <p>Thanks for the link! We should add a reference to it in <a href="https://jira.opendaylight.org/browse/CONTROLLER-232" title="Packet loss on first time ping test" class="issue-link" data-issue-key="CONTROLLER-232"><del>CONTROLLER-232</del></a>. Can you take care of that, please?</p> <p>I’m grabbing the vms from the link you provided. I should have done that sooner, huh? I’m hoping that with those,<br/> I should be able to see it. Questions on that:<br/> 1) do you see the bug when mininet and the controller are in <em>different</em> vms?<br/> 2) from your example, it seems that you have ODL and mininet co-located, and you are using OF1.0. So all<br/> you use to reproduce this bug is 1 VM?<br/> 3) You are running the ‘base’ distribution of ODL, right?</p> <p>Thanks,</p> <p>— flavio</p> <p>Hi Flaviof:</p> <p>I just played with mininet. After setting the logs to TRACE:</p> <p> <logger name="org.opendaylight.controller.hosttracker" level="TRACE"/><br/> <logger name="org.opendaylight.controller.routing" level="TRACE"/><br/> <logger name="org.opendaylight.controller.forwardingrulesmanager" level="TRACE"/></p> <p>I get the following outputs: (It looks better compared to the last time - Luis filed it) Let me know if you need opendaylight.log for the details.</p> <p>mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633' --switch ovsk,protocols=OpenFlow13</p> <ul> <li> <ul> <li> <ul> <li>Creating network</li> <li>Adding controller</li> <li>Adding hosts:<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Adding switches:<br/> have protcol params!<br/> s1 have protcol params!<br/> s2 have protcol params!<br/> s3 have protcol params!<br/> s4 have protcol params!<br/> s5 have protcol params!<br/> s6 have protcol params!<br/> s7 </li> <li>Adding links:<br/> (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7) </li> <li>Configuring hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Starting controller</li> <li>Starting 7 switches<br/> s1 OVSswitch opts: protocols=OpenFlow13<br/> s2 OVSswitch opts: protocols=OpenFlow13<br/> s3 OVSswitch opts: protocols=OpenFlow13<br/> s4 OVSswitch opts: protocols=OpenFlow13<br/> s5 OVSswitch opts: protocols=OpenFlow13<br/> s6 OVSswitch opts: protocols=OpenFlow13<br/> s7 OVSswitch opts: protocols=OpenFlow13</li> </ul> </li> </ul> </li> </ul> <ul> <li> <ul> <li> <ul> <li>Starting CLI:<br/> mininet> <br/> mininet> <br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 h3 h4 h5 h6 h7 h8 <br/> h2 -> h1 h3 h4 h5 h6 h7 h8 <br/> h3 -> h1 h2 h4 h5 h6 h7 h8 <br/> h4 -> h1 h2 h3 h5 h6 h7 h8 <br/> h5 -> h1 h2 h3 h4 h6 h7 h8 <br/> h6 -> h1 h2 h3 h4 h5 h7 h8 <br/> h7 -> h1 h2 h3 h4 h5 h6 h8 <br/> h8 -> h1 h2 h3 h4 h5 h6 h7 </li> <li>Results: 0% dropped (56/56 received)<br/> mininet> <br/> mininet> <br/> mininet> <br/> mininet> quit</li> <li>Stopping 7 switches<br/> s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...</li> <li>Stopping 8 hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Stopping 1 controllers<br/> c0 </li> <li>Done<br/> completed in 39.013 seconds<br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633'</li> <li>Creating network</li> <li>Adding controller</li> <li>Adding hosts:<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Adding switches:<br/> s1 s2 s3 s4 s5 s6 s7 </li> <li>Adding links:<br/> (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7) </li> <li>Configuring hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Starting controller</li> <li>Starting 7 switches<br/> s1 OVSswitch opts: <br/> s2 OVSswitch opts: <br/> s3 OVSswitch opts: <br/> s4 OVSswitch opts: <br/> s5 OVSswitch opts: <br/> s6 OVSswitch opts: <br/> s7 OVSswitch opts: </li> </ul> </li> </ul> </li> </ul> <ul> <li> <ul> <li> <ul> <li>Starting CLI:<br/> mininet> <br/> mininet> <br/> mininet> <br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 h3 h4 h5 h6 h7 X <br/> h2 -> h1 h3 h4 h5 h6 h7 h8 <br/> h3 -> h1 h2 h4 h5 h6 h7 h8 <br/> h4 -> h1 h2 h3 h5 h6 h7 h8 <br/> h5 -> h1 h2 h3 h4 h6 h7 h8 <br/> h6 -> h1 h2 h3 h4 h5 h7 h8 <br/> h7 -> h1 h2 h3 h4 h5 h6 h8 <br/> h8 -> h1 h2 h3 h4 h5 h6 h7 </li> <li>Results: 1% dropped (55/56 received)<br/> mininet> <br/> mininet> <br/> mininet> quit</li> <li>Stopping 7 switches<br/> s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...</li> <li>Stopping 8 hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Stopping 1 controllers<br/> c0 </li> <li>Done<br/> completed in 33.244 seconds<br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> sudo mn --topo tree,4 --controller 'remote,ip=127.0.0.1,port=6633' --switch ovsk,protocols=OpenFlow13</li> <li>Creating network</li> <li>Adding controller</li> <li>Adding hosts:<br/> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 </li> <li>Adding switches:<br/> have protcol params!<br/> s1 have protcol params!<br/> s2 have protcol params!<br/> s3 have protcol params!<br/> s4 have protcol params!<br/> s5 have protcol params!<br/> s6 have protcol params!<br/> s7 have protcol params!<br/> s8 have protcol params!<br/> s9 have protcol params!<br/> s10 have protcol params!<br/> s11 have protcol params!<br/> s12 have protcol params!<br/> s13 have protcol params!<br/> s14 have protcol params!<br/> s15 </li> <li>Adding links:<br/> (h1, s4) (h2, s4) (h3, s5) (h4, s5) (h5, s7) (h6, s7) (h7, s8) (h8, s8) (h9, s11) (h10, s11) (h11, s12) (h12, s12) (h13, s14) (h14, s14) (h15, s15) (h16, s15) (s1, s2) (s1, s9) (s2, s3) (s2, s6) (s3, s4) (s3, s5) (s6, s7) (s6, s8) (s9, s10) (s9, s13) (s10, s11) (s10, s12) (s13, s14) (s13, s15) </li> <li>Configuring hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 </li> <li>Starting controller</li> <li>Starting 15 switches<br/> s1 OVSswitch opts: protocols=OpenFlow13<br/> s2 OVSswitch opts: protocols=OpenFlow13<br/> s3 OVSswitch opts: protocols=OpenFlow13<br/> s4 OVSswitch opts: protocols=OpenFlow13<br/> s5 OVSswitch opts: protocols=OpenFlow13<br/> s6 OVSswitch opts: protocols=OpenFlow13<br/> s7 OVSswitch opts: protocols=OpenFlow13<br/> s8 OVSswitch opts: protocols=OpenFlow13<br/> s9 OVSswitch opts: protocols=OpenFlow13<br/> s10 OVSswitch opts: protocols=OpenFlow13<br/> s11 OVSswitch opts: protocols=OpenFlow13<br/> s12 OVSswitch opts: protocols=OpenFlow13<br/> s13 OVSswitch opts: protocols=OpenFlow13<br/> s14 OVSswitch opts: protocols=OpenFlow13<br/> s15 OVSswitch opts: protocols=OpenFlow13</li> </ul> </li> </ul> </li> </ul> <ul> <li> <ul> <li> <ul> <li>Starting CLI:<br/> mininet> <br/> mininet> <br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 X h14 h15 h16 <br/> h2 -> h1 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h3 -> h1 h2 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h4 -> h1 h2 h3 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h5 -> h1 h2 h3 h4 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h6 -> h1 h2 h3 h4 h5 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h7 -> h1 h2 h3 h4 h5 h6 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h8 -> h1 h2 h3 h4 h5 h6 h7 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h9 -> h1 h2 h3 h4 h5 h6 h7 h8 h10 h11 h12 h13 h14 h15 h16 <br/> h10 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h11 h12 h13 h14 h15 h16 <br/> h11 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h12 h13 h14 h15 h16 <br/> h12 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h13 h14 h15 h16 <br/> h13 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h14 h15 h16 <br/> h14 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h15 h16 <br/> h15 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h16 <br/> h16 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 </li> <li>Results: 0% dropped (239/240 received)<br/> mininet> <br/> mininet> <br/> mininet> quit</li> <li>Stopping 15 switches<br/> s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...s8 ...s9 ...s10 ...s11 ...s12 ...s13 ...s14 ...s15 ...</li> <li>Stopping 16 hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 </li> <li>Stopping 1 controllers<br/> c0 </li> <li>Done<br/> completed in 71.186 seconds<br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> sudo mn --topo tree,4 --controller 'remote,ip=127.0.0.1,port=6633'</li> <li>Creating network</li> <li>Adding controller</li> <li>Adding hosts:<br/> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 </li> <li>Adding switches:<br/> s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s12 s13 s14 s15 </li> <li>Adding links:<br/> (h1, s4) (h2, s4) (h3, s5) (h4, s5) (h5, s7) (h6, s7) (h7, s8) (h8, s8) (h9, s11) (h10, s11) (h11, s12) (h12, s12) (h13, s14) (h14, s14) (h15, s15) (h16, s15) (s1, s2) (s1, s9) (s2, s3) (s2, s6) (s3, s4) (s3, s5) (s6, s7) (s6, s8) (s9, s10) (s9, s13) (s10, s11) (s10, s12) (s13, s14) (s13, s15) </li> <li>Configuring hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 </li> <li>Starting controller</li> <li>Starting 15 switches<br/> s1 OVSswitch opts: <br/> s2 OVSswitch opts: <br/> s3 OVSswitch opts: <br/> s4 OVSswitch opts: <br/> s5 OVSswitch opts: <br/> s6 OVSswitch opts: <br/> s7 OVSswitch opts: <br/> s8 OVSswitch opts: <br/> s9 OVSswitch opts: <br/> s10 OVSswitch opts: <br/> s11 OVSswitch opts: <br/> s12 OVSswitch opts: <br/> s13 OVSswitch opts: <br/> s14 OVSswitch opts: <br/> s15 OVSswitch opts: </li> </ul> </li> </ul> </li> </ul> <ul> <li> <ul> <li> <ul> <li>Starting CLI:<br/> mininet> <br/> mininet> <br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 X <br/> h2 -> h1 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h3 -> h1 h2 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h4 -> h1 h2 h3 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h5 -> h1 h2 h3 h4 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h6 -> h1 h2 h3 h4 h5 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h7 -> h1 h2 h3 h4 h5 h6 h8 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h8 -> h1 h2 h3 h4 h5 h6 h7 h9 h10 h11 h12 h13 h14 h15 h16 <br/> h9 -> h1 h2 h3 h4 h5 h6 h7 h8 h10 h11 h12 h13 h14 h15 h16 <br/> h10 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h11 h12 h13 h14 h15 h16 <br/> h11 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h12 h13 h14 h15 h16 <br/> h12 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h13 h14 h15 h16 <br/> h13 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h14 h15 h16 <br/> h14 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h15 h16 <br/> h15 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h16 <br/> h16 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 </li> <li>Results: 0% dropped (239/240 received)<br/> mininet> <br/> mininet> <br/> mininet> quit</li> <li>Stopping 15 switches<br/> s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...s8 ...s9 ...s10 ...s11 ...s12 ...s13 ...s14 ...s15 ...</li> <li>Stopping 16 hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16 </li> <li>Stopping 1 controllers<br/> c0 </li> <li>Done<br/> completed in 62.396 seconds<br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633'*** Creating network</li> <li>Adding controller</li> <li>Adding hosts:<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Adding switches:<br/> s1 s2 s3 s4 s5 s6 s7 </li> <li>Adding links:<br/> (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7) </li> <li>Configuring hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Starting controller</li> <li>Starting 7 switches<br/> s1 OVSswitch opts: <br/> s2 OVSswitch opts: <br/> s3 OVSswitch opts: <br/> s4 OVSswitch opts: <br/> s5 OVSswitch opts: <br/> s6 OVSswitch opts: <br/> s7 OVSswitch opts: </li> </ul> </li> </ul> </li> </ul> <ul> <li> <ul> <li> <ul> <li>Starting CLI:<br/> mininet> <br/> mininet> <br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 h3 X h5 h6 h7 h8 <br/> h2 -> h1 h3 h4 h5 h6 h7 h8 <br/> h3 -> h1 h2 h4 h5 h6 h7 h8 <br/> h4 -> h1 h2 h3 h5 h6 h7 h8 <br/> h5 -> h1 h2 h3 h4 h6 h7 h8 <br/> h6 -> h1 h2 h3 h4 h5 h7 h8 <br/> h7 -> h1 h2 h3 h4 h5 h6 h8 <br/> h8 -> h1 h2 h3 h4 h5 h6 h7 </li> <li>Results: 1% dropped (55/56 received)<br/> mininet> <br/> mininet> <br/> mininet> quit</li> <li>Stopping 7 switches<br/> s1 ..s2 .<br/> ..s3 ...s4 ...s5 ...s6 ...s7 ...</li> <li>Stopping 8 hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Stopping 1 controllers<br/> c0 </li> <li>Done<br/> completed in 46.181 seconds<br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> <br/> mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633' --switch ovsk,protocols=OpenFlow13</li> <li>Creating network</li> <li>Adding controller</li> <li>Adding hosts:<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Adding switches:<br/> have protcol params!<br/> s1 have protcol params!<br/> s2 have protcol params!<br/> s3 have protcol params!<br/> s4 have protcol params!<br/> s5 have protcol params!<br/> s6 have protcol params!<br/> s7 </li> <li>Adding links:<br/> (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7) </li> <li>Configuring hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Starting controller</li> <li>Starting 7 switches<br/> s1 OVSswitch opts: protocols=OpenFlow13<br/> s2 OVSswitch opts: protocols=OpenFlow13<br/> s3 OVSswitch opts: protocols=OpenFlow13<br/> s4 OVSswitch opts: protocols=OpenFlow13<br/> s5 OVSswitch opts: protocols=OpenFlow13<br/> s6 OVSswitch opts: protocols=OpenFlow13<br/> s7 OVSswitch opts: protocols=OpenFlow13</li> </ul> </li> </ul> </li> </ul> <ul> <li> <ul> <li> <ul> <li>Starting CLI:<br/> mininet> <br/> mininet> pingall</li> <li>Ping: testing ping reachability<br/> h1 -> h2 X h4 h5 h6 h7 h8 <br/> h2 -> h1 h3 h4 h5 h6 h7 h8 <br/> h3 -> h1 h2 h4 h5 h6 h7 h8 <br/> h4 -> h1 h2 h3 h5 h6 h7 h8 <br/> h5 -> h1 h2 h3 h4 h6 h7 h8 <br/> h6 -> h1 h2 h3 h4 h5 h7 h8 <br/> h7 -> h1 h2 h3 h4 h5 h6 h8 <br/> h8 -> h1 h2 h3 h4 h5 h6 h7 </li> <li>Results: 1% dropped (55/56 received)<br/> mininet> <br/> mininet> <br/> mininet> quit</li> <li>Stopping 7 switches<br/> s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...</li> <li>Stopping 8 hosts<br/> h1 h2 h3 h4 h5 h6 h7 h8 </li> <li>Stopping 1 controllers<br/> c0 </li> <li>Done<br/> completed in 71.230 seconds<br/> mininet@mininet-vm:~\> </li> </ul> </li> </ul> </li> </ul> <p>Thanks,<br/> Madhusudhan</p> <p>OK, the results of my test:</p> <p>1) With 2 CPU Cores, it does not work at all with OF10 or OF13 plugins with even a small topology of tree,1 (1 switch), here is the debug log:</p> <p>2014-04-29 16:48:55.098 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.1, MAC 2e02c8cff6a0, HostNodeConnector <span class="error">[nodeConnector=OF|1@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]</span><br/> 2014-04-29 16:48:55.098 PDT <span class="error">[Thread-84]</span> DEBUG o.o.c.h.internal.HostTracker - New Host Learned: MAC: 2e02c8cff6a0 IP: 10.0.0.1<br/> 2014-04-29 16:48:55.099 PDT <span class="error">[Thread-84]</span> DEBUG o.o.c.h.internal.HostTracker - Notifying Applications for Host 10.0.0.1 Being Added<br/> 2014-04-29 16:48:55.098 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42<br/> 2014-04-29 16:48:55.099 PDT <span class="error">[Thread-84]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Adding policyMatch(DIP = 10.0.0.1) Action(outPort= OF|1@OF|00:00:00:00:00:00:00:01) to node OF|00:00:00:00:00:00:00:01<br/> 2014-04-29 16:48:55.099 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector <span class="error">[nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]</span><br/> 2014-04-29 16:48:55.099 PDT <span class="error">[Thread-84]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Inside installPerHostRules<br/> 2014-04-29 16:48:55.100 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42<br/> 2014-04-29 16:48:55.101 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Host Added to ARPPending List, IP: /10.0.0.2<br/> 2014-04-29 16:48:55.101 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - hostFind(): Host Not Found for IP: IP=<span class="error">[10.0.0.2]</span>, Inititated Host Discovery ...<br/> 2014-04-29 16:48:55.101 PDT <span class="error">[Thread-84]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Successfully installed policy FlowEntry[flowName = <span class="error">[10.0.0.1/32 on N OF|00:00:00:00:00:00:00:01]</span>, groupName = 10.0.0.1/32, node = OF|00:00:00:00:00:00:00:01, flow = Flow[match = Match [fields=</p> {NW_DST=NW_DST(10.0.0.1), DL_TYPE=DL_TYPE(2048)} <p>, matches=2176], actions = [SET_DL_DST<span class="error">[address = 2e02c8cff6a0]</span>, OUTPUT<span class="error">[OF|1@OF|00:00:00:00:00:00:00:01]</span>], priority = 1, id = 0, idleTimeout = 0, hardTimeout = 0]] on switch OF|00:00:00:00:00:00:00:01<br/> 2014-04-29 16:48:55.101 PDT <span class="error">[Thread-84]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Leaving installPerHostRules<br/> 2014-04-29 16:48:55.102 PDT <span class="error">[Thread-84]</span> DEBUG o.o.c.h.internal.HostTracker - Notifying Topology Manager for Host 10.0.0.1 Being Added<br/> 2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 98<br/> 2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Handle punted IP packet: Ethernet: <span class="error">[DestinationMACAddress: de700b56a32c, SourceMACAddress: 2e02c8cff6a0, EtherType: 0800]</span><br/> 2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Host Added to ARPPending List, IP: /10.0.0.2<br/> 2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - hostFind(): Host Not Found for IP: IP=<span class="error">[10.0.0.2]</span>, Inititated Host Discovery ...<br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.h.internal.HostTracker - New Host Learned: MAC: de700b56a32c IP: 10.0.0.2<br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.h.internal.HostTracker - Host Removed from ARPPending List, IP: IP=<span class="error">[10.0.0.2]</span><br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.h.internal.HostTracker - Notifying Applications for Host 10.0.0.2 Being Added<br/> 2014-04-29 16:48:55.104 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector <span class="error">[nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]</span><br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Adding policyMatch(DIP = 10.0.0.2) Action(outPort= OF|2@OF|00:00:00:00:00:00:00:01) to node OF|00:00:00:00:00:00:00:01<br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Inside installPerHostRules<br/> 2014-04-29 16:48:55.105 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host<br/> 2014-04-29 16:48:55.105 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42<br/> 2014-04-29 16:48:55.105 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector <span class="error">[nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]</span><br/> 2014-04-29 16:48:55.106 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host<br/> 2014-04-29 16:48:55.106 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Successfully installed policy FlowEntry[flowName = <span class="error">[10.0.0.2/32 on N OF|00:00:00:00:00:00:00:01]</span>, groupName = 10.0.0.2/32, node = OF|00:00:00:00:00:00:00:01, flow = Flow[match = Match [fields=</p> {NW_DST=NW_DST(10.0.0.2), DL_TYPE=DL_TYPE(2048)} <p>, matches=2176], actions = [SET_DL_DST<span class="error">[address = de700b56a32c]</span>, OUTPUT<span class="error">[OF|2@OF|00:00:00:00:00:00:00:01]</span>], priority = 1, id = 0, idleTimeout = 0, hardTimeout = 0]] on switch OF|00:00:00:00:00:00:00:01<br/> 2014-04-29 16:48:55.106 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Leaving installPerHostRules<br/> 2014-04-29 16:48:55.106 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42<br/> 2014-04-29 16:48:55.106 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.h.internal.HostTracker - Notifying Topology Manager for Host 10.0.0.2 Being Added<br/> 2014-04-29 16:48:55.107 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector <span class="error">[nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]</span><br/> 2014-04-29 16:48:55.107 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host<br/> 2014-04-29 16:48:55.110 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42<br/> 2014-04-29 16:49:05.099 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector <span class="error">[nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]</span><br/> 2014-04-29 16:49:05.100 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host</p> <p>2) With 3+ CPU cores, OF10 plugin works good until tree, 6 (63 switches, the the maximum I have tried), with OF13 the maximum is tree,4 (15 switches), after that I see packet loss</p> <p>From: Flavio Fernandes <ffernand@redhat.com><br/> Subject: Re: ODL <a href="https://jira.opendaylight.org/browse/CONTROLLER-232" title="Packet loss on first time ping test" class="issue-link" data-issue-key="CONTROLLER-232"><del>CONTROLLER-232</del></a><br/> Date: April 30, 2014 at 5:54:09 AM EDT<br/> To: Luis Gomez <ecelgp@gmail.com>, Colin Dixon <colin@colindixon.com><br/> Cc: Madhusudhan Kandadai <madhusudhan.opendaylight@yahoo.com>, Giovanni Meo <gmeo@cisco.com></p> <p>Hi Luis,</p> <p>Thank you for your log! So the fact that the packet is getting lost is indeed due to hostTracker not<br/> being done learning about the host before the exception packet is processed. That is good, because<br/> we can add some mechanism for addressing that, should we deem it to be important enough.</p> <p>Colin: if you look at the highlighted lines (below) and the code, it is very easy to see what happens.</p> <p>A time '2014-04-29 16:48:55.102 PDT’, the exception packet comes to SimpleForwardingImpl<br/> but it ends up getting dropped due to hostFind(dIP) call returning null. The interesting part is that immediately after, not even a second later, hostTracker learns about the host. <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Assuming we want to solve this race, all we need is a way to hang on to the packet for a short while. That is<br/> my plan of action, assuming you agree. My gut tells me we should fix it, because 1) it is pretty simple stuff and 2) we should not be worrying about how many cores are in use. I would like to talk to you about the implementation, still.</p> <p>Thanks,</p> <p>— flavio</p> <p>private void handlePuntedIPPacket(IPv4 pkt, NodeConnector incomingNodeConnector) {<br/> ...<br/> HostNodeConnector destHost = hostTracker.hostFind(dIP);<br/> /*</p> <ul> <li>In cases when incoming and outgoing connectors are in the same node, there is no need</li> <li>to verify that there is a route. Because of that, we will only need routing.getRoute()</li> <li>if we know that src and dst nodes are different.<br/> */<br/> if (destHost != null<br/> … </li> </ul> <p>===</p> <p><a href="https://www.dropbox.com/s/hu86tbj1qcbsfh8/Screenshot%202014-04-30%2005.32.06.png" class="external-link" target="_blank" rel="nofollow noopener">https://www.dropbox.com/s/hu86tbj1qcbsfh8/Screenshot%202014-04-30%2005.32.06.png</a></p> <p>2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 98 <br/> 2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> TRACE o.o.c.s.s.i.SimpleForwardingImpl - Handle punted IP packet: Ethernet: [DestinationMACAddress: de700b56a32c, SourceMACAdd\<br/> ress: 2e02c8cff6a0, EtherType: 0800] <br/> 2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - Host Added to ARPPending List, IP: /10.0.0.2 <br/> 2014-04-29 16:48:55.102 PDT <span class="error">[SwitchEvent Thread]</span> DEBUG o.o.c.h.internal.HostTracker - hostFind(): Host Not Found for IP: IP=<span class="error">[10.0.0.2]</span>, Inititated Host Discovery ... <br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.h.internal.HostTracker - New Host Learned: MAC: de700b56a32c IP: 10.0.0.2 <br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.h.internal.HostTracker - Host Removed from ARPPending List, IP: IP=<span class="error">[10.0.0.2]</span> <br/> 2014-04-29 16:48:55.104 PDT <span class="error">[Thread-85]</span> DEBUG o.o.c.h.internal.HostTracker - Notifying Applications for Host 10.0.0.2 Being Added</p> <p>This bug is part of the project to Move all ADSAL associated component bugs to ADSAL.</p> Duplicate OPNFLWPLUG-61 OPNFLWPLUG-68 Development External issue ID 590 External issue URL Rank 0|i02j4n: