[CONTROLLER-232] Packet loss on first time ping test Created: 25/Mar/14  Updated: 25/Jul/23  Resolved: 05/May/15

Status: Resolved
Project: controller
Component/s: adsal
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Luis Gomez Assignee: Luis Gomez
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Issue Links:
Duplicate
is duplicated by OPNFLWPLUG-61 ping between hosts not working with m... Resolved
is duplicated by OPNFLWPLUG-68 pingall on Mininet with 15 nodes caus... Resolved
External issue ID: 590

 Description   

After setting up mininet with a normal topology like tree,2. We do the ping test and first time, there is always some packet loss:

mininet@mininet-vm:~$ sudo mn --controller=remote,ip=192.168.60.180 --topo tree,2

mininet> pingall

      • Ping: testing ping reachability
        h1 -> X h3 h4
        h2 -> h1 h3 h4
        h3 -> h1 h2 X
        h4 -> h1 h2 h3
      • Results: 16% dropped (10/12 received)
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 h3 h4
        h2 -> h1 h3 h4
        h3 -> h1 h2 h4
        h4 -> h1 h2 h3
      • Results: 0% dropped (12/12 received)

 Comments   
Comment by Abhijit Kumbhare [ 14/Apr/14 ]

Should be fixed in the stable release.

Comment by Madhusudhan Ananderi [ 21/Apr/14 ]

Hello, Can we have an update on this activity? Thanks!

Comment by Flavio Fernandes [ 28/Apr/14 ]

From: Flavio Fernandes <ffernand@redhat.com>
Subject: Re: ODL CONTROLLER-232
Date: April 28, 2014 at 6:29:02 PM EDT
To: Luis Gomez <ecelgp@gmail.com>

Hi Luis,

I spent most of today trying to reproduce the problem using the vms you pointed me to, and
the latest integration-base build. I failed miserably (see below). :/ What I really would like to see
is the log file when it does not work. Can you please set the 3 logs to trace and reproduce the
problem and send them to me?

W/out those, I’m pretty unsure on whether my assumption for the fix is going to do anything.

Sorry,

— flavio

mininet@mininet-vm:~/controller-base/opendaylight\> more version.properties
org.opendaylight.controller.version = 0.1
org.opendaylight.controller.build.scm.version = 61bad4207cc91cd14d8d38a255ad6549c20ff54e
org.opendaylight.controller.build.user = jenkins-controller
org.opendaylight.controller.build.workspace = **********
org.opendaylight.controller.build.timestamp = 1398697205857
org.opendaylight.controller.build.machine = **********

—

mininet@mininet-vm:~/controller-base/opendaylight\> ./run.sh -of13
setLogLevel org.opendaylight.controller.samples.simpleforwarding.internal TRACE
setLogLevel org.opendaylight.controller.hosttracker.internal TRACE

setLogLevel org.opendaylight.controller.routing TRACE

...

mininet@mininet-vm:~\> ~/integration/vm/scripts/start_mininet_of13.sh

      • Creating network
      • Adding controller
      • Adding hosts:
        h1 h2 h3 h4
      • Adding switches:
        have protcol params!
        s1 have protcol params!
        s2 have protcol params!
        s3
      • Adding links:
        (h1, s2) (h2, s2) (h3, s3) (h4, s3) (s1, s2) (s1, s3)
      • Configuring hosts
        h1 h2 h3 h4
      • Starting controller
      • Starting 3 switches
        s1 OVSswitch opts: protocols=OpenFlow13
        s2 OVSswitch opts: protocols=OpenFlow13
        s3 OVSswitch opts: protocols=OpenFlow13
      • Starting CLI:
        mininet>
        mininet>
        mininet>
        mininet>
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 h3 h4
        h2 -> h1 h3 h4
        h3 -> h1 h2 h4
        h4 -> h1 h2 h3
      • Results: 0% dropped (12/12 received)
        mininet>

On Apr 28, 2014, at 11:57 AM, Luis Gomez <ecelgp@gmail.com> wrote:

Hi Flavio,

Yes I use same VM to run controller + mininet and I use latest base edition. With this setup I see same issue with old 1.0 plugin and new 1.3 plugin.

Now after your mail I have tried with different setup: controller VM + mininet VM and you are right I do not see the problem with old OF1.0 plugin (at least with tree,2 and tree,3 topologies) but still I see it with OF1.3 plugin.

BR/Luis

On Apr 28, 2014, at 5:32 AM, Flavio Fernandes <ffernand@redhat.com> wrote:

Hello Luis!

Thanks for the kind words. The way I have been testing so far has been by building/running ODL straight out
of my laptop: Mac OS X. I tend to use the latest and greatest 1.2-snapshot codebase.

Also, I have mininet running from a virtual machine. I downloaded mininet VM from the mininet’s
website: mininet-2.1.0-130919-ubuntu-13.04-server-amd64-ovf. To run the VM, I use vmware fusion.
I gave the VM 1 Gb and 1 processor. Using that setup, I was able to reproduce the original issues. What I think happened is that the race is now slightly tighter and the difference in our setups is enough to run differently.

Thanks for the link! We should add a reference to it in CONTROLLER-232. Can you take care of that, please?

I’m grabbing the vms from the link you provided. I should have done that sooner, huh? I’m hoping that with those,
I should be able to see it. Questions on that:
1) do you see the bug when mininet and the controller are in different vms?
2) from your example, it seems that you have ODL and mininet co-located, and you are using OF1.0. So all
you use to reproduce this bug is 1 VM?
3) You are running the ‘base’ distribution of ODL, right?

Thanks,

— flavio

Comment by Madhusudhan Ananderi [ 28/Apr/14 ]

Hi Flaviof:

I just played with mininet. After setting the logs to TRACE:

<logger name="org.opendaylight.controller.hosttracker" level="TRACE"/>
<logger name="org.opendaylight.controller.routing" level="TRACE"/>
<logger name="org.opendaylight.controller.forwardingrulesmanager" level="TRACE"/>

I get the following outputs: (It looks better compared to the last time - Luis filed it) Let me know if you need opendaylight.log for the details.

mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633' --switch ovsk,protocols=OpenFlow13

      • Creating network
      • Adding controller
      • Adding hosts:
        h1 h2 h3 h4 h5 h6 h7 h8
      • Adding switches:
        have protcol params!
        s1 have protcol params!
        s2 have protcol params!
        s3 have protcol params!
        s4 have protcol params!
        s5 have protcol params!
        s6 have protcol params!
        s7
      • Adding links:
        (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7)
      • Configuring hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Starting controller
      • Starting 7 switches
        s1 OVSswitch opts: protocols=OpenFlow13
        s2 OVSswitch opts: protocols=OpenFlow13
        s3 OVSswitch opts: protocols=OpenFlow13
        s4 OVSswitch opts: protocols=OpenFlow13
        s5 OVSswitch opts: protocols=OpenFlow13
        s6 OVSswitch opts: protocols=OpenFlow13
        s7 OVSswitch opts: protocols=OpenFlow13
      • Starting CLI:
        mininet>
        mininet>
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 h3 h4 h5 h6 h7 h8
        h2 -> h1 h3 h4 h5 h6 h7 h8
        h3 -> h1 h2 h4 h5 h6 h7 h8
        h4 -> h1 h2 h3 h5 h6 h7 h8
        h5 -> h1 h2 h3 h4 h6 h7 h8
        h6 -> h1 h2 h3 h4 h5 h7 h8
        h7 -> h1 h2 h3 h4 h5 h6 h8
        h8 -> h1 h2 h3 h4 h5 h6 h7
      • Results: 0% dropped (56/56 received)
        mininet>
        mininet>
        mininet>
        mininet> quit
      • Stopping 7 switches
        s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...
      • Stopping 8 hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Stopping 1 controllers
        c0
      • Done
        completed in 39.013 seconds
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633'
      • Creating network
      • Adding controller
      • Adding hosts:
        h1 h2 h3 h4 h5 h6 h7 h8
      • Adding switches:
        s1 s2 s3 s4 s5 s6 s7
      • Adding links:
        (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7)
      • Configuring hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Starting controller
      • Starting 7 switches
        s1 OVSswitch opts:
        s2 OVSswitch opts:
        s3 OVSswitch opts:
        s4 OVSswitch opts:
        s5 OVSswitch opts:
        s6 OVSswitch opts:
        s7 OVSswitch opts:
      • Starting CLI:
        mininet>
        mininet>
        mininet>
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 h3 h4 h5 h6 h7 X
        h2 -> h1 h3 h4 h5 h6 h7 h8
        h3 -> h1 h2 h4 h5 h6 h7 h8
        h4 -> h1 h2 h3 h5 h6 h7 h8
        h5 -> h1 h2 h3 h4 h6 h7 h8
        h6 -> h1 h2 h3 h4 h5 h7 h8
        h7 -> h1 h2 h3 h4 h5 h6 h8
        h8 -> h1 h2 h3 h4 h5 h6 h7
      • Results: 1% dropped (55/56 received)
        mininet>
        mininet>
        mininet> quit
      • Stopping 7 switches
        s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...
      • Stopping 8 hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Stopping 1 controllers
        c0
      • Done
        completed in 33.244 seconds
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\> sudo mn --topo tree,4 --controller 'remote,ip=127.0.0.1,port=6633' --switch ovsk,protocols=OpenFlow13
      • Creating network
      • Adding controller
      • Adding hosts:
        h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
      • Adding switches:
        have protcol params!
        s1 have protcol params!
        s2 have protcol params!
        s3 have protcol params!
        s4 have protcol params!
        s5 have protcol params!
        s6 have protcol params!
        s7 have protcol params!
        s8 have protcol params!
        s9 have protcol params!
        s10 have protcol params!
        s11 have protcol params!
        s12 have protcol params!
        s13 have protcol params!
        s14 have protcol params!
        s15
      • Adding links:
        (h1, s4) (h2, s4) (h3, s5) (h4, s5) (h5, s7) (h6, s7) (h7, s8) (h8, s8) (h9, s11) (h10, s11) (h11, s12) (h12, s12) (h13, s14) (h14, s14) (h15, s15) (h16, s15) (s1, s2) (s1, s9) (s2, s3) (s2, s6) (s3, s4) (s3, s5) (s6, s7) (s6, s8) (s9, s10) (s9, s13) (s10, s11) (s10, s12) (s13, s14) (s13, s15)
      • Configuring hosts
        h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
      • Starting controller
      • Starting 15 switches
        s1 OVSswitch opts: protocols=OpenFlow13
        s2 OVSswitch opts: protocols=OpenFlow13
        s3 OVSswitch opts: protocols=OpenFlow13
        s4 OVSswitch opts: protocols=OpenFlow13
        s5 OVSswitch opts: protocols=OpenFlow13
        s6 OVSswitch opts: protocols=OpenFlow13
        s7 OVSswitch opts: protocols=OpenFlow13
        s8 OVSswitch opts: protocols=OpenFlow13
        s9 OVSswitch opts: protocols=OpenFlow13
        s10 OVSswitch opts: protocols=OpenFlow13
        s11 OVSswitch opts: protocols=OpenFlow13
        s12 OVSswitch opts: protocols=OpenFlow13
        s13 OVSswitch opts: protocols=OpenFlow13
        s14 OVSswitch opts: protocols=OpenFlow13
        s15 OVSswitch opts: protocols=OpenFlow13
      • Starting CLI:
        mininet>
        mininet>
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 X h14 h15 h16
        h2 -> h1 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h3 -> h1 h2 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h4 -> h1 h2 h3 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h5 -> h1 h2 h3 h4 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h6 -> h1 h2 h3 h4 h5 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h7 -> h1 h2 h3 h4 h5 h6 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h8 -> h1 h2 h3 h4 h5 h6 h7 h9 h10 h11 h12 h13 h14 h15 h16
        h9 -> h1 h2 h3 h4 h5 h6 h7 h8 h10 h11 h12 h13 h14 h15 h16
        h10 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h11 h12 h13 h14 h15 h16
        h11 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h12 h13 h14 h15 h16
        h12 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h13 h14 h15 h16
        h13 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h14 h15 h16
        h14 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h15 h16
        h15 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h16
        h16 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15
      • Results: 0% dropped (239/240 received)
        mininet>
        mininet>
        mininet> quit
      • Stopping 15 switches
        s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...s8 ...s9 ...s10 ...s11 ...s12 ...s13 ...s14 ...s15 ...
      • Stopping 16 hosts
        h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
      • Stopping 1 controllers
        c0
      • Done
        completed in 71.186 seconds
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\> sudo mn --topo tree,4 --controller 'remote,ip=127.0.0.1,port=6633'
      • Creating network
      • Adding controller
      • Adding hosts:
        h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
      • Adding switches:
        s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s12 s13 s14 s15
      • Adding links:
        (h1, s4) (h2, s4) (h3, s5) (h4, s5) (h5, s7) (h6, s7) (h7, s8) (h8, s8) (h9, s11) (h10, s11) (h11, s12) (h12, s12) (h13, s14) (h14, s14) (h15, s15) (h16, s15) (s1, s2) (s1, s9) (s2, s3) (s2, s6) (s3, s4) (s3, s5) (s6, s7) (s6, s8) (s9, s10) (s9, s13) (s10, s11) (s10, s12) (s13, s14) (s13, s15)
      • Configuring hosts
        h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
      • Starting controller
      • Starting 15 switches
        s1 OVSswitch opts:
        s2 OVSswitch opts:
        s3 OVSswitch opts:
        s4 OVSswitch opts:
        s5 OVSswitch opts:
        s6 OVSswitch opts:
        s7 OVSswitch opts:
        s8 OVSswitch opts:
        s9 OVSswitch opts:
        s10 OVSswitch opts:
        s11 OVSswitch opts:
        s12 OVSswitch opts:
        s13 OVSswitch opts:
        s14 OVSswitch opts:
        s15 OVSswitch opts:
      • Starting CLI:
        mininet>
        mininet>
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 X
        h2 -> h1 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h3 -> h1 h2 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h4 -> h1 h2 h3 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h5 -> h1 h2 h3 h4 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h6 -> h1 h2 h3 h4 h5 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h7 -> h1 h2 h3 h4 h5 h6 h8 h9 h10 h11 h12 h13 h14 h15 h16
        h8 -> h1 h2 h3 h4 h5 h6 h7 h9 h10 h11 h12 h13 h14 h15 h16
        h9 -> h1 h2 h3 h4 h5 h6 h7 h8 h10 h11 h12 h13 h14 h15 h16
        h10 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h11 h12 h13 h14 h15 h16
        h11 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h12 h13 h14 h15 h16
        h12 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h13 h14 h15 h16
        h13 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h14 h15 h16
        h14 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h15 h16
        h15 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h16
        h16 -> h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15
      • Results: 0% dropped (239/240 received)
        mininet>
        mininet>
        mininet> quit
      • Stopping 15 switches
        s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...s8 ...s9 ...s10 ...s11 ...s12 ...s13 ...s14 ...s15 ...
      • Stopping 16 hosts
        h1 h2 h3 h4 h5 h6 h7 h8 h9 h10 h11 h12 h13 h14 h15 h16
      • Stopping 1 controllers
        c0
      • Done
        completed in 62.396 seconds
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633'*** Creating network
      • Adding controller
      • Adding hosts:
        h1 h2 h3 h4 h5 h6 h7 h8
      • Adding switches:
        s1 s2 s3 s4 s5 s6 s7
      • Adding links:
        (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7)
      • Configuring hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Starting controller
      • Starting 7 switches
        s1 OVSswitch opts:
        s2 OVSswitch opts:
        s3 OVSswitch opts:
        s4 OVSswitch opts:
        s5 OVSswitch opts:
        s6 OVSswitch opts:
        s7 OVSswitch opts:
      • Starting CLI:
        mininet>
        mininet>
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 h3 X h5 h6 h7 h8
        h2 -> h1 h3 h4 h5 h6 h7 h8
        h3 -> h1 h2 h4 h5 h6 h7 h8
        h4 -> h1 h2 h3 h5 h6 h7 h8
        h5 -> h1 h2 h3 h4 h6 h7 h8
        h6 -> h1 h2 h3 h4 h5 h7 h8
        h7 -> h1 h2 h3 h4 h5 h6 h8
        h8 -> h1 h2 h3 h4 h5 h6 h7
      • Results: 1% dropped (55/56 received)
        mininet>
        mininet>
        mininet> quit
      • Stopping 7 switches
        s1 ..s2 .
        ..s3 ...s4 ...s5 ...s6 ...s7 ...
      • Stopping 8 hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Stopping 1 controllers
        c0
      • Done
        completed in 46.181 seconds
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\>
        mininet@mininet-vm:~\> sudo mn --topo tree,3 --controller 'remote,ip=127.0.0.1,port=6633' --switch ovsk,protocols=OpenFlow13
      • Creating network
      • Adding controller
      • Adding hosts:
        h1 h2 h3 h4 h5 h6 h7 h8
      • Adding switches:
        have protcol params!
        s1 have protcol params!
        s2 have protcol params!
        s3 have protcol params!
        s4 have protcol params!
        s5 have protcol params!
        s6 have protcol params!
        s7
      • Adding links:
        (h1, s3) (h2, s3) (h3, s4) (h4, s4) (h5, s6) (h6, s6) (h7, s7) (h8, s7) (s1, s2) (s1, s5) (s2, s3) (s2, s4) (s5, s6) (s5, s7)
      • Configuring hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Starting controller
      • Starting 7 switches
        s1 OVSswitch opts: protocols=OpenFlow13
        s2 OVSswitch opts: protocols=OpenFlow13
        s3 OVSswitch opts: protocols=OpenFlow13
        s4 OVSswitch opts: protocols=OpenFlow13
        s5 OVSswitch opts: protocols=OpenFlow13
        s6 OVSswitch opts: protocols=OpenFlow13
        s7 OVSswitch opts: protocols=OpenFlow13
      • Starting CLI:
        mininet>
        mininet> pingall
      • Ping: testing ping reachability
        h1 -> h2 X h4 h5 h6 h7 h8
        h2 -> h1 h3 h4 h5 h6 h7 h8
        h3 -> h1 h2 h4 h5 h6 h7 h8
        h4 -> h1 h2 h3 h5 h6 h7 h8
        h5 -> h1 h2 h3 h4 h6 h7 h8
        h6 -> h1 h2 h3 h4 h5 h7 h8
        h7 -> h1 h2 h3 h4 h5 h6 h8
        h8 -> h1 h2 h3 h4 h5 h6 h7
      • Results: 1% dropped (55/56 received)
        mininet>
        mininet>
        mininet> quit
      • Stopping 7 switches
        s1 ..s2 ...s3 ...s4 ...s5 ...s6 ...s7 ...
      • Stopping 8 hosts
        h1 h2 h3 h4 h5 h6 h7 h8
      • Stopping 1 controllers
        c0
      • Done
        completed in 71.230 seconds
        mininet@mininet-vm:~\>

Thanks,
Madhusudhan

Comment by Luis Gomez [ 30/Apr/14 ]

OK, the results of my test:

1) With 2 CPU Cores, it does not work at all with OF10 or OF13 plugins with even a small topology of tree,1 (1 switch), here is the debug log:

2014-04-29 16:48:55.098 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.1, MAC 2e02c8cff6a0, HostNodeConnector [nodeConnector=OF|1@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]
2014-04-29 16:48:55.098 PDT [Thread-84] DEBUG o.o.c.h.internal.HostTracker - New Host Learned: MAC: 2e02c8cff6a0 IP: 10.0.0.1
2014-04-29 16:48:55.099 PDT [Thread-84] DEBUG o.o.c.h.internal.HostTracker - Notifying Applications for Host 10.0.0.1 Being Added
2014-04-29 16:48:55.098 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42
2014-04-29 16:48:55.099 PDT [Thread-84] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Adding policyMatch(DIP = 10.0.0.1) Action(outPort= OF|1@OF|00:00:00:00:00:00:00:01) to node OF|00:00:00:00:00:00:00:01
2014-04-29 16:48:55.099 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector [nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]
2014-04-29 16:48:55.099 PDT [Thread-84] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Inside installPerHostRules
2014-04-29 16:48:55.100 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42
2014-04-29 16:48:55.101 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Host Added to ARPPending List, IP: /10.0.0.2
2014-04-29 16:48:55.101 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - hostFind(): Host Not Found for IP: IP=[10.0.0.2], Inititated Host Discovery ...
2014-04-29 16:48:55.101 PDT [Thread-84] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Successfully installed policy FlowEntry[flowName = [10.0.0.1/32 on N OF|00:00:00:00:00:00:00:01], groupName = 10.0.0.1/32, node = OF|00:00:00:00:00:00:00:01, flow = Flow[match = Match [fields=

{NW_DST=NW_DST(10.0.0.1), DL_TYPE=DL_TYPE(2048)}

, matches=2176], actions = [SET_DL_DST[address = 2e02c8cff6a0], OUTPUT[OF|1@OF|00:00:00:00:00:00:00:01]], priority = 1, id = 0, idleTimeout = 0, hardTimeout = 0]] on switch OF|00:00:00:00:00:00:00:01
2014-04-29 16:48:55.101 PDT [Thread-84] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Leaving installPerHostRules
2014-04-29 16:48:55.102 PDT [Thread-84] DEBUG o.o.c.h.internal.HostTracker - Notifying Topology Manager for Host 10.0.0.1 Being Added
2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 98
2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Handle punted IP packet: Ethernet: [DestinationMACAddress: de700b56a32c, SourceMACAddress: 2e02c8cff6a0, EtherType: 0800]
2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Host Added to ARPPending List, IP: /10.0.0.2
2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - hostFind(): Host Not Found for IP: IP=[10.0.0.2], Inititated Host Discovery ...
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.h.internal.HostTracker - New Host Learned: MAC: de700b56a32c IP: 10.0.0.2
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.h.internal.HostTracker - Host Removed from ARPPending List, IP: IP=[10.0.0.2]
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.h.internal.HostTracker - Notifying Applications for Host 10.0.0.2 Being Added
2014-04-29 16:48:55.104 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector [nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Adding policyMatch(DIP = 10.0.0.2) Action(outPort= OF|2@OF|00:00:00:00:00:00:00:01) to node OF|00:00:00:00:00:00:00:01
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Inside installPerHostRules
2014-04-29 16:48:55.105 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host
2014-04-29 16:48:55.105 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42
2014-04-29 16:48:55.105 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector [nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]
2014-04-29 16:48:55.106 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host
2014-04-29 16:48:55.106 PDT [Thread-85] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Successfully installed policy FlowEntry[flowName = [10.0.0.2/32 on N OF|00:00:00:00:00:00:00:01], groupName = 10.0.0.2/32, node = OF|00:00:00:00:00:00:00:01, flow = Flow[match = Match [fields=

{NW_DST=NW_DST(10.0.0.2), DL_TYPE=DL_TYPE(2048)}

, matches=2176], actions = [SET_DL_DST[address = de700b56a32c], OUTPUT[OF|2@OF|00:00:00:00:00:00:00:01]], priority = 1, id = 0, idleTimeout = 0, hardTimeout = 0]] on switch OF|00:00:00:00:00:00:00:01
2014-04-29 16:48:55.106 PDT [Thread-85] DEBUG o.o.c.s.s.i.SimpleForwardingImpl - Leaving installPerHostRules
2014-04-29 16:48:55.106 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42
2014-04-29 16:48:55.106 PDT [Thread-85] DEBUG o.o.c.h.internal.HostTracker - Notifying Topology Manager for Host 10.0.0.2 Being Added
2014-04-29 16:48:55.107 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector [nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]
2014-04-29 16:48:55.107 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host
2014-04-29 16:48:55.110 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 42
2014-04-29 16:49:05.099 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Received for Host: IP 10.0.0.2, MAC de700b56a32c, HostNodeConnector [nodeConnector=OF|2@OF|00:00:00:00:00:00:00:01, vlan=0, staticHost=false, arpSendCountDown=0]
2014-04-29 16:49:05.100 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - hostListener returned without adding the host

2) With 3+ CPU cores, OF10 plugin works good until tree, 6 (63 switches, the the maximum I have tried), with OF13 the maximum is tree,4 (15 switches), after that I see packet loss

Comment by Flavio Fernandes [ 30/Apr/14 ]

From: Flavio Fernandes <ffernand@redhat.com>
Subject: Re: ODL CONTROLLER-232
Date: April 30, 2014 at 5:54:09 AM EDT
To: Luis Gomez <ecelgp@gmail.com>, Colin Dixon <colin@colindixon.com>
Cc: Madhusudhan Kandadai <madhusudhan.opendaylight@yahoo.com>, Giovanni Meo <gmeo@cisco.com>

Hi Luis,

Thank you for your log! So the fact that the packet is getting lost is indeed due to hostTracker not
being done learning about the host before the exception packet is processed. That is good, because
we can add some mechanism for addressing that, should we deem it to be important enough.

Colin: if you look at the highlighted lines (below) and the code, it is very easy to see what happens.

A time '2014-04-29 16:48:55.102 PDT’, the exception packet comes to SimpleForwardingImpl
but it ends up getting dropped due to hostFind(dIP) call returning null. The interesting part is that immediately after, not even a second later, hostTracker learns about the host.

Assuming we want to solve this race, all we need is a way to hang on to the packet for a short while. That is
my plan of action, assuming you agree. My gut tells me we should fix it, because 1) it is pretty simple stuff and 2) we should not be worrying about how many cores are in use. I would like to talk to you about the implementation, still.

Thanks,

— flavio

private void handlePuntedIPPacket(IPv4 pkt, NodeConnector incomingNodeConnector) {
...
HostNodeConnector destHost = hostTracker.hostFind(dIP);
/*

  • In cases when incoming and outgoing connectors are in the same node, there is no need
  • to verify that there is a route. Because of that, we will only need routing.getRoute()
  • if we know that src and dst nodes are different.
    */
    if (destHost != null
    …

===

https://www.dropbox.com/s/hu86tbj1qcbsfh8/Screenshot%202014-04-30%2005.32.06.png

2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Received a frame of size: 98
2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] TRACE o.o.c.s.s.i.SimpleForwardingImpl - Handle punted IP packet: Ethernet: [DestinationMACAddress: de700b56a32c, SourceMACAdd\
ress: 2e02c8cff6a0, EtherType: 0800]
2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - Host Added to ARPPending List, IP: /10.0.0.2
2014-04-29 16:48:55.102 PDT [SwitchEvent Thread] DEBUG o.o.c.h.internal.HostTracker - hostFind(): Host Not Found for IP: IP=[10.0.0.2], Inititated Host Discovery ...
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.h.internal.HostTracker - New Host Learned: MAC: de700b56a32c IP: 10.0.0.2
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.h.internal.HostTracker - Host Removed from ARPPending List, IP: IP=[10.0.0.2]
2014-04-29 16:48:55.104 PDT [Thread-85] DEBUG o.o.c.h.internal.HostTracker - Notifying Applications for Host 10.0.0.2 Being Added

Comment by Carol Sanders [ 05/May/15 ]

This bug is part of the project to Move all ADSAL associated component bugs to ADSAL.

Generated at Wed Feb 07 19:52:31 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.