[OPNFLWPLUG-30] ping between hosts not working with OF10 Created: 15/Jan/14  Updated: 27/Sep/21  Due: 17/Jan/14  Resolved: 26/Jan/14

Status: Resolved
Project: OpenFlowPlugin
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Moiz Raja Assignee: Vaclav Demcak
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: File flowModsWithOvs20.pcapng     File flowmod.pcapng    
External issue ID: 306

 Description   

After adding a topology in mininet pinging two hosts does not work. Initially there were some issues with the md-sal/ad-sal adaptation layer. These issues have been addressed by this commit https://git.opendaylight.org/gerrit/#/c/4238/

After the controller changes the ping between two hosts is still not working.



 Comments   
Comment by Moiz Raja [ 18/Jan/14 ]

Since 01/16/2014 ping should be working for OF1.3. It is not working yet for OF1.0. On investigation I have atleast ruled out the compatibility layer because it behaves similarly for both 1.3 and 1.0. I got to the point where the ModelDrivenSwitchImpl#addFlow is called twice for a ping as would be expected - since two flows need to be programmed for a ping to work. However only the last added flow ever gets installed on the switch. It's almost like the addFlow call modifies the first flow instead of adding another.

Would be good if someone with more openflow knowledge could look at this.

Comment by Moiz Raja [ 21/Jan/14 ]

Attachment flowmod.pcapng has been added with description: flowmod capture

Comment by Moiz Raja [ 21/Jan/14 ]

Michal,

I have attached a wireshark capture which shows the flow mod messages that are sent by the controller when a ping is done between two hosts. From my looking at the trace it seemed to me that openflowjava may not be properly serializing the match field for OF1.0. The reason for this suspicion is that wireshark is not able to decode the match field for OF1.0 whereas it is able to do it properly for OF1.3.

From debugging the code I saw that the match field was being passed on to openflowjava ConnectionAdapterImpl@flowMod in the FlowModInput object in the _matchV10 field (_match is null). Hopefully you can start debugging from this point and see what may be going on.

Comment by Michal Polkorab [ 21/Jan/14 ]

Hi Moiz,

I am unable to reproduce this issue (although pinging host for longer period of time). I can see no errors in the osgi console either.

FlowMod messages in attached wireshark capture look OK.

Michal

Comment by Moiz Raja [ 21/Jan/14 ]

Michal,

As per our conversation - one difference between your environment and mine is that you are using ovs 1.1 and I am using ovs 2.0. Please try out this experiment with ovs 2.0 and see if you can reproduce this problem.

Comment by Michal Polkorab [ 22/Jan/14 ]

Attachment flowModsWithOvs20.pcapng has been added with description: FlowMod messages while pinging

Comment by Michal Polkorab [ 22/Jan/14 ]

Hello,

i tried with OVS 2.0 and the result is same - ping working for me. I can see no errors in the osgi console, packets look ok on the wire (see attachment).

Michal

Comment by Madhusudhan Ananderi [ 22/Jan/14 ]

ARP Handler - Ping defect

There is a defect for this one using OF10 mininet (sudo mn --controller=remote,ip=127.0.0.1 --topo tree,2). It was working fine two days ago for OF13 mininet, but when I try to retest it now (By taking the latest plugin using OF13 mininet, sudo mn --controller=remote,ip=127.0.0.1 --topo tree,2 --switch ovsk,protocols=OpenFlow13), it is failing now for OF13mininet too.

-Madhusudhan

Comment by Hideyuki Tai [ 23/Jan/14 ]

It was failed to ping between hosts with OF 1.0 Mininet.

I used the latest artifacts for the Base Edition.

https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/distributions-base/0.1.0-SNAPSHOT/distributions-base-0.1.0-20140123.052516-515-osgipackage.zip

Using this, I started up the controller like this:

$ /run.sh -of13

After starting the controller, I run OF 1.0 mininet, and execute ping command.
The only first ping packet was successfully forwarded,
but all other ping packets was not forwarded.

I saw the following in the log:

2014-01-23 08:03:32.976 EST [nioEventLoopGroup-9-3] INFO o.o.o.p.impl.core.OFFrameDecoder - OF Protocol message received, type:17
2014-01-23 08:03:32.976 EST [nioEventLoopGroup-9-3] ERROR o.o.o.protocol.impl.core.OFDecoder - Message deserialization failed
2014-01-23 08:03:32.977 EST [nioEventLoopGroup-9-3] ERROR o.o.o.protocol.impl.core.OFDecoder - readerIndex(399) + length(32) exceeds writerIndex(427): SlicedByteBuf(ridx: 399, widx: 427, cap: 427/427, unwrapped: UnpooledUnsafeDirectByteBuf(ridx: 0, widx: 52, cap: 512))
java.lang.IndexOutOfBoundsException: readerIndex(399) + length(32) exceeds writerIndex(427): SlicedByteBuf(ridx: 399, widx: 427, cap: 427/427, unwrapped: UnpooledUnsafeDirectByteBuf(ridx: 0, widx: 52, cap: 512))
at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1160) ~[bundlefile:na]
at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:668) ~[bundlefile:na]
at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:676) ~[bundlefile:na]
at org.opendaylight.openflowjava.protocol.impl.util.ByteBufUtils.decodeNullTerminatedString(ByteBufUtils.java:210) ~[bundlefile:na]
at org.opendaylight.openflowjava.protocol.impl.deserialization.factories.OF10StatsReplyMessageFactory.setTable(OF10StatsReplyMessageFactory.java:202) ~[bundlefile:na]
at org.opendaylight.openflowjava.protocol.impl.deserialization.factories.OF10StatsReplyMessageFactory.bufferToMessage(OF10StatsReplyMessageFactory.java:104) ~[bundlefile:na]
at org.opendaylight.openflowjava.protocol.impl.deserialization.factories.OF10StatsReplyMessageFactory.bufferToMessage(OF10StatsReplyMessageFactory.java:60) ~[bundlefile:na]
at org.opendaylight.openflowjava.protocol.impl.deserialization.DeserializationFactory.bufferToMessage(DeserializationFactory.java:41) ~[bundlefile:na]
at org.opendaylight.openflowjava.protocol.impl.core.OFDecoder.decode(OFDecoder.java:47) [bundlefile:na]
at org.opendaylight.openflowjava.protocol.impl.core.OFDecoder.decode(OFDecoder.java:26) [bundlefile:na]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:337) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:323) [bundlefile:na]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:337) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:323) [bundlefile:na]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:337) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:323) [bundlefile:na]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:253) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:337) [bundlefile:na]
at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:323) [bundlefile:na]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785) [bundlefile:na]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100) [bundlefile:na]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:480) [bundlefile:na]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:447) [bundlefile:na]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:341) [bundlefile:na]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101) [bundlefile:na]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_15]

Comment by Madhusudhan Ananderi [ 24/Jan/14 ]

We see the same problem in our lab/setup. Here are the details.

Please check if there is a bug for this otherwise open one:

After doing h1 ping h4 (mininet OF1.0) BAD

mininet@mininet-vm:~\> sudo ovs-ofctl dump-flows s1
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=14.77s, table=0, n_packets=0, n_bytes=0, idle_age=14, priority=1,ip actions=output:1
mininet@mininet-vm:~\> sudo ovs-ofctl dump-flows s2
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=16.318s, table=0, n_packets=30, n_bytes=2940, idle_age=2, priority=1,ip actions=mod_dl_dst:c6:4e:04:b3:d8:0a,output:2
mininet@mininet-vm:~\> sudo ovs-ofctl dump-flows s3
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=19.223s, table=0, n_packets=0, n_bytes=0, idle_age=19, priority=1,ip actions=output:3

After doing h1 ping h4 (mininet OF1.3) GOOD

mininet@mininet-vm:~\> sudo ovs-ofctl -O OpenFlow13 dump-flows s1
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=7.492s, table=0, n_packets=6, n_bytes=588, send_flow_rem priority=1,ip,nw_dst=10.0.0.4 actions=output:2
cookie=0x0, duration=7.524s, table=0, n_packets=6, n_bytes=588, send_flow_rem priority=1,ip,nw_dst=10.0.0.1 actions=output:1
mininet@mininet-vm:~\> sudo ovs-ofctl -O OpenFlow13 dump-flows s2
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=9.314s, table=0, n_packets=8, n_bytes=784, send_flow_rem priority=1,ip,nw_dst=10.0.0.4 actions=output:3
cookie=0x0, duration=9.35s, table=0, n_packets=8, n_bytes=784, send_flow_rem priority=1,ip,nw_dst=10.0.0.1 actions=output:1
mininet@mininet-vm:~\> sudo ovs-ofctl -O OpenFlow13 dump-flows s3
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x0, duration=11.194s, table=0, n_packets=10, n_bytes=980, send_flow_rem priority=1,ip,nw_dst=10.0.0.4 actions=output:2
cookie=0x0, duration=11.23s, table=0, n_packets=10, n_bytes=980, send_flow_rem priority=1,ip,nw_dst=10.0.0.1 actions=output:3

Comment by Michal Polkorab [ 24/Jan/14 ]

I have checked the OF10StatsReplyMessageFactory.setTable() method. Looks fine.

Can you provide wireshark capture with the failed MultiPartReply message ?
And detailed usecase would be great, because I am still unable to reproduce the issue.

Thanks
Michal

Comment by Michal Polkorab [ 25/Jan/14 ]

Hello,

the deserialization issue is fixed by: https://git.opendaylight.org/gerrit/#/c/4767/

It is a temporary fix, because this issue is caused by the ovs switch (sending wrong data / OF1.0 Table Statistics). Change will be reverted once the ovs switch will work.

Michal

Comment by Ed Warnicke [ 25/Jan/14 ]

Please verify fix

Comment by Vaclav Demcak [ 26/Jan/14 ]

Test env:
openflowplugin controller (bulded 25.1. [CET 19:05])
mininet v. 2.0.1, ovs 2.0.0, REST by postman

INPUT 1
http://localhost:8080/restconf/config/opendaylight-inventory:nodes/node/openflow:1/table/2/flow/22
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<flow xmlns="urn:opendaylight:flow:inventory">
<cookie>0</cookie>
<hard-timeout>0</hard-timeout>
<id>22</id>
<idle-timeout>0</idle-timeout>
<instructions>
<instruction>
<order>1</order>
<apply-actions>
<action>
<order>1</order>
<output-action>
<output-node-connector>openflow:2:2</output-node-connector>
</output-action>
</action>
</apply-actions>
<order>0</order>
</instruction>
</instructions>
<match>
<in-port>MD_SAL|openflow:2:1</in-port>
</match>
<priority>5</priority>
<table_id>2</table_id>
</flow>

INPUT 2
http://localhost:8080/restconf/config/opendaylight-inventory:nodes/node/openflow:1/table/2/flow/23
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<flow xmlns="urn:opendaylight:flow:inventory">
<cookie>0</cookie>
<hard-timeout>0</hard-timeout>
<id>23</id>
<idle-timeout>0</idle-timeout>
<instructions>
<instruction>
<order>1</order>
<apply-actions>
<action>
<order>1</order>
<output-action>
<output-node-connector>openflow:2:1</output-node-connector>
</output-action>
</action>
</apply-actions>
<order>0</order>
</instruction>
</instructions>
<match>
<in-port>MD_SAL|openflow:2:2</in-port>
</match>
<priority>5</priority>
<table_id>2</table_id>
</flow>

RESULT
h1 ping h2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_req=2 ttl=64 time=0.348 ms
64 bytes from 10.0.0.2: icmp_req=3 ttl=64 time=0.043 ms

Comment by Vaclav Demcak [ 26/Jan/14 ]

fix couple of misstakes in reported test data:

testd minintes:
sudo mn --topo tree,2 --controller 'remote,ip=127.0.0.1,port=6653' --switch ovsk,protocols=OpenFlow13
input / output reported before
----------
sudo mn --topo tree,2 --controller 'remote,ip=127.0.0.1,port=6653' --switch ovsk,protocols=OpenFlow10

Input:
http://localhost:8080/restconf/config/opendaylight-inventory:nodes/node/openflow:2/table/2/flow/22
same xml (only table_id has to be 0)

http://localhost:8080/restconf/config/opendaylight-inventory:nodes/node/openflow:2/table/2/flow/23
same xml (only table_id has to be 0)

mininet> h1 ping h2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_req=1 ttl=64 time=1.48 ms
64 bytes from 10.0.0.2: icmp_req=2 ttl=64 time=0.047 ms
64 bytes from 10.0.0.2: icmp_req=3 ttl=64 time=0.083 ms

mininet@debian:~$ sudo ovs-ofctl -O OpenFlow10 dump-flows s2
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=7.234s, table=0, n_packets=0, n_bytes=0, idle_age=7, priority=1,in_port=1 actions=output:2
cookie=0x0, duration=133.039s, table=0, n_packets=5, n_bytes=210, idle_age=59, priority=1,in_port=2 actions=output:1

Comment by Vaclav Demcak [ 26/Jan/14 ]

mininet@debian:~$ sudo ovs-ofctl -O OpenFlow10 dump-flows s2
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=454.651s, table=0, n_packets=18, n_bytes=1316, idle_age=123, priority=1,in_port=1 actions=output:2
cookie=0x0, duration=580.456s, table=0, n_packets=18, n_bytes=1316, idle_age=123, priority=1,in_port=2 actions=output:1

Generated at Wed Feb 07 20:31:23 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.