[OPNFLWPLUG-293] of-flow: flow not deleted from device not operational table (but deleted from operational flow) Created: 22/Sep/14  Updated: 27/Sep/21  Resolved: 27/May/15

Status: Resolved
Project: OpenFlowPlugin
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Peter Gubka Assignee: Anil Vishnoi
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Linux
Platform: PC


Attachments: File f32.pcap     Text File karaf.log    
External issue ID: 2028

 Description   

odl: distribution-karaf-0.2.0-Helium-RC1-6-v201409220013

env: mininet 2.1.0, ovs 2.0.0

flow: integration.git /test/csit/variables/xmls/f32.xml

steps to reproduce:

1) add flow:
PUT request sent to url http://pce-guest37.cisco.com:8181/restconf/config/opendaylight-inventory:nodes/node/openflow:1/table/2/flow/155
with f32.xml as a content

2) check operational ds
GEThttp://pce-guest37.cisco.com:8181/restconf/operational/opendaylight-inventory:nodes/node/openflow:1/table/2/flow/155

GEThttp://pce-guest37.cisco.com:8181/restconf/operational/opendaylight-inventory:nodes/node/openflow:1/table/2

returned 200 OK with flow details

OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0xa, duration=8.969s, table=2, n_packets=0, n_bytes=0, idle_timeout=300, hard_timeout=600, send_flow_rem priority=32,ip,nw_dst=10.0.0.0/24 actions=set_mpls_ttl(1)

3) delete
DEL request sent to url http://pce-guest37.cisco.com:8181/restconf/config/opendaylight-inventory:nodes/node/openflow:1/table/2/flow/155
DEL Response 200

4) check:
present on device
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0xa, duration=39.373s, table=2, n_packets=0, n_bytes=0, idle_timeout=300, hard_timeout=600, send_flow_rem priority=32,ip,nw_dst=10.0.0.0/24 actions=set_mpls_ttl(1)

not present in "flow"
GET request sent to url http://pce-guest37.cisco.com:8181/restconf/operational/opendaylight-inventory:nodes/node/openflow:1/table/2/flow/155
GET response 404

but present in "table"
GET request sent to url http://pce-guest37.cisco.com:8181/restconf/operational/opendaylight-inventory:nodes/node/openflow:1/table/2
GET response 200
Response content: <table xmlns="urn:opendaylight:flow:inventory"><id>2</id><flow-table-statistics xmlns="urn:opendaylight:flow:table:statistics"><packets-looked-up>0</packets-looked-up><active-flows>1</active-flows><packets-matched>0</packets-matched></flow-table-statistics><flow-cookie-map><cookie>10</cookie><flow-ids>#UF$TABLE*2-1</flow-ids><flow-ids>155</flow-ids></flow-cookie-map><flow><id>#UF$TABLE*2-1</id><flags>SEND_FLOW_REM</flags><match><ipv4-destination>10.0.0.0/24</ipv4-destination><ethernet-match><ethernet-type><type>2048</type></ethernet-type></ethernet-match></match><priority>32</priority><idle-timeout>300</idle-timeout><instructions><instruction><order>0</order><apply-actions><action><order>0</order><set-mpls-ttl-action><mpls-ttl>1</mpls-ttl></set-mpls-ttl-action></action></apply-actions></instruction></instructions><flow-statistics xmlns="urn:opendaylight:flow:statistics"><duration><nanosecond>234000000</nanosecond><second>49</second></duration><byte-count>0</byte-count><packet-count>0</packet-count></flow-statistics><table_id>2</table_id><hard-timeout>600</hard-timeout><cookie>10</cookie></flow></table>

expected: to be deleted from everywhere



 Comments   
Comment by Peter Gubka [ 22/Sep/14 ]

Attachment karaf.log has been added with description: log file

Comment by Anil Vishnoi [ 22/Sep/14 ]

Hi Peter,

In the reported scenario, statistics manager is working as expected. Statistics manage to their respective flow/group/meter etc. In this scenario, when user installs the flow, it augment the flow statistics to flow/155 in operational data store. Then when you remove the flow (flow/155) from config data store, statistics manager also remove the respective flow statistics from operational data store.
BUT because this flow was not removed from the switch, in the next cycle of statistics data, it receives the statistics of the that flow. Now statistics manager try to search the respective flow in config data store to augment its statistics data, but it won't find it because it was already removed from the config data store. So it augment this flow as a alien flow using the alien ID (#UF$TABLE*2-1).

So the main issue here is why flow was not removed from the switch when user deleted the flow from config data store. Because if it would have been removed from switch, statistics manager should not have received any flow statistics update from switch and therefore you should not have seen any statistics augmented against the alien id.

Can you please update the bug with the details of the features you installed in karaf for testing this scenario?.If someone won't be able to look at it by tomorrow morning my time, i will have a look at it.

Thanks
Anil

Comment by Peter Gubka [ 23/Sep/14 ]

I use:
feature:install odl-mdsal-broker odl-restconf odl-bgpcep-all odl-netconf-all odl-netconf-connector-all odl-openflowplugin-all odl-openflowplugin-drop-test odl-sfcofl2 odl-sfc-test-consumer

Comment by Anil Vishnoi [ 23/Sep/14 ]

Hi Peter,

I tried to recreate this issue in my local setup. My local setup is running mininet (2.0.0) and ovs (2.1.2). I used the same karaf RC build which you mentioned for the testing. In my setup when i tried to install the flow (f32.xml), i get following error message from switch

Type: Error in action description - OFPET_BAD_ACTION (2)
Code: Action can’t apply for this match or Set-Field missing prerequisite - OFPBAC_MATCH_INCONSISTENT (0x000a)

As per specs this is valid error message.

Although this flow didn't get installed in switch, but it was augmented in the config data store, so i sent DELETE REST call to delete it, and i can see that flow_mod (for delete) also reaches to the switch. But this flow mod for delete also results in error from switch side. I get following error message from switch (which is obvious because we are trying to delete the flow which doesn't exist in the switch):

Type: Problem modifying flow entry - OFPET_FLOW_MOD_FAILED (5)
Code: Unspecified error - OFPFMFC_UNKNOWN (0x0000)

From controller perspective, i can see that its sending flow mod for both installing and removing the flow as expected. That make me believe that its an issue from switch side. It seems to be a bug in OVS 2.0.0 version.

In my environment i don't have OVS version==2.0.0, can you please favor me and capture the wireshark dump (.pcap file) in your environment and attach to the bug report. You just have to start the wireshark, send the flow add request, confirm that flow got installed on switch and then immediately delete the from using restconf. Through this pcap file, i want to confirm two things,

1) That switch is receiving both the flow mod (for installing and removing the flow)
2) If switch throws any error for the flow_mod for flow delete.

1st observation will confirm that controller is indeed sending appropriate flow mods to add/remove flow. Secondly, if we see error for flow delete flow mod, it can give us further pointer on why it's not deleting the flow. There can be two possible reasons, 1)controller is not sending appropriate flow_mod for flow deletion (very unlikely because its not putting any extra intelligence for flow deletion compared to flow add) OR 2) there is some issue from switch side.

Please let me know if you need any help in capturing the wireshark dump.

Thanks
Anil

Comment by Peter Gubka [ 23/Sep/14 ]

please filter ports 6633 and 8181 for the traffic you are interested in

Comment by Peter Gubka [ 23/Sep/14 ]

Attachment f32.pcap has been added with description: pcap file

Comment by Anil Vishnoi [ 23/Sep/14 ]

So this pcap file shows that there was no error while installing the flow, but it throws error for the flow mod that deletes the flow. Error thrown is same as i get in my environment. Flow mode sent by controller also looks fine to me. So i believe OVS== 2.0.0 is buggy for mpls related flows. Initial implementation of mpls related actions (set and decrement) was introduced in 1.11.0 version of OVS, so its highly likely that its not fully bug free. As per specs OVS should not allow this flow to be installed and that's what we see in ovs==2.1.2 version.

Comment by Abhijit Kumbhare [ 23/Sep/14 ]

This is not a blocking bug for Helium (Peter agrees).

Comment by Abhijit Kumbhare [ 06/May/15 ]

Peter,

Can you retest and update this old bug? Also check the priority of the bug - should it be critical or major?

Abhijit

Comment by Peter Gubka [ 07/May/15 ]

i updated the importance to "normal" as it is very odl, i'll recheck it on monday, had different task today

Comment by Peter Gubka [ 27/May/15 ]

tested with lithium odl and ovs 2.3.0
you were right, the flow is not correct. ovs rejected the flow

closing bug as invalid

Generated at Wed Feb 07 20:32:05 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.