[CONTROLLER-872] Flow state not written to node after it is removed and re-added Created: 20/Sep/14  Updated: 19/Oct/17  Resolved: 05/May/15

Status: Resolved
Project: controller
Component/s: adsal
Affects Version/s: Helium
Fix Version/s: None

Type: Bug
Reporter: Srini Seetharaman Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: File dummy1.pcap    
External issue ID: 2000

 Description   

When flows are written to a node and it offine and returns after a few mins, the flow rules are not written to the node. Even an explicit write does not help.

Here are the flows written to a node on Table #50 before the node goes offline. When the node returns, these rules are not written to the node. Sometimes I see the flow LOADBALANCER_FORWARD_FLOW1_10.0.0.5 being written. Sometimes I see LOADBALANCER_REVERSE_FLOW_10.0.0.5_10.0.0.8 written. It is not deterministic.

<table xmlns="urn:opendaylight:flow:inventory">
<id>50</id>
<flow>
<id>LOADBALANCER_REVERSE_FLOW_10.0.0.5_10.0.0.8</id>
<flow-name>LOADBALANCER_REVERSE_FLOW_10.0.0.5_10.0.0.8</flow-name>
<match>...</match>
<priority>32768</priority>
<idle-timeout>0</idle-timeout>
<instructions>...</instructions>
<barrier>true</barrier>
<table_id>50</table_id>
<hard-timeout>0</hard-timeout>
</flow>
<flow>
<id>DEFAULT_PIPELINE_FLOW_50</id>
<flow-name>DEFAULT_PIPELINE_FLOW_50</flow-name>
<match/>
<priority>0</priority>
<idle-timeout>0</idle-timeout>
<instructions>...</instructions>
<barrier>true</barrier>
<table_id>50</table_id>
<hard-timeout>0</hard-timeout>
</flow>
<flow>
<id>LOADBALANCER_FORWARD_FLOW1_10.0.0.5</id>
<flow-name>LOADBALANCER_FORWARD_FLOW1_10.0.0.5</flow-name>
<match>...</match>
<priority>32768</priority>
<idle-timeout>0</idle-timeout>
<instructions>...</instructions>
<barrier>true</barrier>
<table_id>50</table_id>
<hard-timeout>0</hard-timeout>
</flow>
<flow>
<id>LOADBALANCER_REVERSE_FLOW_10.0.0.5_10.0.0.7</id>
<flow-name>LOADBALANCER_REVERSE_FLOW_10.0.0.5_10.0.0.7</flow-name>
<match>...</match>
<priority>32768</priority>
<idle-timeout>0</idle-timeout>
<instructions>...</instructions>
<barrier>true</barrier>
<table_id>50</table_id>
<hard-timeout>0</hard-timeout>
</flow>
<flow>
<id>LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.8</id>
<flow-name>LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.8</flow-name>
<match>...</match>
<priority>32769</priority>
<idle-timeout>0</idle-timeout>
<instructions>...</instructions>
<barrier>true</barrier>
<table_id>50</table_id>
<hard-timeout>0</hard-timeout>
</flow>
<flow>
<id>LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.7</id>
<flow-name>LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.7</flow-name>
<match>...</match>
<priority>32769</priority>
<idle-timeout>0</idle-timeout>
<instructions>...</instructions>
<barrier>true</barrier>
<table_id>50</table_id>
<hard-timeout>0</hard-timeout>
</flow>
</table>

 Comments   
Comment by Srini Seetharaman [ 20/Sep/14 ]

This issue was noticed in the OVSDB project.

Here is the DEBUG log when the flow is attempted to be rewritten on an InventoryListen.notifyNode() call when the node is added back. It says TransactionSuccess worked, but not rules written to the node.

2014-09-20 09:46:14.177 PDT [md-sal-binding-notification-182] DEBUG o.o.o.openstack.netvirt.LBaaSHandler - notifyNode: Node OF|00:00:4a:a8:82:09:59:48 update CHANGED from Controller's invente
2014-09-20 09:46:14.178 PDT [md-sal-binding-notification-182] DEBUG o.o.o.o.n.api.LoadBalancerProvider - Performing ADD rules for VIP 10.0.0.5 and 2 members
2014-09-20 09:46:14.179 PDT [md-sal-binding-notification-182] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow LOADBALANCER_FORWARD_FLOW1_10.0.0.5
2014-09-20 09:46:14.670 PDT [pool-28-thread-1] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow DEFAULT_PIPELINE_FLOW_90
2014-09-20 09:46:14.682 PDT [md-sal-binding-notification-182] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.7
2014-09-20 09:46:14.684 PDT [nioEventLoopGroup-13-3] WARN o.o.o.p.i.c.ResponseExpectedRpcListener - Request for RpcResultKey [xid=41, outputClazz=org.opendaylight.yang.gen.v1.urn.opendaylige
2014-09-20 09:46:15.172 PDT [pool-28-thread-1] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow DEFAULT_PIPELINE_FLOW_100
2014-09-20 09:46:15.185 PDT [md-sal-binding-notification-182] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.8
2014-09-20 09:46:15.187 PDT [nioEventLoopGroup-13-3] WARN o.o.o.p.i.c.ResponseExpectedRpcListener - Request for RpcResultKey [xid=43, outputClazz=org.opendaylight.yang.gen.v1.urn.opendaylige
2014-09-20 09:46:15.674 PDT [pool-28-thread-1] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow DEFAULT_PIPELINE_FLOW_110
2014-09-20 09:46:15.686 PDT [md-sal-binding-notification-182] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow LOADBALANCER_REVERSE_FLOW_10.0.0.5_10.0.0.7
2014-09-20 09:46:16.188 PDT [md-sal-binding-notification-182] DEBUG o.o.o.o.n.p.o.AbstractServiceInstance - Transaction success for write of Flow LOADBALANCER_REVERSE_FLOW_10.0.0.5_10.0.0.8

Comment by Srini Seetharaman [ 20/Sep/14 ]

Attached is the wireshark pcap of the OpenFlow messages. I see only the OFPFC_MODIFY for two of the flows:

  • LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.7
  • LOADBALANCER_FORWARD_FLOW2_10.0.0.5_10.0.0.8
Comment by Srini Seetharaman [ 20/Sep/14 ]

Attachment dummy1.pcap has been added with description: Pcap of OpenFlow messages

Comment by Srini Seetharaman [ 24/Sep/14 ]

It looks like the bug is resolved.

Previously, when a switch with the default OVSDB pipeline rules was disconnected by doing del-manager and del-controller, and then connected by doing set-mananger, the full rule set was not written.

But, now it works fine.

Comment by Carol Sanders [ 05/May/15 ]

This bug is part of the project to Move all ADSAL associated component bugs to ADSAL.

Generated at Wed Feb 07 19:54:05 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.