[CONTROLLER-1407] Clustering: The flows are not re-installed when switch is disconnected and reconnected to same follower node in a 3 node cluster Created: 21/Aug/15  Updated: 26/Jan/16  Resolved: 26/Jan/16

Status: Resolved
Project: controller
Component/s: clustering
Affects Version/s: Lithium
Fix Version/s: None

Type: Bug
Reporter: Sanjib Mohapatra Assignee: Muthukumaran Kothandaraman
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: Other


Attachments: File All Controllers Log.rar    
External issue ID: 4164
Priority: High

 Description   

In 3 node cluster, Applied flows in a Follower node and switch is connected to another follower node. The flows are not re-installed when switch is disconnected and reconnected to same follower node .

1. All three nodes (c1, c2 and c3) of clusters are running in VM mode.

2. Following features are installed in the cluster nodes.

feature:install odl-restconf-all
feature:install odl-mdsal-all
feature:install odl-openflowplugin-all

3. Verified all 2550 and 2551 ports are up in all cluster nodes.

root@mininet-vm:~# netstat -na | grep 2550
tcp6 0 0 10.183.181.43:2550 :::* LISTEN
tcp6 0 0 10.183.181.43:33607 10.183.181.41:2550 ESTABLISHED
tcp6 0 0 10.183.181.43:2550 10.183.181.41:46032 ESTABLISHED
tcp6 0 0 10.183.181.43:44406 10.183.181.42:2550 ESTABLISHED
tcp6 0 0 10.183.181.43:2550 10.183.181.42:36122 ESTABLISHED
root@mininet-vm:~# netstat -na | grep 2551
tcp6 0 0 10.183.181.43:2551 :::* LISTEN
tcp6 0 0 10.183.181.43:2551 10.183.181.41:46793 ESTABLISHED
tcp6 0 0 10.183.181.43:56530 10.183.181.41:2551 ESTABLISHED
tcp6 0 0 10.183.181.43:2551 10.183.181.42:57604 ESTABLISHED
tcp6 0 0 10.183.181.43:41741 10.183.181.42:2551 ESTABLISHED

4. via Jconsole found that c1 is leader (opendayligh controller --> config data -> Inventory module -> shard -> attribute) , c2 and c3 are followers.

5. Connected Mininet OF switch to follower c2.

root@mininet-vm:/home/mininet/integration/test/tools/odl-mdsal-clustering-tests/clustering-performance-test# sudo mn –

controller=remote,ip=10.183.181.42 --topo=linear,1 --switch ovsk,protocols=OpenFlow13

6. Sent 10 flows from another follower c3.

root@mininet-vm:/home/mininet/integration/test/tools/odl-mdsal-clustering-tests/clustering-performance-test# ./flow_config_blaster.py --flows 10 --auth –

no-delete

Cycle 0:
Thread 0:
Adding 10 flows on 1 nodes
Thread 0:
Add time: 0.86,
Add success rate: 11.67, Add total rate: 11.67
Add Results:

{200: 10}

Total success rate: 11.67, Total rate: 11.67
Measured rate: 6.15 (52.72% of Total success rate)
Measured time: 1.62s

      • Total flows added: 10
        HTTP[OK] results: 10

root@mininet-vm:/home/mininet/integration/test/tools/odl-mdsal-clustering-tests/clustering-performance-test#

7. Verify 10 Flows are installed in switch.

mininet> sh ovs-ofctl -O Openflow13 dump-flows s1
OFPST_FLOW reply (OF1.3) (xid=0x2):
cookie=0x6, duration=45.883s, table=0, n_packets=0, n_bytes=0, idle_timeout=65000, hard_timeout=65000, priority=2,ip,nw_dst=10.0.0.7 actions=drop
cookie=0x5, duration=45.911s, table=0, n_packets=0, n_bytes=0, idle_timeout=65000, hard_timeout=65000, priority=2,ip,nw_dst=10.0.0.6 actions=drop
............
.............
...............

8. Disconnect the switch and connect the switch back to Follower node c2.

mininet> sh ovs-ofctl -O Openflow13 dump-flows s1
OFPST_FLOW reply (OF1.3) (xid=0x2):

I see no flows.



 Comments   
Comment by Sanjib Mohapatra [ 21/Aug/15 ]

Attachment All Controllers Log.rar has been added with description: All Controller nodes karaf logs

Comment by Sanjib Mohapatra [ 21/Aug/15 ]

An important comment:
Flows are always installed, When i connect with the Leader cluster node. Even if i disconnect and reconnect with Leader cluster node, flows are always installed.

Thanks
Sanjib

Comment by Sanjib Mohapatra [ 03/Nov/15 ]

This issue is there in SR2. Moving the importance of it to critical. As it is a must for basic cluster HA operation.

Comment by Muthukumaran Kothandaraman [ 20/Jan/16 ]

With Beryllium master and Helium plugin combination, this issue is not reproducible consistently. This has to be investigated further with Helium and Lithium plugin combination.

Generated at Wed Feb 07 19:55:28 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.