[CONTROLLER-1130] Clustering: odl-inventory not able to show 250 mininet switches (SR2) Created: 29/Jan/15  Updated: 17/Feb/15  Resolved: 17/Feb/15

Status: Resolved
Project: controller
Component/s: mdsal
Affects Version/s: Post-Helium
Fix Version/s: None

Type: Bug
Reporter: Peter Gubka Assignee: Mark Mozolewski
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Linux
Platform: PC

Attachments: Zip Archive 3-node_karaf.log.zip     Text File sr2-karaf.log    
External issue ID: 2653
Priority: High

 Description   

I am unable to see 250 switches in inventory ds.

used build
https://nexus.opendaylight.org/content/repositories/staging/org/opendaylight/integration/distribution-karaf/0.2.2-Helium-SR2/distribution-karaf-0.2.2-Helium-SR2.tar.gz

steps to reproduce:
1) karaf start (i did with default values, which means 2G for java, 512m perm)
2) feature:install odl-restconf odl-mdsal-clustering
3) feature:install odl-openflowplugin-all
4) when cpu usage lowers down, start mininet with 250 modes
sudo mn --topo linear,250,1 --controller 'remote,ip=10.25.2.9,port=6653' --switch ovsk,protocols=OpenFlow13

5) then requests for /restconf/operational/opendaylight-inventory:nodes/
hrr

In [28]: url
Out[28]: 'http://10.195.91.134:8181/restconf/operational/opendaylight-inventory:nodes/'

In [29]: hdr
Out[29]:

{'Accept': 'application/json'}

In [30]: rsp = requests.get(url, headers=hdr, stream=False, auth=('admin', 'admin'))

In [31]: len(json.loads(rsp.content)['nodes']['node'])
Out[31]: 202

I got once even http status 500.

 Comments   
Comment by Peter Gubka [ 29/Jan/15 ]

Attachment sr2-karaf.log has been added with description: karaf log file

Comment by Moiz Raja [ 29/Jan/15 ]

Peter, Can you confirm that with IMDS this works as expected. If yes then we can move this bug to the Clustering bucket.

Comment by Peter Gubka [ 30/Jan/15 ]

(In reply to Moiz Raja from comment #1)
> Peter, Can you confirm that with IMDS this works as expected. If yes then we
> can move this bug to the Clustering bucket.

Yes, i tried it without installling odl-mdsal-clustering and i was able to attach all 250 switches.

Comment by Mark Mozolewski [ 06/Feb/15 ]

Helium-SR2 Build:

  • Confirmed standalone correctly operates
  • Confirmed cluster mode (still 1 node) has issues for same test:

Also confirmed that latest Controller code (post-Helium -

Comment by Mark Mozolewski [ 06/Feb/15 ]

Helium-SR2 Build:
-----------------

  • Confirmed 1-node standalone (IMDS) operates correctly.
  • Confirmed 1-node clustering (Distributed DS) has issues for same mininet test.
    Details: Lots of exceptions, left with 2l+ live threads at end of mininet adding 250 switches, REST interface is locked out, etc.

Latest Controller Code (post-Helium - MASTER - 543f919):
--------------------------------------------------------

  • Both standalone and clustering mode operate correctly (all 250 switches in inventory w/o error).

I am debugging the issue and consulting with clustering team about process for any update for Helium SR2 code.

Comment by Mark Mozolewski [ 06/Feb/15 ]

IGNORE - see comment #4

Comment by Mark Mozolewski [ 17/Feb/15 ]

Since latest Lithium code does NOT show the issue various fixes since Helium SR2 for AskTimeoutExceptions, etc. probably addressed this test failure.

It is advised to use the Lithium release for those that are not required to use Helium-SR<#> releases. If an end-user has a similar issue and MUST use a Helium release we will debug further propose a patch then.

(Re-open as needed)

Comment by Mark Mozolewski [ 17/Feb/15 ]

ODL1 Machine:
Inventory operational shard Leader
*Connected mininet to this machine.

ODL2 Machine:
Inventory config shard Leader.

192 of 250 switches accounted for in datastore.

Comment by Mark Mozolewski [ 17/Feb/15 ]

Attachment 3-node_karaf.log.zip has been added with description: Karaf logs from 3-node run of Helium-SR2 code.

Generated at Wed Feb 07 19:54:46 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.