[OVSDB-294] Errors in 3-node Cluster whan all 3 nodes used as OVS Managers (odl-ovsdb-openstack) Created: 09/Feb/16  Updated: 22/Mar/16  Resolved: 22/Mar/16

Status: Resolved
Project: ovsdb
Component/s: openstack.net-virt
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Venkatrangan Govindarajan Assignee: ranjithkumar_t
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: File 5290_odl1.tar.gz     File 5290_odl2.tar.gz     File 5290_odl3.tar.gz     Zip Archive karaf_logs.zip    
External issue ID: 5290
Priority: Normal

 Description   

Am probably jumping ahead here. Just tried to test once with all nodes as OVS Managers and one node to handle neutron requests, Faced issue even while stacking.

ODL1->handle neutron
ODL1,ODL2,ODL3 -> OVS Managers
Local.conf: https://gist.github.com/gvranganvtn/42239afc7caece28b3c0

Exception in ODL1
16-02-09 17:18:49,774 | ERROR | n-invoker-impl-0 | apshotBackedReadWriteTransaction | 136 - org.opendaylight.controller.sal-core-spi - 1.3.0.SNAPSHOT | Tx: member-1-chn-411-txn-1 Failed Read of /(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)network-topology/topology/topology[

{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)topology-id=ovsdb:1}

]/node/node[

{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)node-id=ovsdb://uuid/7379aa64-0416-45e2-9531-351397cc742a}

]
java.lang.IllegalArgumentException: Metadata not available for modification NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)manager-entry, modificationType=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)manager-entry[

{(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)target=tcp:12.13.14.1:6640}

]=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)manager-entry[

{(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)target=tcp:12.13.14.1:6640}

], modificationType=DELETE, childModification={}], (urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)manager-entry[

{(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)target=tcp:12.13.14.2:6640}

]=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)manager-entry[

{(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)target=tcp:12.13.14.2:6640}

], modificationType=DELETE, childModification={}], (urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)manager-entry[

{(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)target=tcp:12.13.14.3:6640}

]=NodeModification [identifier=(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)manager-entry[

{(urn:opendaylight:params:xml:ns:yang:ovsdb?revision=2015-01-05)target=tcp:12.13.14.3:6640}

], modificationType=DELETE, childModification={}]}]
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)[64:com.google.guava:18.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:197)[80:org.opendaylight.yangtools.yang-data-impl:0.8.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:134)[80:org.opendaylight.yangtools.yang-data-impl:0.8.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:247)[80:org.opendaylight.yangtools.yang-data-impl:0.8.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:165)[80:org.opendaylight.yangtools.yang-data-impl:0.8.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:206)[80:org.opendaylight.yangtools.yang-data-impl:0.8.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:134)[80:org.opendaylight.yangtools.yang-data-impl:0.8.0.SNAPSHOT]
@

Will attach all logs shorthy

 Comments   
Comment by Anil Vishnoi [ 10/Feb/16 ]

Venkat, this is the setup which i am using, where i point OpenStack to controller-1 and connect all the devices to all the three controller and things looks fine.

Are you using dev stack setup? Do you see that the cluster is up and running fine before devstack set manager to ovsdb to connect to these controller?

Comment by Venkatrangan Govindarajan [ 18/Feb/16 ]

Tried the same test today, Did not observe the mentioned error messgae, But observed other exceptions in Karaf log

ODL1-> Neutron

ODL1,ODL2,ODL3 -> OVS

Creaetd Network + Instances + Router

ODL2 Down, Then removed and recreated instances, ODL2 UP and ODL3 down

Then such exceptions wewre observed in ODL1 and ODL2.

22:37:12,514 | ERROR | ntDispatcherImpl | EventDispatcher | 280 - org.opendaylight.ovsdb.openstack.net-virt - 1.2.1.Beryllium | Exception in dispatching event NorthboundEvent [handler=NEUTRON_PORT, action=DELETE, port=NeutronPort [portUUID=e2741f2c-78e3-4c33-8bd5-3a3333e9b3cc, networkUUID=e3c05fe4-715c-4776-af1e-ddd0ef4991d2, name=, adminStateUp=true, status=null, macAddress=fa:16:3e:66:3f:82, fixedIPs=[Neutron_IPs

{ipAddress='1.2.3.5', subnetUUID='1c9de44c-8c06-402e-b3b8-d5cb726f9e47'}

], deviceID=7a05b622-c7c6-44de-a4b9-ac3f4becbd22, deviceOwner=compute:nova, tenantID=1051edc9d3e34e6b9a6598c1e9abc71f, securityGroups=[NeutronSecurityGroup{securityGroupUUID='98e4c22d-010d-4f2c-9cd8-4fab8f944058', securityGroupName='default', securityGroupDescription='Default security group', securityGroupTenantID='1051edc9d3e34e6b9a6598c1e9abc71f', securityRules=[NeutronSecurityRule

{securityRuleUUID='df317fe0-7d9a-40aa-bf99-37bc4567b255', securityRuleDirection='ingress', securityRuleProtocol='null', securityRulePortMin=null, securityRulePortMax=null, securityRuleEthertype='IPv6', securityRuleRemoteIpPrefix='null', securityRemoteGroupID=98e4c22d-010d-4f2c-9cd8-4fab8f944058, securityRuleGroupID='98e4c22d-010d-4f2c-9cd8-4fab8f944058', securityRuleTenantID='1051edc9d3e34e6b9a6598c1e9abc71f'}

, NeutronSecurityRule

{securityRuleUUID='07f6dd0d-bd0c-49c1-b387-e2d5308ee1bd', securityRuleDirection='egress', securityRuleProtocol='null', securityRulePortMin=null, securityRulePortMax=null, securityRuleEthertype='IPv4', securityRuleRemoteIpPrefix='null', securityRemoteGroupID=null, securityRuleGroupID='98e4c22d-010d-4f2c-9cd8-4fab8f944058', securityRuleTenantID='1051edc9d3e34e6b9a6598c1e9abc71f'}

, NeutronSecurityRule

{securityRuleUUID='597764ac-7823-4974-bf15-39140d9624bb', securityRuleDirection='ingress', securityRuleProtocol='null', securityRulePortMin=null, securityRulePortMax=null, securityRuleEthertype='IPv4', securityRuleRemoteIpPrefix='null', securityRemoteGroupID=98e4c22d-010d-4f2c-9cd8-4fab8f944058, securityRuleGroupID='98e4c22d-010d-4f2c-9cd8-4fab8f944058', securityRuleTenantID='1051edc9d3e34e6b9a6598c1e9abc71f'}

, NeutronSecurityRule

{securityRuleUUID='a8d48c85-778c-43e8-9bf1-559442929913', securityRuleDirection='egress', securityRuleProtocol='null', securityRulePortMin=null, securityRulePortMax=null, securityRuleEthertype='IPv6', securityRuleRemoteIpPrefix='null', securityRemoteGroupID=null, securityRuleGroupID='98e4c22d-010d-4f2c-9cd8-4fab8f944058', securityRuleTenantID='1051edc9d3e34e6b9a6598c1e9abc71f'}

]]], bindinghostID=odl-os-compute-node, bindingvnicType=normal, bindingvnicType=normal], subnet=null, router=null, routerInterface=null, floatingIP=null, network=null, loadBalancer=null, loadBalancerPool=null, loadBalancerPoolMember=null, neutronsecurityRule=null]
java.lang.NullPointerException
at org.opendaylight.ovsdb.openstack.netvirt.impl.NeutronL3Adapter.cleanupFloatingIPRules(NeutronL3Adapter.java:1457)[280:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.Beryllium]
at org.opendaylight.ovsdb.openstack.netvirt.impl.NeutronL3Adapter.handleNeutronPortEvent(NeutronL3Adapter.java:439)[280:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.Beryllium]
at org.opendaylight.ovsdb.openstack.netvirt.PortHandler.doNeutronPortDeleted(PortHandler.java:127)[280:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.Beryllium]
@
------------

Comment by Venkatrangan Govindarajan [ 18/Feb/16 ]

Attachment 5290_odl1.tar.gz has been added with description: ODL1 Logs

Comment by Venkatrangan Govindarajan [ 18/Feb/16 ]

Attachment 5290_odl2.tar.gz has been added with description: ODL2 Logs

Comment by Venkatrangan Govindarajan [ 18/Feb/16 ]

Attachment 5290_odl3.tar.gz has been added with description: ODL3 logs

Comment by Venkatrangan Govindarajan [ 18/Feb/16 ]

Attached today's logs.

Comment by ranjithkumar_t [ 22/Mar/16 ]

Tested on Beryllium latest Build and its working as expected. Hence closing this bug.

Comment by ranjithkumar_t [ 22/Mar/16 ]

Attachment karaf_logs.zip has been added with description: karaf logs

Generated at Wed Feb 07 20:36:01 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.