https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/NETVIRT-1106 netvirt <p>Create Multiple VPNs test fails, because another L3VPN, created earlier (with the same name and ID) was not deleted properly by executing deleteL3VPN call.</p> <p><a href="https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-openstack-ocata-gate-stateful-oxygen/764/odl_1/odl1_karaf.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-openstack-ocata-gate-stateful-oxygen/764/odl_1/odl1_karaf.log.gz</a></p> NETVIRT-1106

failures in Create Multiple VPNs test

Bug Medium Resolved Cannot Reproduce Philippe Guibert Valentina Krasnobaeva Fri, 16 Feb 2018 13:41:47 +0000 Sat, 7 Jul 2018 00:50:38 +0000 Sat, 7 Jul 2018 00:50:38 +0000 Oxygen Fluorine Oxygen-SR1 Fluorine General 0 6 <p>Please provide an update on this issue.  Is this a true blocker?  Any ETA on resolution?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=pguibert6WIND" class="user-hover" rel="pguibert6WIND">pguibert6WIND</a>/<a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=valentina.krasnobaeva" class="user-hover" rel="valentina.krasnobaeva">valentina.krasnobaeva</a> - We need to get this resolved urgently, it's blocking the imminent Oxygen release.</p> <p>Hi,</p> <p>ticket is not updated yet.</p> <ul> <li>a partial fix has been done <a href="https://git.opendaylight.org/gerrit/#/c/69115/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/69115/</a></li> <li>also by <a href="https://git.opendaylight.org/gerrit/#/c/68486/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/68486/</a> on oxygen.</li> </ul> <p>once integrated, ticket should be updated.</p> <p> </p> <p>reproduced while perfoming operations with dualstack networks</p> <p>createdL3VPN, which was associated, then disassociated from dualstack network can't be removed, because of exception below: </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2018-03-19T17:25:15,646 | INFO | ForkJoinPool-1-worker-1 | UpdateIdEntryJob | 251 - org.opendaylight.genius.idmanager-impl - 0.4.0.SNAPSHOT | Updated id entry with idValues [], idKey nexthop.10006311.1.1.2/32, pool nextHopPointerPool.174325811 2018-03-19T17:25:15,646 | TRACE | org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.vpn._interface.op.data.VpnInterfaceOpDataEntry_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-0 | VpnInterfaceOpListener | 367 - org.opendaylight.netvirt.vpnmanager-impl - 0.6.0.SNAPSHOT | VpnInterfaceOpListener updated: original VpnInterfaceOpDataEntry{getDpnId=119626733642592, getGatewayMacAddress=00:00:00:00:f5:ad, getLportTag=13, getName=403d9f6c-3db0-4db9-bbd4-892c383e6499, getVpnInstanceName=7ab8cd92-48ca-49b5-94e1-b2921a2616ab, getVpnInterfaceState=Active, isScheduledForRemove=true, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.AdjacenciesOp=AdjacenciesOp{getAdjacency=[Adjacency{getAdjacencyType=PrimaryAdjacency, getIpAddress=11.1.1.2/32, getLabel=100065, getMacAddress=fa:16:3e:69:6e:d4, getNextHopIpList=[10.100.0.11], getSubnetId=Uuid [_value=66be7703-7332-4c32-b375-cca8ab873963], getVrfId=3206:2, isPhysNetworkFunc=false, augmentations={}}, Adjacency{getAdjacencyType=PrimaryAdjacency, getIpAddress=2002:db7:0:2:f816:3eff:fe69:6ed4/128, getLabel=100067, getMacAddress=fa:16:3e:69:6e:d4, getNextHopIpList=[], getSubnetId=Uuid [_value=02c5bb95-6b46-4a8b-b3c4-6329d82178a5], getVrfId=3206:2, isPhysNetworkFunc=false, augmentations={}}]}}} updated VpnInterfaceOpDataEntry{getDpnId=119626733642592, getGatewayMacAddress=00:00:00:00:f5:ad, getLportTag=13, getName=403d9f6c-3db0-4db9-bbd4-892c383e6499, getVpnInstanceName=7ab8cd92-48ca-49b5-94e1-b2921a2616ab, getVpnInterfaceState=Active, isScheduledForRemove=true, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.AdjacenciesOp=AdjacenciesOp{getAdjacency=[Adjacency{getAdjacencyType=PrimaryAdjacency, getIpAddress=2002:db7:0:2:f816:3eff:fe69:6ed4/128, getLabel=100067, getMacAddress=fa:16:3e:69:6e:d4, getNextHopIpList=[], getSubnetId=Uuid [_value=02c5bb95-6b46-4a8b-b3c4-6329d82178a5], getVrfId=3206:2, isPhysNetworkFunc=false, augmentations={}}]}}} 2018-03-19T17:25:15,649 | INFO | ForkJoinPool-1-worker-0 | UpdateIdEntryJob | 251 - org.opendaylight.genius.idmanager-impl - 0.4.0.SNAPSHOT | Updated id entry with idValues [], idKey 3206:2.11.1.1.2/32, pool vpnservices.174325811 2018-03-19T17:25:16,059 | DEBUG | org.opendaylight.yang.gen.v1.urn.ericsson.params.xml.ns.yang.ebgp.rev150901.bgp.Vrfs_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-1 | BgpConfigurationManager | 339 - org.opendaylight.netvirt.bgpmanager-impl - 0.6.0.SNAPSHOT | received remove Vrfs config val 3206:2 2018-03-19T17:25:16,059 | DEBUG | org.opendaylight.yang.gen.v1.urn.ericsson.params.xml.ns.yang.ebgp.rev150901.bgp.Networks_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-0 | BgpConfigurationManager | 339 - org.opendaylight.netvirt.bgpmanager-impl - 0.6.0.SNAPSHOT | received remove Networks config val 2002:db7:0:2::/64 2018-03-19T17:25:16,060 | INFO | ResourceBatchingManager-0 | ResourceBatchingManager | 260 - org.opendaylight.genius.mdsalutil-api - 0.4.0.SNAPSHOT | Total taken ##time = 3ms for resourceList of size 3 for resourceType BGP-RESOURCES 2018-03-19T17:25:16,061 | DEBUG | org.opendaylight.yang.gen.v1.urn.ericsson.params.xml.ns.yang.ebgp.rev150901.bgp.Networks_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-0 | BgpRouter | 339 - org.opendaylight.netvirt.bgpmanager-impl - 0.6.0.SNAPSHOT | Deleting BGP route - rd:3206:2 prefix:2002:db7:0:2::/64 2018-03-19T17:25:16,061 | ERROR | org.opendaylight.yang.gen.v1.urn.ericsson.params.xml.ns.yang.ebgp.rev150901.bgp.Networks_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-0 | BgpConfigurationManager | 339 - org.opendaylight.netvirt.bgpmanager-impl - 0.6.0.SNAPSHOT | networks Delete received exception; Config store updated; undo with Add if needed. org.apache.thrift.TApplicationException: BGP RD 3206:2 not configured at org.apache.thrift.TApplicationException.read(TApplicationException.java:111) ~[145:org.apache.thrift:0.9.3] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79) [145:org.apache.thrift:0.9.3] at org.opendaylight.netvirt.bgpmanager.thrift.gen.BgpConfigurator$Client.recv_withdrawRoute(BgpConfigurator.java:406) [339:org.opendaylight.netvirt.bgpmanager-impl:0.6.0.SNAPSHOT] at org.opendaylight.netvirt.bgpmanager.thrift.gen.BgpConfigurator$Client.withdrawRoute(BgpConfigurator.java:387) [339:org.opendaylight.netvirt.bgpmanager-impl:0.6.0.SNAPSHOT] at org.opendaylight.netvirt.bgpmanager.thrift.client.BgpRouter.dispatchInternal(BgpRouter.java:306) [339:org.opendaylight.netvirt.bgpmanager-impl:0.6.0.SNAPSHOT] at org.opendaylight.netvirt.bgpmanager.thrift.client.BgpRouter.dispatch(BgpRouter.java:215) [339:org.opendaylight.netvirt.bgpmanager-impl:0.6.0.SNAPSHOT] at org.opendaylight.netvirt.bgpmanager.thrift.client.BgpRouter.delPrefix(BgpRouter.java:470) [339:org.opendaylight.netvirt.bgpmanager-impl:0.6.0.SNAPSHOT] at org.opendaylight.netvirt.bgpmanager.BgpConfigurationManager$NetworksReactor.remove(BgpConfigurationManager.java:1134) [339:org.opendaylight.netvirt.bgpmanager-impl:0.6.0.SNAPSHOT] at org.opendaylight.netvirt.bgpmanager.BgpConfigurationManager$NetworksReactor.remove(BgpConfigurationManager.java:1052) [339:org.opendaylight.netvirt.bgpmanager-impl:0.6.0.SNAPSHOT] at org.opendaylight.genius.datastoreutils.AsyncDataTreeChangeListenerBase$DataTreeChangeHandler.run(AsyncDataTreeChangeListenerBase.java:154) [260:org.opendaylight.genius.mdsalutil-api:0.4.0.SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?] at java.lang.Thread.run(Thread.java:748) [?:?] 2018-03-19T17:25:16,062 | DEBUG | org.opendaylight.yang.gen.v1.urn.ericsson.params.xml.ns.yang.ebgp.rev150901.bgp.Networks_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-0 | BgpConfigurationManager | 339 - org.opendaylight.netvirt.bgpmanager-impl - 0.6.0.SNAPSHOT | received remove Networks config val 11.1.1.2/32 2018-03-19T17:25:16,062 | DEBUG | org.opendaylight.yang.gen.v1.urn.ericsson.params.xml.ns.yang.ebgp.rev150901.bgp.Networks_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-0 | BgpRouter | 339 - org.opendaylight.netvirt.bgpmanager-impl - 0.6.0.SNAPSHOT | Deleting BGP route - rd:3206:2 prefix:11.1.1.2/32 2018-03-19T17:25:16,063 | ERROR | org.opendaylight.yang.gen.v1.urn.ericsson.params.xml.ns.yang.ebgp.rev150901.bgp.Networks_AsyncDataTreeChangeListenerBase-DataTreeChangeHandler-0 | BgpConfigurationManager | 339 - org.opendaylight.netvirt.bgpmanager-impl - 0.6.0.SNAPSHOT | networks Delete received exception; Config store updated; undo with Add if needed. org.apache.thrift.TApplicationException: BGP RD 3206:2 not configured</pre> </div></div> <p> As we can see from trace, exception happens, when we execute associateNetworks RPC call</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=valentina.krasnobaeva" class="user-hover" rel="valentina.krasnobaeva">valentina.krasnobaeva</a> - I see you've re-opened this bug. Note that it's currently marked as a blocker against Oxygen, and that it's far too late to wait for a fix. Unless a fix is going to be merged immediately I'm going to recommend to the TSC that we don't delay the Oxygen release for this. We need to get an autorelease build (getting lucky and avoiding all "sporadic" failures) and have all projects sign-off on the test results by Wednesday. /cc <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=shague" class="user-hover" rel="shague">shague</a></p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=valentina.krasnobaeva" class="user-hover" rel="valentina.krasnobaeva">valentina.krasnobaeva</a> is there a CSIT test failure or are these just exceptions coming out from manual testing? As <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=dfarrell07" class="user-hover" rel="dfarrell07">dfarrell07</a> mentions this bug was tagged as a blocker so reopening it makes visible to the oxygen release again. Can we lower this to critical?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=valentina.krasnobaeva" class="user-hover" rel="valentina.krasnobaeva">valentina.krasnobaeva</a> - I'm going to go ahead and lower this to critical. I don't think it's realistic to get a fix in for Oxygen. It would have to be a massive regression to justify waiting for it at all.</p> <p><a href="https://git.opendaylight.org/gerrit/70421" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/70421</a></p> Development Rank 0|i03ai7: