OpenDaylight JIRA

OpenDaylight JIRA https://jira.opendaylight.org This file is an XML representation of an issue en-us

8.20.10

820010

22-06-2022

[BGPCEP-601] B and C: rejoining isolated node causes exabgp unable to connect to any cluster node https://jira.opendaylight.org/browse/BGPCEP-601 bgpcep <p>The problem observed in <a href="https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-3node-periodic-bgpclustering-ha-only-boron/86" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-3node-periodic-bgpclustering-ha-only-boron/86</a> and <a href="https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-3node-periodic-bgpclustering-ha-only-carbon/82/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-3node-periodic-bgpclustering-ha-only-carbon/82/</a></p> <p>The suite with karaf restart works well, but the suite when the bgp leader is isolated does not work well.<br/> <a href="https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-ha-only-boron/86/archives/" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-ha-only-boron/86/archives/</a><br/> Robot logs shows that ExaGbp is connected well to node3 (10.29.13.32). <br/> Connected to peer neighbor 10.29.13.32 local-ip 10.29.13.14 local-as 64496 peer-as 64496 router-id 10.29.13.14 family-allowed in-open (out)<br/> After the node isolation it reconnected to node2(10.29.12.215), but connection was broken just after.<br/> Connected to peer neighbor 10.29.12.215 local-ip 10.29.13.14 local-as 64496 peer-as 64496 router-id 10.29.13.14 family-allowed in-open (out)<br/> Peer 10.29.12.215 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,0)]</span> error<span class="error">[Cease / Unspecific]</span></p> <p>Tue, 22 Nov 2016 03:44:18 | INFO | 22747 | reactor | New peer setup: neighbor 10.29.13.32 local-ip 10.29.13.14 local-as 64496 peer-as 64496 router-id 10.29.13.14 family-allowed in-open<br/> Tue, 22 Nov 2016 03:44:18 | INFO | 22747 | reactor | New peer setup: neighbor 10.29.12.30 local-ip 10.29.13.14 local-as 64496 peer-as 64496 router-id 10.29.13.14 family-allowed in-open<br/> Tue, 22 Nov 2016 03:44:18 | INFO | 22747 | reactor | New peer setup: neighbor 10.29.12.215 local-ip 10.29.13.14 local-as 64496 peer-as 64496 router-id 10.29.13.14 family-allowed in-open<br/> Tue, 22 Nov 2016 03:44:18 | WARNING | 22747 | configuration | Loaded new configuration successfully<br/> Tue, 22 Nov 2016 03:44:21 | INFO | 22747 | network | Peer 10.29.12.30 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:21 | INFO | 22747 | network | Peer 10.29.12.215 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:21 | INFO | 22747 | network | Connected to peer neighbor 10.29.13.32 local-ip 10.29.13.14 local-as 64496 peer-as 64496 router-id 10.29.13.14 family-allowed in-open (out)<br/> Tue, 22 Nov 2016 03:44:25 | INFO | 22747 | network | Peer 10.29.12.30 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:26 | INFO | 22747 | network | Peer 10.29.12.215 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:30 | INFO | 22747 | network | Peer 10.29.12.30 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:31 | INFO | 22747 | network | Peer 10.29.12.215 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:35 | INFO | 22747 | network | Peer 10.29.12.30 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:36 | INFO | 22747 | network | Peer 10.29.12.215 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:41 | INFO | 22747 | network | Peer 10.29.12.30 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,5)]</span> error<span class="error">[Cease / Connection Rejected]</span><br/> Tue, 22 Nov 2016 03:44:42 | INFO | 22747 | network | Connected to peer neighbor 10.29.12.215 local-ip 10.29.13.14 local-as 64496 peer-as 64496 router-id 10.29.13.14 family-allowed in-open (out)<br/> Tue, 22 Nov 2016 03:44:43 | INFO | 22747 | network | Peer 10.29.12.215 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,0)]</span> error<span class="error">[Cease / Unspecific]</span><br/> Tue, 22 Nov 2016 03:44:43 | INFO | 22747 | network | Peer 10.29.13.32 ASN 64496 out loop, peer reset, message <span class="error">[notification received (6,0)]</span> error<span class="error">[Cease / Unspecific]</span></p> <p>In the logs of node 2 we can find<br/> 2016-11-22 03:44:42,074 | INFO | entLoopGroup-8-6 | AbstractBGPSessionNegotiator | 264 - org.opendaylight.bgpcep.bgp-rib-impl - 0.6.2.SNAPSHOT | BGP Session with peer <span class="error">[id: 0x059dff3e, L:/10.29.12.215:1790 - R:/10.29.13.14:42429]</span> established successfully.<br/> ...<br/> 2016-11-22 03:44:43,120 | WARN | ult-dispatcher-2 | ConcurrentDOMDataBroker | 172 - org.opendaylight.controller.sal-distributed-datastore - 1.4.2.SNAPSHOT | Tx: DOM-CHAIN-12-1 Error during phase CAN_COMMIT, starting Abort<br/> OptimisticLockFailedException</p> {message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was deleted by other transaction.]]} <p>So it looks like the peer configuration was deleted and should not have been done.</p> <p>This bug may be similar to <a href="https://bugs.opendaylight.org/show_bug.cgi?id=6788" class="external-link" target="_blank" rel="nofollow noopener">https://bugs.opendaylight.org/show_bug.cgi?id=6788</a>, but logs dont show the same problem.</p> <p>Operating System: All<br/> Platform: All</p> BGPCEP-601

B and C: rejoining isolated node causes exabgp unable to connect to any cluster node

Bug Resolved Done Claudio David Gasparini Peter Gubka Tue, 22 Nov 2016 15:10:02 +0000 Sun, 3 Mar 2019 11:49:57 +0000 Mon, 20 Feb 2017 16:09:21 +0000 Bugzilla Migration Bugzilla Migration BGP 0 3 <p>partial fix<br/> master<br/> <a href="https://git.opendaylight.org/gerrit/#/c/48732/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/48732/</a><br/> Boron<br/> <a href="https://git.opendaylight.org/gerrit/#/c/48767/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/48767/</a></p> <p>the error is cause of BUG-7322. Once its fixed we shouldn't longer experiment this wrong behavior under BGP.</p> <p>The issue is that the bgp services are trying to access the datastore to delete some data once the service managed by singleton tries to close them. Since the datastore cant make any progress once isolated the futures fail with unchecked exceptions which seem to prevent callbacks to other services.</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/51621/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/51621/</a> should improve unchecked exception handling in singleton service, but the underlying issue that the service is trying to write something into the datastore when it cant make progress doesn't seem right. If you need to cleanup the datastore maybe when the service is started it could do the cleanup as the first thing?</p> <p>master<br/> <a href="https://git.opendaylight.org/gerrit/#/c/51893/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/51893/</a></p> <p>boron<br/> <a href="https://git.opendaylight.org/gerrit/#/c/51974/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/51974/</a></p> Blocks BGPCEP-392 MDSAL-212 Development External References bgpcep_bgpclustering_ha_txt_050_bgp_ha_karaf_isolation_verify_exabgp_still_connected External issue ID 7222 External issue URL Issue Type ODL SR Target Milestone Rank 0|i02dan: