https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/BGPCEP-588 bgpcep <p>Seen sporadically both in CI and local environment</p> <p><a href="https://jenkins.opendaylight.org/releng/job/bgpcep-verify-boron-mvn33-openjdk8/352/console" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/job/bgpcep-verify-boron-mvn33-openjdk8/352/console</a></p> <p>07:04:46 Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.058 sec - in org.opendaylight.protocol.bgp.rib.impl.AddPathBasePathsTest<br/> 07:04:47 Running org.opendaylight.protocol.bgp.rib.impl.AddPathNPathsTest<br/> 12:49:15 Build timed out (after 360 minutes). Marking the build as failed.<br/> 12:49:15 <span class="error">&#91;ssh-agent&#93;</span> Stopped.<br/> 12:49:15 Build was aborted</p> <p>Console output is attached. HOLD_TIMER_EXPIRED message is reported indicating that somehow keepalives are not getting exchanged correctly (e.g. lock-up in sending/receiving messages)</p> <p>Operating System: All<br/> Platform: All</p> BGPCEP-588

AddPath unit-tests sometimes hangs during execution

Bug Resolved Done Ajay L Ajay L Fri, 21 Oct 2016 01:41:38 +0000 Sun, 3 Mar 2019 11:49:56 +0000 Tue, 8 Nov 2016 14:26:58 +0000 Bugzilla Migration Bugzilla Migration BGP 0 2 <p>Attachment org.opendaylight.protocol.bgp.rib.impl.AddPathNPathsTest-output.txt has been added with description: Fail UT console output</p> <p>Thread dump when the test is in hung state is below. Channel#write() seems to get stuck if peer connection is closed at the same time and this causes EventLoopGroup#shutdownGracefully().awaitUninterruptibly() called from main() to block</p> <p>Name: main<br/> State: WAITING on io.netty.util.concurrent.DefaultPromise@6f5d2c39<br/> Total blocked: 7 Total waited: 28</p> <p>Stack trace: <br/> java.lang.Object.wait(Native Method)<br/> java.lang.Object.wait(Object.java:502)<br/> io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:258)<br/> io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:34)<br/> org.opendaylight.protocol.bgp.rib.impl.BGPDispatcherImpl.close(BGPDispatcherImpl.java:126)<br/> org.opendaylight.protocol.bgp.rib.impl.AbstractAddPathTest.tearDown(AbstractAddPathTest.java:194)<br/> org.opendaylight.protocol.bgp.rib.impl.AddPathNPathsTest.tearDown(AddPathNPathsTest.java:34)<br/> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br/> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)<br/> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)<br/> java.lang.reflect.Method.invoke(Method.java:498)<br/> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)<br/> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)<br/> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)<br/> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)<br/> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)<br/> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)<br/> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)<br/> org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)<br/> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)<br/> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)<br/> org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)<br/> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)<br/> org.junit.runners.ParentRunner.run(ParentRunner.java:309)<br/> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)<br/> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)<br/> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)<br/> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)<br/> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)<br/> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)<br/> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)</p> <p>Name: epollEventLoopGroup-5-2<br/> State: BLOCKED on org.opendaylight.protocol.bgp.rib.impl.BGPSessionImpl@60d922f7 owned by: epollEventLoopGroup-5-5<br/> Total blocked: 3 Total waited: 8</p> <p>Stack trace: <br/> org.opendaylight.protocol.bgp.rib.impl.BGPSessionImpl.write(BGPSessionImpl.java)<br/> org.opendaylight.protocol.bgp.rib.impl.ChannelOutputLimiter.write(ChannelOutputLimiter.java:54)<br/> org.opendaylight.protocol.bgp.rib.impl.AdjRibOutListener.processRouteChange(AdjRibOutListener.java:128)<br/> org.opendaylight.protocol.bgp.rib.impl.AdjRibOutListener.processSupportedFamilyRoutes(AdjRibOutListener.java:102)<br/> org.opendaylight.protocol.bgp.rib.impl.AdjRibOutListener.onDataTreeChanged(AdjRibOutListener.java:94)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher$1.invokeListener(InMemoryDOMStoreTreeChangePublisher.java:38)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher$1.invokeListener(InMemoryDOMStoreTreeChangePublisher.java:32)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.lambda$null$0(QueuedNotificationManager.java:125)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$$Lambda$77/284438421.accept(Unknown Source)<br/> java.lang.Iterable.forEach(Iterable.java:75)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.lambda$new$1(QueuedNotificationManager.java:123)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$$Lambda$29/796240305.invokeListener(Unknown Source)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$NotificationTask.invokeListener(QueuedNotificationManager.java:432)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$NotificationTask.run(QueuedNotificationManager.java:420)<br/> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.runTask(QueuedNotificationManager.java:252)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.submitNotifications(QueuedNotificationManager.java:202)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.submitNotification(QueuedNotificationManager.java:167)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher.notifyListeners(InMemoryDOMStoreTreeChangePublisher.java:55)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:118)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.lookupAndNotify(AbstractDOMStoreTreeChangePublisher.java:106)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.processCandidateTree(AbstractDOMStoreTreeChangePublisher.java:67)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher.publishChange(InMemoryDOMStoreTreeChangePublisher.java:80)</p> <ul class="alternate" type="square"> <li>locked org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher@6c64c55<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore.commit(InMemoryDOMDataStore.java:249)</li> <li>locked org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore@4f6ce954<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreThreePhaseCommitCohort.commit(InMemoryDOMStoreThreePhaseCommitCohort.java:105)<br/> org.opendaylight.controller.md.sal.dom.store.impl.ChainedTransactionCommitImpl.commit(ChainedTransactionCommitImpl.java:26)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.commitAll(CommitCoordinationTask.java:211)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.commitBlocking(CommitCoordinationTask.java:186)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.call(CommitCoordinationTask.java:65)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.call(CommitCoordinationTask.java:29)<br/> java.util.concurrent.FutureTask.run(FutureTask.java:266)<br/> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)<br/> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)<br/> com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:58)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.SerializedDOMDataBroker.submit(SerializedDOMDataBroker.java:70)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.DOMDataBrokerTransactionChainImpl.submit(DOMDataBrokerTransactionChainImpl.java:97)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.DOMForwardedWriteTransaction.submit(DOMForwardedWriteTransaction.java:144)<br/> org.opendaylight.protocol.bgp.rib.impl.LocRibWriter.onDataTreeChanged(LocRibWriter.java:143)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher$1.invokeListener(InMemoryDOMStoreTreeChangePublisher.java:38)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher$1.invokeListener(InMemoryDOMStoreTreeChangePublisher.java:32)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.lambda$null$0(QueuedNotificationManager.java:125)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$$Lambda$77/284438421.accept(Unknown Source)<br/> java.lang.Iterable.forEach(Iterable.java:75)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.lambda$new$1(QueuedNotificationManager.java:123)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$$Lambda$29/796240305.invokeListener(Unknown Source)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$NotificationTask.invokeListener(QueuedNotificationManager.java:432)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$NotificationTask.run(QueuedNotificationManager.java:420)<br/> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.runTask(QueuedNotificationManager.java:252)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.submitNotifications(QueuedNotificationManager.java:202)<br/> org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager.submitNotification(QueuedNotificationManager.java:167)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher.notifyListeners(InMemoryDOMStoreTreeChangePublisher.java:55)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:118)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:129)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.notifyNode(AbstractDOMStoreTreeChangePublisher.java:125)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.lookupAndNotify(AbstractDOMStoreTreeChangePublisher.java:106)<br/> org.opendaylight.controller.sal.core.spi.data.AbstractDOMStoreTreeChangePublisher.processCandidateTree(AbstractDOMStoreTreeChangePublisher.java:67)<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher.publishChange(InMemoryDOMStoreTreeChangePublisher.java:80)</li> <li>locked org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreTreeChangePublisher@6c64c55<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore.commit(InMemoryDOMDataStore.java:249)</li> <li>locked org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore@4f6ce954<br/> org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMStoreThreePhaseCommitCohort.commit(InMemoryDOMStoreThreePhaseCommitCohort.java:105)<br/> org.opendaylight.controller.md.sal.dom.store.impl.ChainedTransactionCommitImpl.commit(ChainedTransactionCommitImpl.java:26)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.commitAll(CommitCoordinationTask.java:211)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.commitBlocking(CommitCoordinationTask.java:186)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.call(CommitCoordinationTask.java:65)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.CommitCoordinationTask.call(CommitCoordinationTask.java:29)<br/> java.util.concurrent.FutureTask.run(FutureTask.java:266)<br/> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)<br/> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)<br/> com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:58)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.SerializedDOMDataBroker.submit(SerializedDOMDataBroker.java:70)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.DOMDataBrokerTransactionChainImpl.submit(DOMDataBrokerTransactionChainImpl.java:97)<br/> org.opendaylight.controller.md.sal.dom.broker.impl.DOMForwardedWriteTransaction.submit(DOMForwardedWriteTransaction.java:144)<br/> org.opendaylight.protocol.bgp.rib.impl.AdjRibInWriter.removePeer(AdjRibInWriter.java:234)<br/> org.opendaylight.protocol.bgp.rib.impl.BGPPeer.cleanup(BGPPeer.java:369)<br/> org.opendaylight.protocol.bgp.rib.impl.BGPPeer.releaseConnection(BGPPeer.java:413)</li> <li>locked org.opendaylight.protocol.bgp.rib.impl.BGPPeer@25184d46<br/> org.opendaylight.protocol.bgp.rib.impl.BGPPeer.onSessionDown(BGPPeer.java:381)<br/> org.opendaylight.protocol.bgp.rib.impl.BGPSessionImpl.endOfInput(BGPSessionImpl.java:283)</li> <li>locked org.opendaylight.protocol.bgp.rib.impl.BGPSessionImpl@5ca3b073<br/> org.opendaylight.protocol.bgp.rib.impl.BGPSessionImpl.channelInactive(BGPSessionImpl.java:481)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:219)<br/> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:212)<br/> io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:360)<br/> io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:325)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:219)<br/> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:212)<br/> io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:360)<br/> io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:325)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:219)<br/> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:212)<br/> io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1275)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233)<br/> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:219)<br/> io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:872)<br/> io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:679)<br/> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)<br/> io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:307)<br/> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)<br/> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)<br/> java.lang.Thread.run(Thread.java:745)</li> </ul> <p><a href="https://git.opendaylight.org/gerrit/#/c/47885/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/47885/</a></p> <p>Boron<br/> <a href="https://git.opendaylight.org/gerrit/#/c/48107/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/48107/</a></p> Development External issue ID 7006 External issue URL Issue Type ODL SR Target Milestone Rank 0|i02d7r: