https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/CONTROLLER-799 controller <p>The following exception is seen when clustering is started up (after being installed).</p> <p>Exception in thread "pool-37-thread-4" org.opendaylight.controller.cluster.datastore.exceptions.TimeoutException: Sending message class org.opendaylight.controller.cluster.datastore.messages.RegisterChangeListener to actor Actor<a href="#-2143120719" target="_blank" rel="noopener">akka://opendaylight-cluster-data/user/shardmanager-config/member-1-shard-inventory-config#-2143120719</a> failed<br/> at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeLocalOperation(ActorContext.java:166)<br/> at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeLocalShardOperation(ActorContext.java:254)<br/> at org.opendaylight.controller.cluster.datastore.DistributedDataStore.registerChangeListener(DistributedDataStore.java:85)<br/> at org.opendaylight.controller.md.sal.dom.broker.impl.DOMDataBrokerImpl.registerDataChangeListener(DOMDataBrokerImpl.java:84)<br/> at org.opendaylight.controller.md.sal.dom.broker.impl.DOMDataBrokerImpl.registerDataChangeListener(DOMDataBrokerImpl.java:36)<br/> at org.opendaylight.controller.sal.dom.broker.osgi.DOMDataBrokerProxy.registerDataChangeListener(DOMDataBrokerProxy.java:40)<br/> at org.opendaylight.controller.sal.dom.broker.osgi.DOMDataBrokerProxy.registerDataChangeListener(DOMDataBrokerProxy.java:15)<br/> at org.opendaylight.controller.md.sal.binding.impl.AbstractForwardedDataBroker.registerDataChangeListener(AbstractForwardedDataBroker.java:81)<br/> at org.opendaylight.controller.md.sal.binding.impl.ForwardedBackwardsCompatibleDataBroker.registerDataChangeListener(ForwardedBackwardsCompatibleDataBroker.java:116)<br/> at org.opendaylight.controller.md.sal.binding.util.AbstractBindingSalConsumerInstance.registerDataChangeListener(AbstractBindingSalConsumerInstance.java:105)<br/> at org.opendaylight.controller.md.statistics.manager.StatisticsProvider.start(StatisticsProvider.java:100)<br/> at org.opendaylight.controller.md.statistics.manager.StatisticsManagerActivator.onSessionInitiated(StatisticsManagerActivator.java:26)<br/> at org.opendaylight.controller.md.sal.binding.util.BindingContextUtils.createProviderContextAndInitialize(BindingContextUtils.java:57)<br/> at org.opendaylight.controller.sal.binding.impl.RootBindingAwareBroker.registerProvider(RootBindingAwareBroker.java:146)<br/> at org.opendaylight.controller.sal.binding.api.AbstractBindingAwareProvider.onBrokerAvailable(AbstractBindingAwareProvider.java:22)<br/> at org.opendaylight.controller.sal.binding.api.AbstractBrokerAwareActivator$1$1.run(AbstractBrokerAwareActivator.java:34)<br/> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)<br/> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<br/> at java.lang.Thread.run(Thread.java:745)<br/> Caused by: java.util.concurrent.TimeoutException: Futures timed out after <span class="error">&#91;5 seconds&#93;</span><br/> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)<br/> at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)<br/> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)<br/> at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)<br/> at scala.concurrent.Await$.result(package.scala:107)<br/> at scala.concurrent.Await.result(package.scala)<br/> at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeLocalOperation(ActorContext.java:164)</p> <p>Operating System: Mac OS<br/> Platform: PC</p> CONTROLLER-799

Clustering : Exception seen at clustering startup in karaf distribution

Bug Resolved Done Harman Singh Jan Medved Wed, 10 Sep 2014 05:55:31 +0000 Tue, 28 Oct 2014 15:10:14 +0000 Tue, 28 Oct 2014 15:10:14 +0000 Helium mdsal 0 6 <p>And the following exceptions are being seen when the controller exits:Exception in thread "pool-37-thread-4" org.opendaylight.controller.cluster.datastore.exceptions.TimeoutException: Sending message class org.opendaylight.controller.cluster.datastore.messages.RegisterChangeListener to actor Actor<a href="#-2143120719" target="_blank" rel="noopener">akka://opendaylight-cluster-data/user/shardmanager-config/member-1-shard-inventory-config#-2143120719</a> failed<br/> at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeLocalOperation(ActorContext.java:166)<br/> at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeLocalShardOperation(ActorContext.java:254)<br/> at org.opendaylight.controller.cluster.datastore.DistributedDataStore.registerChangeListener(DistributedDataStore.java:85)<br/> at org.opendaylight.controller.md.sal.dom.broker.impl.DOMDataBrokerImpl.registerDataChangeListener(DOMDataBrokerImpl.java:84)<br/> at org.opendaylight.controller.md.sal.dom.broker.impl.DOMDataBrokerImpl.registerDataChangeListener(DOMDataBrokerImpl.java:36)<br/> at org.opendaylight.controller.sal.dom.broker.osgi.DOMDataBrokerProxy.registerDataChangeListener(DOMDataBrokerProxy.java:40)<br/> at org.opendaylight.controller.sal.dom.broker.osgi.DOMDataBrokerProxy.registerDataChangeListener(DOMDataBrokerProxy.java:15)<br/> at org.opendaylight.controller.md.sal.binding.impl.AbstractForwardedDataBroker.registerDataChangeListener(AbstractForwardedDataBroker.java:81)<br/> at org.opendaylight.controller.md.sal.binding.impl.ForwardedBackwardsCompatibleDataBroker.registerDataChangeListener(ForwardedBackwardsCompatibleDataBroker.java:116)<br/> at org.opendaylight.controller.md.sal.binding.util.AbstractBindingSalConsumerInstance.registerDataChangeListener(AbstractBindingSalConsumerInstance.java:105)<br/> at org.opendaylight.controller.md.statistics.manager.StatisticsProvider.start(StatisticsProvider.java:100)<br/> at org.opendaylight.controller.md.statistics.manager.StatisticsManagerActivator.onSessionInitiated(StatisticsManagerActivator.java:26)<br/> at org.opendaylight.controller.md.sal.binding.util.BindingContextUtils.createProviderContextAndInitialize(BindingContextUtils.java:57)<br/> at org.opendaylight.controller.sal.binding.impl.RootBindingAwareBroker.registerProvider(RootBindingAwareBroker.java:146)<br/> at org.opendaylight.controller.sal.binding.api.AbstractBindingAwareProvider.onBrokerAvailable(AbstractBindingAwareProvider.java:22)<br/> at org.opendaylight.controller.sal.binding.api.AbstractBrokerAwareActivator$1$1.run(AbstractBrokerAwareActivator.java:34)<br/> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)<br/> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<br/> at java.lang.Thread.run(Thread.java:745)<br/> Caused by: java.util.concurrent.TimeoutException: Futures timed out after <span class="error">&#91;5 seconds&#93;</span><br/> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)<br/> at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)<br/> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)<br/> at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)<br/> at scala.concurrent.Await$.result(package.scala:107)<br/> at scala.concurrent.Await.result(package.scala)<br/> at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeLocalOperation(ActorContext.java:164)</p> <p>Trying to get the inventory nodes via Restconf from config space results in the following error:</p> <p>errors: [error-type: application, error-tag: operation-failed, error-message: Problem to get data from transaction.error-info: ReadFailedException</p> {message=Error reading data for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes, errorList=[RpcError [message=Error reading data for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.datastore.exceptions.TimeoutException: Sending message class org.opendaylight.controller.protobuff.messages.transaction.ShardTransactionMessages$CreateTransaction to actor ActorSelection[Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-config/member-1-shard-inventory-config)] failed]]} <p> at org.opendaylight.controller.cluster.datastore.TransactionProxy$NoOpTransactionContext.readData(TransactionProxy.java:762) at org.opendaylight.controller.cluster.datastore.TransactionProxy.read(TransactionProxy.java:232) at org.opendaylight.controller.md.sal.dom.broker.impl.DOMForwardedReadOnlyTransaction.read(DOMForwardedReadOnlyTransaction.java:40) at org.opendaylight.controller.sal.restconf.impl.BrokerFacade.readDataViaTransaction(BrokerFacade.java:181) at org.opendaylight.controller.sal.restconf.impl.BrokerFacade.readConfigurationData(BrokerFacade.java:77) at org.opendaylight.controller.sal.restconf.impl.RestconfImpl.readConfigurationData(RestconfImpl.java:631) at org.opendaylight.controller.sal.restconf.impl.StatisticsRestconfServiceWrapper.readConfigurationData(StatisticsRestconfServiceWrapper.java:84) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716) at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:69) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:240) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:77) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.TimeoutException: Sending message class org.opendaylight.controller.protobuff.messages.transaction.ShardTransactionMessages$CreateTransaction to actor ActorSelection<span class="error">&#91;Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-config/member-1-shard-inventory-config)&#93;</span> failed at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeRemoteOperation(ActorContext.java:186) at org.opendaylight.controller.cluster.datastore.utils.ActorContext.executeShardOperation(ActorContext.java:232) at org.opendaylight.controller.cluster.datastore.TransactionProxy.createTransactionIfMissing(TransactionProxy.java:355) at org.opendaylight.controller.cluster.datastore.TransactionProxy.read(TransactionProxy.java:230) ... 51 more Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection<span class="error">&#91;Anchor(akka://opendaylight-cluster-data/), Path(/user/shardmanager-config/member-1-shard-inventory-config)&#93;</span>] after <span class="error">&#91;5000 ms&#93;</span> at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:333) at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117) at scala.concurrent.Future$InternalCallbackExecutor$.scala$concurrent$Future$InternalCallbackExecutor$$unbatchedExecute(Future.scala:694) at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:691) at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467) at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419) at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423) at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375) ... 1 more , ]]</p> <p>The RegisterDataChangeListener exception and the CreateTransaction exceptions both happen because the Shard is not ready for business. On startup this typically happens because the Shard is recovering.</p> <p>Hi Medved,<br/> I didn't get any exception while running controller in cluster.<br/> I followed this link to enable clustering,<br/> <a href="https://wiki.opendaylight.org/view/Running_and_testing_an_OpenDaylight_Cluster" class="external-link" target="_blank" rel="nofollow noopener">https://wiki.opendaylight.org/view/Running_and_testing_an_OpenDaylight_Cluster</a><br/> I downloaded helium binary from this link,<br/> <a href="http://www.opendaylight.org/software/downloads" class="external-link" target="_blank" rel="nofollow noopener">http://www.opendaylight.org/software/downloads</a><br/> Is there is anyother steps to run the cluster? Am I having the right controller? Where am I missing?</p> <p>The exceptions on startup due to the shard not ready have been alleviated with recent patches. The registerChangeListener and createTransaction operations now wait a period of time for the shard to recover and for shard leader election to complete. The exceptions were most likely caused by shard persistence recovery taking a long time. Recovery time has also been improved since. </p> <p>I think we can close this bug.</p> <p>Not reproducible any more</p> Blocks CONTROLLER-704 Development External issue ID 1780 External issue URL ODL SR Target Milestone Priority Rank 0|i02mmn: