|
Here is the original PUT symptom exception. This does not happen regularly, and the failure is usually hidden by failure in preceding test case.
Sandbox link: https://logs.opendaylight.org/sandbox/jenkins091/netconf-csit-3node-clustering-only-carbon/2/archives/log.html.gz#s1-s4-t12-k2-k1-k3-k7-k1
2016-09-05 11:00:30,639 | WARN | lt-dispatcher-16 | ReliableDeliverySupervisor | 202 - com.typesafe.akka.slf4j - 2.4.7 | Association with remote system [akka.tcp://opendaylight-cluster-data@10.29.12.160:2550] has failed, address is now gated for [5000] ms. Reason: [Disassociated]
2016-09-05 11:00:35,046 | INFO | ult-dispatcher-4 | ShardManager | 219 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Received UnreachableMember: memberName MemberName
{name=member-2}, address: akka.tcp://opendaylight-cluster-data@10.29.12.160:2550
2016-09-05 11:00:35,046 | INFO | lt-dispatcher-21 | BaseTopologyManager | 299 - org.opendaylight.netconf.abstract-topology - 1.2.0.SNAPSHOT | Member is unreachable: Member(address = akka.tcp://opendaylight-cluster-data@10.29.12.160:2550, status = Up)
2016-09-05 11:00:35,046 | INFO | lt-dispatcher-21 | ShardManager | 219 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Received UnreachableMember: memberName MemberName{name=member-2}
, address: akka.tcp://opendaylight-cluster-data@10.29.12.160:2550
2016-09-05 11:00:35,046 | INFO | ult-dispatcher-4 | EntityOwnershipShard | 214 - org.opendaylight.controller.sal-akka-raft - 1.5.0.SNAPSHOT | member-3-shard-entity-ownership-operational: onPeerDown: PeerDown [memberName=member-2, peerId=member-2-shard-entity-ownership-operational]
2016-09-05 11:00:35,505 | WARN | lt-dispatcher-18 | ClusterCoreDaemon | 202 - com.typesafe.akka.slf4j - 2.4.7 | Cluster Node [akka.tcp://opendaylight-cluster-data@10.29.12.58:2550] - Marking node(s) as UNREACHABLE [Member(address = akka.tcp://opendaylight-cluster-data@10.29.12.160:2550, status = Up)]. Node roles [member-3]
2016-09-05 11:00:35,851 | WARN | lt-dispatcher-20 | ReliableDeliverySupervisor | 202 - com.typesafe.akka.slf4j - 2.4.7 | Association with remote system [akka.tcp://opendaylight-cluster-data@10.29.12.160:2550] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://opendaylight-cluster-data@10.29.12.160:2550]] Caused by: [Connection refused: /10.29.12.160:2550]
2016-09-05 11:00:37,457 | WARN | lt-dispatcher-22 | BrokerFacade | 269 - org.opendaylight.netconf.sal-rest-connector - 1.5.0.SNAPSHOT | Exception by reading CONFIGURATION via Restconf: /(urn:opendaylight:test:netconf:crud?revision=2014-10-18)cont
akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://opendaylight-cluster-data@10.29.12.160:2550/user/topology-netconf/netconf-test-device/mountpoint#1693594407]] after [5000 ms]. Sender[null] sent message of type "akka.actor.TypedActor$MethodCall".
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)[201:com.typesafe.akka.actor:2.4.7]
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:331)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:282)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:286)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:238)[201:com.typesafe.akka.actor:2.4.7]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_101]
2016-09-05 11:00:37,461 | ERROR | lt-dispatcher-22 | ExecutionList | 84 - com.google.guava - 18.0.0 | RuntimeException while executing runnable com.google.common.util.concurrent.Futures$6@49064bc1 with executor INSTANCE
org.opendaylight.netconf.sal.restconf.impl.RestconfDocumentedException: errors: [RestconfError [error-type: application, error-tag: operation-failed, error-message: Problem to get data from transaction., error-info: akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://opendaylight-cluster-data@10.29.12.160:2550/user/topology-netconf/netconf-test-device/mountpoint#1693594407]] after [5000 ms]. Sender[null] sent message of type "akka.actor.TypedActor$MethodCall".
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)
at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:331)
at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:282)
at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:286)
at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:238)
at java.lang.Thread.run(Thread.java:745)
]]
at org.opendaylight.netconf.sal.restconf.impl.BrokerFacade.handlingCallback(BrokerFacade.java:612)[269:org.opendaylight.netconf.sal-rest-connector:1.5.0.SNAPSHOT]
at org.opendaylight.netconf.sal.restconf.impl.BrokerFacade$2.onFailure(BrokerFacade.java:386)[269:org.opendaylight.netconf.sal-rest-connector:1.5.0.SNAPSHOT]
at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)[84:com.google.guava:18.0.0]
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)[84:com.google.guava:18.0.0]
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)[84:com.google.guava:18.0.0]
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)[84:com.google.guava:18.0.0]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)[84:com.google.guava:18.0.0]
at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:68)[84:com.google.guava:18.0.0]
at org.opendaylight.netconf.topology.pipeline.tx.ProxyReadOnlyTransaction$2.onComplete(ProxyReadOnlyTransaction.java:73)[298:org.opendaylight.netconf.topology:1.2.0.SNAPSHOT]
at org.opendaylight.netconf.topology.pipeline.tx.ProxyReadOnlyTransaction$2.onComplete(ProxyReadOnlyTransaction.java:57)[298:org.opendaylight.netconf.topology:1.2.0.SNAPSHOT]
at akka.dispatch.OnComplete.internal(Future.scala:258)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.OnComplete.internal(Future.scala:256)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:186)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:183)[201:com.typesafe.akka.actor:2.4.7]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)[201:com.typesafe.akka.actor:2.4.7]
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)[201:com.typesafe.akka.actor:2.4.7]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:409)[201:com.typesafe.akka.actor:2.4.7]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
Caused by: akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://opendaylight-cluster-data@10.29.12.160:2550/user/topology-netconf/netconf-test-device/mountpoint#1693594407]] after [5000 ms]. Sender[null] sent message of type "akka.actor.TypedActor$MethodCall".
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:604)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:126)[201:com.typesafe.akka.actor:2.4.7]
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)[197:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:331)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:282)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:286)[201:com.typesafe.akka.actor:2.4.7]
at akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:238)[201:com.typesafe.akka.actor:2.4.7]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_101]
|