Uploaded image for project: 'netconf'
  1. netconf
  2. NETCONF-131

Netconf crash configuring device on clustered setup: "All available providers exhausted"

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • None
    • None
    • netconf
    • None
    • Operating System: All
      Platform: All

    • 5089

      Tested on stable/beryllium running on a three node cluster.

      After trying to configure a Netconf device on clustered setup, the Netconf crashes with the following errors:

      2016-01-24 02:44:03,837 | WARN | ult-dispatcher-2 | ClusteredNetconfDevice | 242 - org.opendaylight.netconf.topology - 1.0.0.SNAPSHOT | RemoteDevice

      {netconf-test-device}: Unexpected error resolving device sources: org.opendaylight.yangtools.yang.model.repo.api.MissingSchemaSourceException: All available providers exhausted
      2016-01-24 02:44:03,839 | ERROR | lt-dispatcher-27 | Futures$CombinedFuture | 39 - com.google.guava - 18.0.0 | input future failed.
      org.opendaylight.yangtools.yang.model.repo.api.MissingSchemaSourceException: All available providers exhausted
      at org.opendaylight.yangtools.yang.model.repo.util.AbstractSchemaRepository$1.create(AbstractSchemaRepository.java:83)
      at com.google.common.util.concurrent.Futures$FallbackFuture$1.onFailure(Futures.java:471)
      at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
      at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
      at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
      at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
      at com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:902)
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
      at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
      at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
      at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
      at com.google.common.util.concurrent.Futures$FallbackFuture$1.onFailure(Futures.java:492)
      at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
      at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
      at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
      at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
      at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:68)
      at org.opendaylight.controller.cluster.schema.provider.impl.RemoteSchemaProvider$2.onComplete(RemoteSchemaProvider.java:65)
      at org.opendaylight.controller.cluster.schema.provider.impl.RemoteSchemaProvider$2.onComplete(RemoteSchemaProvider.java:58)
      at akka.dispatch.OnComplete.internal(Future.scala:247)
      at akka.dispatch.OnComplete.internal(Future.scala:245)
      at akka.dispatch.japi$CallbackBridge.apply(Future.scala:175)
      at akka.dispatch.japi$CallbackBridge.apply(Future.scala:172)
      at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
      at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
      at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
      at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
      at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
      at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
      at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
      at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
      at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
      at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
      at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
      at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
      at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
      Caused by: org.opendaylight.yangtools.yang.model.repo.api.MissingSchemaSourceException: All available providers exhausted
      at org.opendaylight.yangtools.yang.model.repo.util.AbstractSchemaRepository$1.create(AbstractSchemaRepository.java:83)
      at com.google.common.util.concurrent.Futures$FallbackFuture$1.onFailure(Futures.java:471)
      ... 25 more
      Caused by: org.opendaylight.yangtools.yang.model.repo.api.MissingSchemaSourceException: No providers registered for sourceSourceIdentifier [name=ietf-netconf-monitoring@2010-10-04]
      at org.opendaylight.yangtools.yang.model.repo.util.AbstractSchemaRepository.getSchemaSource(AbstractSchemaRepository.java:92)
      at org.opendaylight.controller.cluster.schema.provider.impl.RemoteYangTextSourceProviderImpl.getYangTextSchemaSource(RemoteYangTextSourceProviderImpl.java:51)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at akka.actor.TypedActor$MethodCall.apply(TypedActor.scala:147)
      at akka.actor.TypedActor$TypedActor$$anonfun$receive$1$$anonfun$applyOrElse$2.apply(TypedActor.scala:311)
      at akka.actor.TypedActor$TypedActor.withContext(TypedActor.scala:299)
      at akka.actor.TypedActor$TypedActor$$anonfun$receive$1.applyOrElse(TypedActor.scala:306)
      at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
      at akka.actor.TypedActor$TypedActor.aroundReceive(TypedActor.scala:246)
      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
      at akka.actor.ActorCell.invoke(ActorCell.scala:487)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
      at akka.dispatch.Mailbox.run(Mailbox.scala:220)
      ... 5 more
      2016-01-24 02:44:03,837 | ERROR | ult-dispatcher-2 | NetconfDevice | 240 - org.opendaylight.netconf.sal-netconf-connector - 1.3.0.SNAPSHOT | RemoteDevice{netconf-test-device}

      : Initialization in sal failed, disconnecting from device
      org.opendaylight.yangtools.yang.model.repo.api.MissingSchemaSourceException: All available providers exhausted
      at org.opendaylight.yangtools.yang.model.repo.util.AbstractSchemaRepository$1.create(AbstractSchemaRepository.java:83)
      at com.google.common.util.concurrent.Futures$FallbackFuture$1.onFailure(Futures.java:471)
      at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
      at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
      at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
      at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
      at com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:902)
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
      at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
      at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
      at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
      at com.google.common.util.concurrent.Futures$FallbackFuture$1.onFailure(Futures.java:492)
      at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
      at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
      at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
      at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
      at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:68)
      at org.opendaylight.controller.cluster.schema.provider.impl.RemoteSchemaProvider$2.onComplete(RemoteSchemaProvider.java:65)
      at org.opendaylight.controller.cluster.schema.provider.impl.RemoteSchemaProvider$2.onComplete(RemoteSchemaProvider.java:58)
      at akka.dispatch.OnComplete.internal(Future.scala:247)
      at akka.dispatch.OnComplete.internal(Future.scala:245)
      at akka.dispatch.japi$CallbackBridge.apply(Future.scala:175)
      at akka.dispatch.japi$CallbackBridge.apply(Future.scala:172)
      at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
      at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
      at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
      at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
      at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
      at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
      at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
      at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
      at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
      at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
      at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
      at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
      at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
      Caused by: org.opendaylight.yangtools.yang.model.repo.api.MissingSchemaSourceException: All available providers exhausted
      at org.opendaylight.yangtools.yang.model.repo.util.AbstractSchemaRepository$1.create(AbstractSchemaRepository.java:83)
      at com.google.common.util.concurrent.Futures$FallbackFuture$1.onFailure(Futures.java:471)
      ... 25 more
      Caused by: org.opendaylight.yangtools.yang.model.repo.api.MissingSchemaSourceException: No providers registered for sourceSourceIdentifier [name=ietf-netconf-monitoring-extension@2013-12-10]
      at org.opendaylight.yangtools.yang.model.repo.util.AbstractSchemaRepository.getSchemaSource(AbstractSchemaRepository.java:92)
      at org.opendaylight.controller.cluster.schema.provider.impl.RemoteYangTextSourceProviderImpl.getYangTextSchemaSource(RemoteYangTextSourceProviderImpl.java:51)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at akka.actor.TypedActor$MethodCall.apply(TypedActor.scala:147)
      at akka.actor.TypedActor$TypedActor$$anonfun$receive$1$$anonfun$applyOrElse$2.apply(TypedActor.scala:311)
      at akka.actor.TypedActor$TypedActor.withContext(TypedActor.scala:299)
      at akka.actor.TypedActor$TypedActor$$anonfun$receive$1.applyOrElse(TypedActor.scala:306)
      at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
      at akka.actor.TypedActor$TypedActor.aroundReceive(TypedActor.scala:246)
      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
      at akka.actor.ActorCell.invoke(ActorCell.scala:487)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
      at akka.dispatch.Mailbox.run(Mailbox.scala:220)
      ... 5 more
      2016-01-24 02:44:03,844 | WARN | ult-dispatcher-2 | NetconfDeviceCommunicator | 240 - org.opendaylight.netconf.sal-netconf-connector - 1.3.0.SNAPSHOT | RemoteDevice

      {netconf-test-device}

      : Session terminated Session closed

      The above log piece was taken from https://jenkins.opendaylight.org/releng/view/netconf/job/netconf-csit-3node-clustering-only-beryllium/17/, from odl1_karaf.log. Look at logs around the test case Wait_For_Device_To_Become_Visible_For_Checker

      I don't know yet if the crash is caused by configuring the device or by trying to see the device on the other node.

            tcere Tomas Cere
            jbehran@cisco.com Jozef Behran
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: