Uploaded image for project: 'controller'
  1. controller
  2. CONTROLLER-1867

Cluster: Member fail to start with odl-netconf-clustered-topology feature

XMLWordPrintable

      The issue only happens in oxygen and was hidden because current CSIT test installs all compatible features (all test), however if we run same test with just odl-netconf-clustered-topology feature (only test) we see that a cluster member that has been killed cannot be restarted:

      https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netconf-csit-3node-clustering-all-oxygen/421/robot-plugin/log.html.gz

      In the karaf log of the restarting member (1):

      https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netconf-csit-3node-clustering-all-oxygen/421/odl_1/odl1_karaf.log.gz

      we see this ERROR:

      2018-10-17T19:33:22,521 | INFO | config-pusher | ConfigPusherImpl | 204 - org.opendaylight.controller.config-persister-impl - 0.8.4.SNAPSHOT | Pushing configuration snapshot 06-clustered-entity-ownership.xml(odl-netconf-connector,odl-netconf-connector)
      2018-10-17T19:33:22,627 | ERROR | opendaylight-cluster-data-akka.actor.default-dispatcher-3 | ActorSystemImpl | 50 - com.typesafe.akka.slf4j - 2.5.11 | Uncaught error from thread [opendaylight-cluster-data-akka.actor.default-dispatcher-4]: org/w3c/dom/ElementTraversal, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[opendaylight-cluster-data]
      java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
      at java.lang.ClassLoader.defineClass1(Native Method) ~[?:?]
      at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[?:?]
      at org.eclipse.osgi.internal.loader.ModuleClassLoader.defineClass(ModuleClassLoader.java:276) ~[?:?]
      at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.defineClass(ClasspathManager.java:655) ~[?:?]
      at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findClassImpl(ClasspathManager.java:578) ~[?:?]
      at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClassImpl(ClasspathManager.java:538) ~[?:?]
      at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClass(ClasspathManager.java:525) ~[?:?]
      at org.eclipse.osgi.internal.loader.ModuleClassLoader.findLocalClass(ModuleClassLoader.java:328) ~[?:?]
      at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:368) ~[?:?]
      at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:446) ~[?:?]
      at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:395) ~[?:?]
      at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:387) ~[?:?]
      at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:150) ~[?:?]
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:?]
      at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) ~[378:wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar:0.0.0]
      at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) ~[?:?]
      at org.opendaylight.controller.blueprint.ext.DataStoreAppConfigDefaultXMLReader.parsePossibleDefaultAppConfigXMLFile(DataStoreAppConfigDefaultXMLReader.java:174) ~[194:org.opendaylight.controller.blueprint:0.8.4.SNAPSHOT]
      at org.opendaylight.controller.blueprint.ext.DataStoreAppConfigDefaultXMLReader.createDefaultInstance(DataStoreAppConfigDefaultXMLReader.java:126) ~[194:org.opendaylight.controller.blueprint:0.8.4.SNAPSHOT]
      at org.opendaylight.controller.blueprint.ext.DataStoreAppConfigMetadata.createDefaultInstance(DataStoreAppConfigMetadata.java:262) ~[194:org.opendaylight.controller.blueprint:0.8.4.SNAPSHOT]
      at org.opendaylight.controller.blueprint.ext.DataStoreAppConfigMetadata.setInitialAppConfig(DataStoreAppConfigMetadata.java:229) ~[194:org.opendaylight.controller.blueprint:0.8.4.SNAPSHOT]
      at org.opendaylight.controller.blueprint.ext.DataStoreAppConfigMetadata.access$200(DataStoreAppConfigMetadata.java:56) ~[194:org.opendaylight.controller.blueprint:0.8.4.SNAPSHOT]
      at org.opendaylight.controller.blueprint.ext.DataStoreAppConfigMetadata$1.onSuccess(DataStoreAppConfigMetadata.java:172) ~[194:org.opendaylight.controller.blueprint:0.8.4.SNAPSHOT]
      at org.opendaylight.controller.blueprint.ext.DataStoreAppConfigMetadata$1.onSuccess(DataStoreAppConfigMetadata.java:165) ~[194:org.opendaylight.controller.blueprint:0.8.4.SNAPSHOT]
      at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1393) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1015) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:868) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:691) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.setResult(AbstractTransformFuture.java:237) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:159) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1015) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:868) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:691) ~[38:com.google.guava:23.6.1.jre]
      at com.google.common.util.concurrent.SettableFuture.set(SettableFuture.java:48) ~[38:com.google.guava:23.6.1.jre]
      at org.opendaylight.controller.cluster.datastore.messages.ReadData.processResponse(ReadData.java:40) ~[226:org.opendaylight.controller.sal-distributed-datastore:1.7.4.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.RemoteTransactionContext$2.onComplete(RemoteTransactionContext.java:250) ~[226:org.opendaylight.controller.sal-distributed-datastore:1.7.4.SNAPSHOT]
      at akka.dispatch.OnComplete.internal(Future.scala:261) ~[44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.OnComplete.internal(Future.scala:258) ~[44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.japi$CallbackBridge.apply(Future.scala:188) ~[44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.japi$CallbackBridge.apply(Future.scala:185) ~[44:com.typesafe.akka.actor:2.5.11]
      at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) ~[353:org.scala-lang.scala-library:2.12.6.v20180426-183816-VFINAL-d7ae7a8]
      at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) ~[44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91) ~[44:com.typesafe.akka.actor:2.5.11]
      at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) ~[353:org.scala-lang.scala-library:2.12.6.v20180426-183816-VFINAL-d7ae7a8]
      at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81) ~[353:org.scala-lang.scala-library:2.12.6.v20180426-183816-VFINAL-d7ae7a8]
      at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91) ~[44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) ~[44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:43) ~[44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [44:com.typesafe.akka.actor:2.5.11]
      at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [44:com.typesafe.akka.actor:2.5.11]
      Caused by: java.lang.ClassNotFoundException: org.w3c.dom.ElementTraversal cannot be found by wrap_file__tmp_karaf-0.8.4-SNAPSHOT_system_xerces_xercesImpl_2.11.0_xercesImpl-2.11.0.jar_0.0.0
      at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:433) ~[?:?]
      at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:395) ~[?:?]
      at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:387) ~[?:?]
      at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:150) ~[?:?]
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:?]
      ... 61 more{comment}

            rovarga Robert Varga
            ecelgp Luis Gomez
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: