Operating System: All
Platform: All

The cause for the revert is in: https://bugs.opendaylight.org/show_bug.cgi?id=6273

The root cause is:

2016-07-26 12:54:04,004 | WARN | oupCloseable-3-1 | ChannelInitializer | 112 - io.netty.common - 4.0.37.Final | Failed to initialize a channel. Closing: [id: 0x3b162577]
java.lang.NoSuchMethodError: org.apache.mina.core.service.SimpleIoProcessorPool.<init>(Ljava/lang/Class;Ljava/util/concurrent/Executor;ILjava/nio/channels/spi/SelectorProvider;)V
at org.apache.sshd.common.io.mina.MinaServiceFactory.<init>(MinaServiceFactory.java:44)
at org.apache.sshd.common.io.mina.MinaServiceFactoryFactory.create(MinaServiceFactoryFactory.java:49)
at org.apache.sshd.common.io.DefaultIoServiceFactoryFactory.create(DefaultIoServiceFactoryFactory.java:49)
at org.apache.sshd.common.helpers.AbstractFactoryManager.getIoServiceFactory(AbstractFactoryManager.java:101)
at org.apache.sshd.client.SshClient.createConnector(SshClient.java:714)
at org.apache.sshd.client.SshClient.start(SshClient.java:441)
at org.opendaylight.netconf.nettyutil.handler.ssh.client.AsyncSshHandler.<clinit>(AsyncSshHandler.java:52)
at org.opendaylight.netconf.client.SshClientChannelInitializer.initialize(SshClientChannelInitializer.java:34)
at org.opendaylight.netconf.client.NetconfClientDispatcherImpl$4.initializeChannel(NetconfClientDispatcherImpl.java:117)
at org.opendaylight.netconf.client.NetconfClientDispatcherImpl$4.initializeChannel(NetconfClientDispatcherImpl.java:114)
at org.opendaylight.protocol.framework.ReconnectPromise$1.initializeChannel(ReconnectPromise.java:52)
at org.opendaylight.protocol.framework.ReconnectPromise$1.initializeChannel(ReconnectPromise.java:49)
at org.opendaylight.protocol.framework.AbstractDispatcher$3.initChannel(AbstractDispatcher.java:194)
at org.opendaylight.protocol.framework.AbstractDispatcher$3.initChannel(AbstractDispatcher.java:191)
at io.netty.channel.ChannelInitializer.channelRegistered(ChannelInitializer.java:68)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRegistered(AbstractChannelHandlerContext.java:136)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRegistered(AbstractChannelHandlerContext.java:122)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRegistered(AbstractChannelHandlerContext.java:115)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRegistered(DefaultChannelPipeline.java:1253)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRegistered(AbstractChannelHandlerContext.java:136)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRegistered(AbstractChannelHandlerContext.java:122)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRegistered(DefaultChannelPipeline.java:784)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:445)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:368)[113:io.netty.transport:4.0.37.Final]
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:418)[113:io.netty.transport:4.0.37.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)[112:io.netty.common:4.0.37.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:394)[113:io.netty.transport:4.0.37.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)[112:io.netty.common:4.0.37.Final]
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)[112:io.netty.common:4.0.37.Final]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_91]

which boils down to sshd-core requiring mina-core-2.0.13, not 2.0.7 as defaulting in karaf somewhere. This should be fixed by pulling in proper version via a feature.

https://git.opendaylight.org/gerrit/42911

Adding mina-core into the feature def doesn't seem to help, needs more analysis to check if karaf isn't adding it onto the classpath.

We can probably handle this in the same way as the Aries Proxy overrides; I'm working on a patch.

I've pushed https://git.opendaylight.org/gerrit/43034 which produces a Karaf assembly with up-to-date Mina and sshd-core.

I forgot to mention that the previous patch (against odlparent) also needs https://git.opendaylight.org/gerrit/42649 against controller to work. You need to rebuild odlparent, controller, then integration/distribution.

Thanks for the patch Stephen, it fixed the issues with netconf with the mina bump, however the karaf client script seems to be broken now:

bin\> ./client -u karaf
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/sshd/client/UserInteraction
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.sshd.client.UserInteraction
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more

(In reply to Tomas Cere from comment #7)
> Thanks for the patch Stephen, it fixed the issues with netconf with the mina
> bump, however the karaf client script seems to be broken now:

Thanks for the test! Robert suspected the sshd-core bump would break Karaf, I was in the process of checking...

(In reply to Tomas Cere from comment #7)
> Thanks for the patch Stephen, it fixed the issues with netconf with the mina
> bump, however the karaf client script seems to be broken now:
>
> bin\> ./client -u karaf

Does it work for you if you edit bin/client so the line containing sshd-core reads

CLASSPATH="$CLASSPATH:$KARAF_HOME/system/org/apache/sshd/sshd-core/0.14.0/sshd-core-0.14.0.jar"

(It logs sensible stuff on my system but I then get the "EC KeyPairGenerator not available" error.)

(In reply to Stephen Kitt from comment #9)
> (In reply to Tomas Cere from comment #7)
> > Thanks for the patch Stephen, it fixed the issues with netconf with the mina
> > bump, however the karaf client script seems to be broken now:
> >
> > bin\> ./client -u karaf
>
> Does it work for you if you edit bin/client so the line containing sshd-core
> reads
>
>
> CLASSPATH="$CLASSPATH:$KARAF_HOME/system/org/apache/sshd/sshd-core/0.14.0/
> sshd-core-0.14.0.jar"
>
> (It logs sensible stuff on my system but I then get the "EC KeyPairGenerator
> not available" error.)

After upgrading my JDK it works for me, I'll update the odlparent patch.

Ok now we have a weird behavior. It works fine the first time when you run karaf and install odl-netconf-connector-all, but when you restart karaf the

java.lang.NoSuchMethodError: org.apache.mina.core.service.SimpleIoProcessorPool.<init>(Ljava/lang/Class;Ljava/util/concurrent/Executor;ILjava/nio/channels/spi/SelectorProvider;)V

is back.

We need karaf-4.1.x, after which we can jump to 1.4.0.

Now that Karaf 4 is in place, any interest in resuming work on updating to 1.6.0 or 1.7.0?

We are currently using 1.6.0, which is what is shipping with karaf-4.1.x – that bump occurred in Oxygen. I would suggest integrating 1.7.0 when we bump to karaf-4.2.x, as keeping the versions aligned is beneficial in side-stepping issues.

Fixed with bump to odlparent-3.0.x. If upgrading to 1.7.0 before karaf-4.2.x comes down (which is not happening in Fluorine) is of interest, please file a separate issue.

Excellent; thanks.  I'll commence testing a variety of netconf devices with Oxygen to benefit from 1.6.0, and file separate bugs if/as they occur.