This was not a inevitable phenomenon, it occured when i tried to connect 500 devices.
I believe that when there are many devices to connect, channelActive may happen later than handleMessage because device will not wait for your hello message. When you handle the message, you need to get session for hello message ,then change state to ESTABLISHED , but the state is IDLE now (Only channelActive can invoke startNegotiation method and change state to OPEN_WAIT.), change will result a IllegalStateException.
And this happens again and again even if reconnect strategy invoke reconnect. At last, the device will not connect success.
Test case show the stack.(https://git.opendaylight.org/gerrit/#/c/69738/4/netconf/netconf-client/src/test/java/org/opendaylight/netconf/client/NetconfClientSessionNegotiatorTest.java)
java.lang.IllegalStateException: Cannot change state from IDLE to ESTABLISHED for chanel
at com.google.common.base.Preconditions.checkState(Preconditions.java:836)
at org.opendaylight.netconf.nettyutil.AbstractNetconfSessionNegotiator.changeState(AbstractNetconfSessionNegotiator.java:236)
at org.opendaylight.netconf.nettyutil.AbstractNetconfSessionNegotiator.getSessionForHelloMessage(AbstractNetconfSessionNegotiator.java:173)
at org.opendaylight.netconf.client.NetconfClientSessionNegotiator.handleMessage(NetconfClientSessionNegotiator.java:68)
at org.opendaylight.netconf.client.NetconfClientSessionNegotiatorTest.testNegotiatorWhenChannelActiveHappenAfterHandleMessage(NetconfClientSessionNegotiatorTest.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)