[NETCONF-888] Make netconf maximum chunk size value configurable Created: 29/Jun/22  Updated: 22/Aug/22  Resolved: 18/Aug/22

Status: Resolved
Project: netconf
Component/s: netconf
Affects Version/s: None
Fix Version/s: 3.0.6, 4.0.1

Type: Improvement Priority: Medium
Reporter: Daniel C Viswanathan Assignee: Daniel C Viswanathan
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Issue split
split to NETCONF-897 Allow netconf-client's maximum chunk ... Resolved
split to NETCONF-898 Allow netconf-impl's maximum chunk to... Resolved

 Description   

Now for changing the maximum chunk size value, we have to change and rebuild the netconf package, it will be good if we make the maximum chunk size configurable

 Comments   
Comment by Daniel C Viswanathan [ 29/Jul/22 ] 
2022-07-29T12:39:28,891 | DEBUG | sshd-NetconfSshClient[1aff52ad]-nio2-thread-7 | NettyAwareChannelSubsystem       | 302 - org.opendaylight.netconf.shaded-sshd - 1.13.8 | handleData(NettyAwareChannelSubsystem[id=0, recipient=0]-NetconfClientSessionImpl[0][netconf]) SSH_MSG_CHANNEL_DATA len=88
2022-07-29T12:39:29,022 | DEBUG | sshd-NetconfSshClient[1aff52ad]-nio2-thread-1 | NettyAwareChannelSubsystem       | 302 - org.opendaylight.netconf.shaded-sshd - 1.13.8 | handleData(NettyAwareChannelSubsystem[id=0, recipient=0]-NetconfClientSessionImpl[0][netconf]) SSH_MSG_CHANNEL_DATA len=4096
2022-07-29T12:39:29,022 | DEBUG | nioEventLoopGroupCloseable-3-1 | NetconfChunkAggregator           | 293 - org.opendaylight.netconf.netty-util - 1.13.8 | Parsed chunk size 17916488, maximum allowed is 16777216
2022-07-29T12:39:29,023 | WARN  | nioEventLoopGroupCloseable-3-1 | DefaultChannelPipeline           | 62 - io.netty.common - 4.1.69.Final | An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: java.lang.IllegalStateException: Maximum chunk size exceeded
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:477) ~[bundleFile:4.1.69.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[bundleFile:4.1.69.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [bundleFile:4.1.69.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) [bundleFile:4.1.69.Final]
        at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) [bundleFile:4.1.69.Final]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [bundleFile:4.1.69.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) [bundleFile:4.1.69.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [bundleFile:4.1.69.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [bundleFile:4.1.69.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [bundleFile:4.1.69.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [bundleFile:4.1.69.Final]
        at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.IllegalStateException: Maximum chunk size exceeded
        at org.opendaylight.netconf.nettyutil.handler.NetconfChunkAggregator.checkChunkSize(NetconfChunkAggregator.java:63) ~[bundleFile:?]
        at org.opendaylight.netconf.nettyutil.handler.NetconfChunkAggregator.decode(NetconfChunkAggregator.java:103) ~[bundleFile:?]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) ~[bundleFile:4.1.69.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) ~[bundleFile:4.1.69.Final]
        ... 11 more
2022-07-29T12:39:29,069 | DEBUG | sshd-NetconfSshClient[1aff52ad]-nio2-thread-2 | NettyAwareChannelSubsystem       | 302 - org.opendaylight.netconf.shaded-sshd - 1.13.8 | handleData(NettyAwareChannelSubsystem[id=0, recipient=0]-NetconfClientSessionImpl[0][netconf]) SSH_MSG_CHANNEL_DATA len=4096
2022-07-29T12:39:29,070 | DEBUG | nioEventLoopGroupCloseable-3-1 | NetconfChunkAggregator           | 293 - org.opendaylight.netconf.netty-util - 1.13.8 | Buffer has 8181 bytes, need 17916488 to complete chunk
2022-07-29T12:39:29,070 | DEBUG | sshd-NetconfSshClient[1aff52ad]-nio2-thread-2 | NettyAwareChannelSubsystem       | 302 - org.opendaylight.netconf.shaded-sshd - 1.13.8 | handleData(NettyAwareChannelSubsystem[id=0, recipient=0]-NetconfClientSessionImpl[0][netconf]) SSH_MSG_CHANNEL_DATA len=4096
2022-07-29T12:39:29,070 | DEBUG | nioEventLoopGroupCloseable-3-1 | NetconfChunkAggregator           | 293 - org.opendaylight.netconf.netty-util - 1.13.8 | Buffer has 8181 bytes, need 17916488 to complete chunk
2022-07-29T12:39:29,070 | DEBUG | nioEventLoopGroupCloseable-3-1 | NetconfChunkAggregator           | 293 - org.opendaylight.netconf.netty-util - 1.13.8 | Buffer has 12277 bytes, need 17916488 to complete chunk
2022-07-29T12:39:29,114 | DEBUG | sshd-NetconfSshClient[1aff52ad]-nio2-thread-3 | NettyAwareChannelSubsystem       | 302 - org.opendaylight.netconf.shaded-sshd - 1.13.8 | handleData(NettyAwareChannelSubsystem[id=0, recipient=0]-NetconfClientSessionImpl[0][netconf]) SSH_MSG_CHANNEL_DATA len=4096
Comment by Robert Varga [ 29/Jul/22 ]

Right, so this this chunk size on the inbound side, hence this needs to only touch NetconfChunkAggregator.

Comment by Robert Varga [ 29/Jul/22 ]

I have updated the baseline patch, but this needs a lot more work to make complete. The issue is that the system property is a global thing, affecting all sessions, client and server, irrespective of their mature. Remaining items:

  1. https://git.opendaylight.org/gerrit/c/netconf/+/101924 allows a client's setting to be changed, we need a similar thing for NetconfServerSessionNegotiator
  2. netconf-topology and netconf-topology-singleton both need to be updated to have the ability to specify this maximum for each device – i.e. update their configuration .yang file and route that information to netconf-client
  3. callhome (not sure where) needs to be updated to take advantage of the facility created in 1.
  4. mdsal-netconf-ssh needs to do the same as callhome, except the tuneable needs to go through NetconfConfiguration (and be properly routed from properties, just as other things are)
Comment by Daniel C Viswanathan [ 02/Aug/22 ]

Thank you for the update, Robert.
I will look into the remaining items.

Generated at Wed Feb 07 20:16:09 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.