[NETCONF-889] Improve EOM aggregator performance Created: 11/Jul/22  Updated: 31/Oct/22  Resolved: 29/Jul/22

Status: Resolved
Project: netconf
Component/s: netconf
Affects Version/s: None
Fix Version/s: 4.0.0, 3.0.6, 2.0.17

Type: Improvement Priority: High
Reporter: Sangwook Ha Assignee: Sangwook Ha
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The decoding performance of DelimiterBasedFrameDecoder, and hence the performance of NetconfEOMAggregator extending DelimiterBasedFrameDecoder becomes the bottleneck of NETCONF channel pipeline when small increments of a long frame are processed. For example, fetching YANG schema files (total 22 MB) from Juniper MX2020 takes about 16 minutes.

The main reason for this poor performance is because delimiter search starts over from the beginning of the frame (readerIndex) for each decode call in the indexOf method.



 Comments   
Comment by Robert Varga [ 12/Jul/22 ]

So there is a pre-existing patch for this.

I also wonder if it would not make sense to also push a fix for Netty.

Comment by Sangwook Ha [ 12/Jul/22 ]

Right, I found that out after submitting a change. The idea is the same, just difference in keeping track of progress.
One thing I wonder, for both changes, is whether it's robust enough to handle change in ByteBuf allocation, which I think should happen in one way or another because the index cannot increase forever.

Comment by Sangwook Ha [ 12/Jul/22 ]

There was an issue with change 75490 as suspected - it fails to process a case where ByteBuf is updated with readerIndex reset to 0.
I fixed the issue and pushed a new patch set with another test case and some cleanup.
rovarga, please take a look.

Generated at Wed Feb 07 20:16:10 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.