[CONTROLLER-478] Buffer leaks while running controller from OpenFlowPlugin distribution Created: 15/May/14  Updated: 25/Jul/23  Due: 27/Jun/14  Resolved: 26/Jun/14

Status: Verified
Project: controller
Component/s: netconf
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Debolina Bandyopadhyay Assignee: Maros Marsalek
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Windows
Platform: PC

Issue Links:
Duplicate
is duplicated by CONTROLLER-454 OSGi Error after starting the control... Resolved
External issue ID: 1017

 Description   

I get this error while running controller.

2014-05-15 00:05:06.560 PDT [Thread-27] INFO o.o.o.protocol.impl.core.TcpHandler - Switch listener started and ready to accept incoming connections on port: 6633

2014-05-15 00:05:06.928 PDT [nettyThreadgroupModule$NioEventLoopGroupCloseable-6
-1] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not
called before it's garbage-collected. Enable advanced leak reporting to find out
where the leak occurred. To enable advanced leak reporting, specify the JVM
option '-Dio.netty.leakDetectionLevel=advanced' or call ResourceLeakDetector.setLevel()

 Comments   
Comment by Michal Polkorab [ 16/May/14 ]

Hello,

I have seen the error log over the past week. However, the issue disappeared after rebuilding the openflowjava and openflowplugin repositories (and running the openflowplugin distribution).

Please try updating code and rebuilding the mentioned repositories. Report if the issue persists (the '-Dio.netty.leakDetectionLevel=advanced' log would help).

Regards
Michal Polkorab

Comment by Michal Polkorab [ 21/May/14 ]

As the issue appeared again I tried to debug it.

It looks like there is some buffer.release() missing in the netconf pipeline. See the log below:

2014-05-21 14:06:10.943 CEST [nioEventLoopGroup-10-1] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected.
Recent access records: 4
#4:
io.netty.buffer.AdvancedLeakAwareByteBuf.getBytes(AdvancedLeakAwareByteBuf.java:235)
org.opendaylight.controller.netconf.nettyutil.handler.ssh.client.SshClientAdapter.writeImpl(SshClientAdapter.java:109)
org.opendaylight.controller.netconf.nettyutil.handler.ssh.client.SshClientAdapter.write(SshClientAdapter.java:104)
org.opendaylight.controller.netconf.nettyutil.handler.ssh.SshHandler.write(SshHandler.java:58)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:701)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:640)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:117)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:701)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:640)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:117)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.access$2000(DefaultChannelHandlerContext.java:29)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.write(DefaultChannelHandlerContext.java:914)
io.netty.channel.DefaultChannelHandlerContext$WriteAndFlushTask.write(DefaultChannelHandlerContext.java:968)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.run(DefaultChannelHandlerContext.java:903)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:354)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
java.lang.Thread.run(Thread.java:722)
#3:
io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:565)
org.opendaylight.controller.netconf.nettyutil.handler.ChunkedFramingMechanismEncoder.encode(ChunkedFramingMechanismEncoder.java:52)
org.opendaylight.controller.netconf.nettyutil.handler.ChunkedFramingMechanismEncoder.encode(ChunkedFramingMechanismEncoder.java:21)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:111)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:701)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:640)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:117)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.access$2000(DefaultChannelHandlerContext.java:29)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.write(DefaultChannelHandlerContext.java:914)
io.netty.channel.DefaultChannelHandlerContext$WriteAndFlushTask.write(DefaultChannelHandlerContext.java:968)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.run(DefaultChannelHandlerContext.java:903)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:354)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
java.lang.Thread.run(Thread.java:722)
#2:
io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:547)
org.opendaylight.controller.netconf.nettyutil.handler.ChunkedFramingMechanismEncoder.encode(ChunkedFramingMechanismEncoder.java:51)
org.opendaylight.controller.netconf.nettyutil.handler.ChunkedFramingMechanismEncoder.encode(ChunkedFramingMechanismEncoder.java:21)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:111)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:701)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:640)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:117)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.access$2000(DefaultChannelHandlerContext.java:29)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.write(DefaultChannelHandlerContext.java:914)
io.netty.channel.DefaultChannelHandlerContext$WriteAndFlushTask.write(DefaultChannelHandlerContext.java:968)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.run(DefaultChannelHandlerContext.java:903)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:354)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
java.lang.Thread.run(Thread.java:722)
#1:
io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:547)
org.opendaylight.controller.netconf.nettyutil.handler.ChunkedFramingMechanismEncoder.encode(ChunkedFramingMechanismEncoder.java:50)
org.opendaylight.controller.netconf.nettyutil.handler.ChunkedFramingMechanismEncoder.encode(ChunkedFramingMechanismEncoder.java:21)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:111)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:701)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:640)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:117)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.access$2000(DefaultChannelHandlerContext.java:29)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.write(DefaultChannelHandlerContext.java:914)
io.netty.channel.DefaultChannelHandlerContext$WriteAndFlushTask.write(DefaultChannelHandlerContext.java:968)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.run(DefaultChannelHandlerContext.java:903)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:354)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
java.lang.Thread.run(Thread.java:722)
Created at:
io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:55)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:99)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:106)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:701)
io.netty.channel.DefaultChannelHandlerContext.write(DefaultChannelHandlerContext.java:640)
io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:117)
io.netty.channel.DefaultChannelHandlerContext.invokeWrite(DefaultChannelHandlerContext.java:647)
io.netty.channel.DefaultChannelHandlerContext.access$2000(DefaultChannelHandlerContext.java:29)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.write(DefaultChannelHandlerContext.java:914)
io.netty.channel.DefaultChannelHandlerContext$WriteAndFlushTask.write(DefaultChannelHandlerContext.java:968)
io.netty.channel.DefaultChannelHandlerContext$AbstractWriteTask.run(DefaultChannelHandlerContext.java:903)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:354)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
java.lang.Thread.run(Thread.java:722)

Comment by Michal Polkorab [ 21/May/14 ]

This is a duplicate bug (unfortunately). It was originally reported in the openflowplugin project.

I haven't marked it as a duplicate because of the previous log, that might be really helpful.

Michal Polkorab

Comment by Maros Marsalek [ 23/Jun/14 ]

Please review https://git.opendaylight.org/gerrit/#/c/8179/

Just one release added. Other buffers should be released correctly.

Comment by Michal Polkorab [ 26/Jun/14 ]

Re-tested: issue fixed.

Michal

Generated at Wed Feb 07 19:53:07 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.