OpenDaylight JIRA

OpenDaylight JIRA https://jira.opendaylight.org This file is an XML representation of an issue en-us

8.20.10

820010

22-06-2022

[OVSDB-316] OVSDB server doesn't close connection after peer is power down https://jira.opendaylight.org/browse/OVSDB-316 ovsdb <p>In passive connection scenario, OVSDB connection is not closed on server side when peer is down before it has chance to send TCP FIN packet. We have seen this issue when peer is power off or crashed.</p> <p>Operating System: All<br/> Platform: All</p> OVSDB-316

OVSDB server doesn't close connection after peer is power down

Bug Resolved Done Vishal Thapar Hsin-Yi Shen Sun, 13 Mar 2016 22:06:01 +0000 Mon, 30 Oct 2017 19:52:45 +0000 Fri, 27 May 2016 21:47:13 +0000 unspecified Library 0 3 <p>master: <a href="https://git.opendaylight.org/gerrit/35436" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/35436</a><br/> beryllium: <a href="https://git.opendaylight.org/gerrit/36152" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/36152</a></p> <p>The main issue with the bug has been fixed. I think we should close this bug and open another to track the open item for configuration.</p> <p>Re-opening the bug as fix is not complete.</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/36152/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/36152/</a> added code to detect when a peer goes down without sending a FIN. However, it only did it for switch initiated connection. In case of ODL initiated connection we are unable to detect peer being down and reconciliation is never triggered.</p> <p>Logs with the fix in place:</p> <p>2016-05-26 16:29:39,075 | TRACE | entLoopGroup-8-1 | JsonRpcEndpoint | 168 - org.opendaylight.ovsdb.library - 1.2.4.SNAPSHOT | Request : null "update" ["b738c559-fd98-4251-9d6d-04bffd76e9ad",{"Manager":{"f1e4c461-19f3-4755-9cc1-38b5e9b9dcd6":{"new":</p> {"_version":["uuid","a77a7a32-0a3d-4fb4-84a0-9f1350e24f46"],"connection_mode":["set",[]],"is_connected":true,"other_config":["map",[]],"external_ids":["map",[]],"status":["map",[["bound_port","6640"],["sec_since_connect","5"],["state","ACTIVE"]]],"_uuid":["uuid","f1e4c461-19f3-4755-9cc1-38b5e9b9dcd6"],"target":"ptcp:6640","inactivity_probe":["set",[]],"max_backoff":["set",[]]} <p>,"old":{"_version":<span class="error">["uuid","23525c95-63de-4bea-a2d4-bafa4853034d"]</span>,"status":["map",[<span class="error">["bound_port","6640"]</span>,<span class="error">["sec_since_connect","0"]</span>,<span class="error">["state","ACTIVE"]</span>]]}}}}]</p> <p>After this log entry I bring down the interface on OVS.</p> <p>2016-05-26 16:30:39,087 | DEBUG | entLoopGroup-8-1 | ExceptionHandler | 168 - org.opendaylight.ovsdb.library - 1.2.4.SNAPSHOT | Get idle state event<br/> 2016-05-26 16:30:39,087 | DEBUG | entLoopGroup-8-1 | ExceptionHandler | 168 - org.opendaylight.ovsdb.library - 1.2.4.SNAPSHOT | Reader idle state. Send echo message to peer<br/> 2016-05-26 16:30:39,087 | TRACE | entLoopGroup-8-1 | JsonRpcEndpoint | 168 - org.opendaylight.ovsdb.library - 1.2.4.SNAPSHOT | getClient Request : </p> {"id":"46b0eba4-3429-4a85-854f-f4c0136e75da","method":"echo","params":[]} <p>2016-05-26 16:30:57,991 | INFO | entLoopGroup-8-1 | OvsdbConnectionService | 168 - org.opendaylight.ovsdb.library - 1.2.4.SNAPSHOT | Connection closed ConnectionInfo <span class="error">[Remote-address=192.168.56.111, Remote-port=6640, Local-address192.168.56.1, Local-port=58166, type=ACTIVE]</span><br/> 2016-05-26 16:30:57,992 | INFO | entLoopGroup-8-1 | OvsdbConnectionManager | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Library disconnected ACTIVE from /192.168.56.111:6640 to /192.168.56.1:58166. Cleaning up the operational data store<br/> 2016-05-26 16:30:57,995 | INFO | entLoopGroup-8-1 | OvsdbConnectionManager | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Disconnected/Failed connection ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address <span class="error">[_value=192.168.56.111]</span>, _value=<span class="error">[1, 9, 2, ., 1, 6, 8, ., 5, 6, ., 1, 1, 1]</span>], getRemotePort=PortNumber <span class="error">[_value=6640]</span>, augmentations={}} was controller initiated, attempting reconnection<br/> 2016-05-26 16:30:57,998 | INFO | ult-dispatcher-3 | OvsdbConnectionManager | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Entity{type='ovsdb', id=/(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)network-topology/topology/topology[</p> {(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)topology-id=ovsdb:1} <p>]/node/node[</p> {(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)node-id=ovsdb://192.168.56.111:6640} <p>]} has no owner, cleaning up the operational data store<br/> 2016-05-26 16:30:57,999 | TRACE | entLoopGroup-8-1 | JsonRpcDecoder | 168 - org.opendaylight.ovsdb.library - 1.2.4.SNAPSHOT | readable bytes 0, records read 8, incomplete record bytes 0<br/> 2016-05-26 16:30:58,000 | INFO | sdb-reconciler-1 | ConnectionReconciliationTask | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Retry(1) connection to Ovsdb Node ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address <span class="error">[_value=192.168.56.111]</span>, _value=<span class="error">[1, 9, 2, ., 1, 6, 8, ., 5, 6, ., 1, 1, 1]</span>], getRemotePort=PortNumber <span class="error">[_value=6640]</span>, augmentations={}} <br/> 2016-05-26 16:30:58,001 | INFO | sdb-reconciler-1 | OvsdbConnectionManager | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Connecting to 192.168.56.111:6640<br/> 2016-05-26 16:31:19,025 | WARN | sdb-reconciler-1 | OvsdbConnectionService | 168 - org.opendaylight.ovsdb.library - 1.2.4.SNAPSHOT | bootstrap.connect failed<br/> java.net.ConnectException: Connection timed out: no further information: /192.168.56.111:6640<br/> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)<span class="error">[:1.8.0_60]</span><br/> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)<span class="error">[:1.8.0_60]</span><br/> at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)<span class="error">[125:io.netty.transport:4.0.33.Final]</span><br/> at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)<span class="error">[125:io.netty.transport:4.0.33.Final]</span><br/> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)<span class="error">[125:io.netty.transport:4.0.33.Final]</span><br/> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)<span class="error">[125:io.netty.transport:4.0.33.Final]</span><br/> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)<span class="error">[125:io.netty.transport:4.0.33.Final]</span><br/> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)<span class="error">[125:io.netty.transport:4.0.33.Final]</span><br/> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)<span class="error">[124:io.netty.common:4.0.33.Final]</span><br/> at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)<span class="error">[124:io.netty.common:4.0.33.Final]</span><br/> at java.lang.Thread.run(Thread.java:745)<span class="error">[:1.8.0_60]</span><br/> 2016-05-26 16:31:19,028 | WARN | sdb-reconciler-1 | OvsdbConnectionManager | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Failed to connect to OVSDB Node ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address <span class="error">[_value=192.168.56.111]</span>, _value=<span class="error">[1, 9, 2, ., 1, 6, 8, ., 5, 6, ., 1, 1, 1]</span>], getRemotePort=PortNumber <span class="error">[_value=6640]</span>, augmentations={}}<br/> 2016-05-26 16:31:19,028 | WARN | sdb-reconciler-1 | ConnectionReconciliationTask | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Connection retry(1) failed for ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address <span class="error">[_value=192.168.56.111]</span>, _value=<span class="error">[1, 9, 2, ., 1, 6, 8, ., 5, 6, ., 1, 1, 1]</span>], getRemotePort=PortNumber <span class="error">[_value=6640]</span>, augmentations={}}.<br/> 2016-05-26 16:31:29,029 | INFO | sdb-reconciler-1 | ConnectionReconciliationTask | 174 - org.opendaylight.ovsdb.southbound-impl - 1.2.4.SNAPSHOT | Retry(2) connection to Ovsdb Node ConnectionInfo{getRemoteIp=IpAddress [_ipv4Address=Ipv4Address <span class="error">[_value=192.168.56.111]</span>, _value=<span class="error">[1, 9, 2, ., 1, 6, 8, ., 5, 6, ., 1, 1, 1]</span>], getRemotePort=PortNumber <span class="error">[_value=6640]</span>, augmentations={}}</p> <p>Also, in this case we never hit the readTimeOutException because reconciliation logic has already torn down the connection.</p> <p>stable/beryllium: <a href="https://git.opendaylight.org/gerrit/39477" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/39477</a></p> <p>(In reply to Vishal Thapar from comment #4)<br/> > stable/beryllium: <a href="https://git.opendaylight.org/gerrit/39477" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/39477</a></p> <p>Boron: <a href="https://git.opendaylight.org/gerrit/#/c/39511/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/39511/</a></p> Development External issue ID 5506 External issue URL Rank 0|i021zb: