https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/NETCONF-186 netconf <p>1. Setup a 3-node cluster.<br/> 2. Configure a Netconf device and wait for it to connect (testool simulated one is fine).<br/> 3. Kill -9 the ODL.<br/> 4. Now when you perform an operation on the device (tested to create a piece of data) the two nodes that remain will work just fine.<br/> 5. After the dead node is brought back up, it fails to mount the device (any access to the data part of the netconf connector will result in "Mount point does not exist) but the device itself is still present in the topology.</p> <p>See <a href="https://git.opendaylight.org/gerrit/37335/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/37335/</a> for a suite that performs this test.</p> <p>Operating System: All<br/> Platform: All</p> NETCONF-186

Clustering: Device not mounted on node after the node reboots

Bug Resolved Done Rudolf Brisuda Jozef Behran Tue, 19 Apr 2016 15:48:32 +0000 Fri, 15 Mar 2019 22:22:23 +0000 Mon, 24 Oct 2016 13:15:11 +0000 netconf 0 4 <p>&gt; it fails to mount the device</p> <p>Testtool logs confirm the new session was established. INFO level of karaf.log does not show why the mount operation failed, here is a complete segment:</p> <p>2016-05-03 12:09:02,449 | INFO | lt-dispatcher-40 | BaseTopologyManager | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Member is Up: Member(address <br/> = akka.tcp://opendaylight-cluster-data@10.30.32.233:2550, status = Up)<br/> 2016-05-03 12:09:02,449 | INFO | lt-dispatcher-16 | ShardManager | 173 - org.opendaylight.controller.sal-distributed-datastore - 1.3.1.Beryllium-SR1 | updatePeerAddress <br/> for peer member-1-shard-inventory-operational with address akka.tcp://opendaylight-cluster-data@10.30.32.233:2550/user/shardmanager-operational/member-1-shard-inventory-operational<br/> 2016-05-03 12:09:02,449 | INFO | lt-dispatcher-16 | ShardManager | 173 - org.opendaylight.controller.sal-distributed-datastore - 1.3.1.Beryllium-SR1 | updatePeerAddress <br/> for peer member-1-shard-entity-ownership-operational with address akka.tcp://opendaylight-cluster-data@10.30.32.233:2550/user/shardmanager-operational/member-1-shard-entity-ownership-operati<br/> onal<br/> 2016-05-03 12:09:02,449 | INFO | lt-dispatcher-16 | ShardManager | 173 - org.opendaylight.controller.sal-distributed-datastore - 1.3.1.Beryllium-SR1 | updatePeerAddress <br/> for peer member-1-shard-topology-operational with address akka.tcp://opendaylight-cluster-data@10.30.32.233:2550/user/shardmanager-operational/member-1-shard-topology-operational<br/> 2016-05-03 12:09:02,449 | INFO | lt-dispatcher-16 | ShardManager | 173 - org.opendaylight.controller.sal-distributed-datastore - 1.3.1.Beryllium-SR1 | updatePeerAddress for peer member-1-shard-toaster-operational with address akka.tcp://opendaylight-cluster-data@10.30.32.233:2550/user/shardmanager-operational/member-1-shard-toaster-operational<br/> 2016-05-03 12:09:02,449 | INFO | lt-dispatcher-16 | ShardManager | 173 - org.opendaylight.controller.sal-distributed-datastore - 1.3.1.Beryllium-SR1 | shard-manager-operational: All Shards are ready - data store operational is ready, available count is 0<br/> 2016-05-03 12:09:02,470 | INFO | NioProcessor-2 | ClientUserAuthServiceNew | 97 - org.apache.sshd.core - 0.14.0 | Received SSH_MSG_USERAUTH_FAILURE<br/> 2016-05-03 12:09:02,474 | INFO | NioProcessor-2 | UserAuthKeyboardInteractive | 97 - org.apache.sshd.core - 0.14.0 | Received Password authentication en-US<br/> 2016-05-03 12:09:02,475 | INFO | NioProcessor-2 | ClientUserAuthServiceNew | 97 - org.apache.sshd.core - 0.14.0 | Received SSH_MSG_USERAUTH_SUCCESS<br/> 2016-05-03 12:09:02,497 | WARN | lt-dispatcher-59 | BaseTopologyManager | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Received a custom identify message from: akka.tcp://opendaylight-cluster-data@10.30.32.98:2550<br/> 2016-05-03 12:09:02,720 | WARN | oupCloseable-6-1 | ClusteredNetconfDevice | 240 - org.opendaylight.netconf.topology - 1.0.1.Beryllium-SR1 | Node netconf-test-device SessionUp, with capabilities NetconfSessionPreferences</p> {capabilities=[urn:ietf:params:netconf:capability:exi:1.0, urn:ietf:params:netconf:capability:candidate:1.0, urn:ietf:params:netconf:base:1.0, urn:ietf:params:netconf:base:1.1], moduleBasedCapabilities=[(urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people?revision=2014-08-18)car-people, (urn:opendaylight:yang:extension:yang-ext?revision=2013-07-09)yang-ext, (urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people?revision=2014-08-18)people, (urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring?revision=2010-10-04)ietf-netconf-monitoring, (urn:opendaylight:test:netconf:crud?revision=2014-10-18)test, (urn:ietf:params:xml:ns:yang:ietf-yang-types?revision=2010-09-24)ietf-yang-types, (urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car?revision=2014-08-18)car, (urn:ietf:params:xml:ns:yang:ietf-inet-types?revision=2010-09-24)ietf-inet-types, (urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring-extension?revision=2013-12-10)ietf-netconf-monitoring-extension, (urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-purchase?revision=2014-08-18)car-purchase], rollback=false, monitoring=true, candidate=true, writableRunning=false} <p>2016-05-03 12:09:02,952 | INFO | lt-dispatcher-17 | NetconfDevice | 238 - org.opendaylight.netconf.sal-netconf-connector - 1.3.1.Beryllium-SR1 | RemoteDevice</p> {netconf-test-device} <p>: Netconf connector initialized successfully<br/> 2016-05-03 12:09:06,476 | WARN | lt-dispatcher-48 | BaseTopologyManager | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Received a custom identify message from: akka.tcp://opendaylight-cluster-data@10.30.32.39:2550<br/> 2016-05-03 12:09:12,434 | INFO | 4]-nio2-thread-1 | ServerSession | 28 - org.apache.sshd.core - 0.12.0 | Server session created from /10.30.32.234:54042<br/> 2016-05-03 12:09:12,443 | INFO | 4]<del>nio2-thread-1 | ServerSession | 28 - org.apache.sshd.core - 0.12.0 | Kex: server</del>&gt;client aes128-ctr hmac-sha1 none<br/> 2016-05-03 12:09:12,443 | INFO | 4]<del>nio2-thread-1 | ServerSession | 28 - org.apache.sshd.core - 0.12.0 | Kex: client</del>&gt;server aes128-ctr hmac-sha1 none<br/> 2016-05-03 12:09:12,508 | INFO | 4]-nio2-thread-3 | ServerUserAuthService | 28 - org.apache.sshd.core - 0.12.0 | Session karaf@/10.30.32.234:54042 authenticated<br/> 2016-05-03 12:09:12,707 | INFO | h for user karaf | command | 45 - org.apache.karaf.log.command - 3.0.3 | ROBOT MESSAGE: Starting test Check_New_Device_Data_Is_Visible_On_node1<br/> 2016-05-03 12:10:53,783 | INFO | h for user karaf | command | 45 - org.apache.karaf.log.command - 3.0.3 | ROBOT MESSAGE: Starting test Kill_node2_Before_Modify</p> <p>(I forgot to mention this was SR1 build, recent Beryllium versions may behave differently.)</p> <p>Interestingly, in the same suite, the same node manages to perform the mount when it gains ownership of the device entity:</p> <p>2016-05-03 12:14:25,236 | INFO | h for user karaf | command | 45 - org.apache.karaf.log.command - 3.0.3 | ROBOT MESSAGE: Starting test Restart_node3_After_Delete_An<br/> d_Dump_Its_Topology_Data<br/> 2016-05-03 12:14:25,241 | WARN | lt-dispatcher-39 | TopologyRoleChangeStrategy | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Gained ownership of entity, r<br/> egistering datastore listener<br/> 2016-05-03 12:14:25,248 | INFO | lt-dispatcher-39 | kka://opendaylight-cluster-data) | 159 - com.typesafe.akka.slf4j - 2.3.14 | Cluster Node [akka.tcp://opendaylight-cluster-data@10.30.32.2<br/> 33:2550] - Trying to join <span class="error">&#91;akka.tcp://opendaylight-cluster-data@10.30.32.233:2550&#93;</span> when already part of a cluster, ignoring<br/> 2016-05-03 12:14:25,254 | WARN | lt-dispatcher-38 | BaseTopologyManager | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | NodeUri [_value=netconf-test-<br/> device] already exists, triggering update..<br/> 2016-05-03 12:14:25,258 | WARN | lt-dispatcher-20 | NetconfDeviceCommunicator | 238 - org.opendaylight.netconf.sal-netconf-connector - 1.3.1.Beryllium-SR1 | RemoteDevice</p> {netconf-test -device} <p>: Session terminated Session closed<br/> 2016-05-03 12:14:25,270 | WARN | lt-dispatcher-38 | LoggingSalNodeWriter | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Delete recieved<br/> 2016-05-03 12:14:25,271 | WARN | lt-dispatcher-38 | LoggingSalNodeWriter | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | NodeId: netconf-test-device<br/> 2016-05-03 12:14:25,272 | WARN | lt-dispatcher-39 | BaseTopologyManager | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Delete part of update succesf<br/> ull, triggering create<br/> 2016-05-03 12:14:25,275 | WARN | lt-dispatcher-18 | LoggingSalNodeWriter | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Init recieved<br/> 2016-05-03 12:14:25,275 | WARN | lt-dispatcher-18 | LoggingSalNodeWriter | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | NodeId: netconf-test-device<br/> 2016-05-03 12:14:25,275 | WARN | lt-dispatcher-18 | LoggingSalNodeWriter | 241 - org.opendaylight.netconf.abstract-topology - 1.0.1.Beryllium-SR1 | Node: Node [_key=NodeKey [_no<br/> deId=Uri <span class="error">&#91;_value=netconf-test-device&#93;</span>], _nodeId=Uri <span class="error">&#91;_value=netconf-test-device&#93;</span>, augmentation=[NetconfNode [_availableCapabilities=AvailableCapabilities [_availableCapability=[], augmentati<br/> on=[]], _clusteredConnectionStatus=ClusteredConnectionStatus [_nodeStatus=[NodeStatus [_node=akka.tcp://opendaylight-cluster-data@10.30.32.233:2550, _status=Unavailable, augmentation=[]]], a<br/> ugmentation=[]], _connectionStatus=Connecting, _host=Host [_ipAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.32.148&#93;</span>, _value=<span class="error">&#91;1, 0, ., 3, 0, ., 3, 2, ., 1, 4, 8&#93;</span>], _value=[1, 0, .<br/> , 3, 0, ., 3, 2, ., 1, 4, 8]], _port=PortNumber <span class="error">&#91;_value=17830&#93;</span>, _unavailableCapabilities=UnavailableCapabilities [_unavailableCapability=[], augmentation=[]]]]]<br/> 2016-05-03 12:14:25,282 | INFO | lt-dispatcher-38 | AbstractNetconfTopology | 240 - org.opendaylight.netconf.topology - 1.0.1.Beryllium-SR1 | Connecting RemoteDevice</p> {Uri [_value=ne tconf-test-device]} <p> , with config Node{getNodeId=Uri <span class="error">&#91;_value=netconf-test-device&#93;</span>, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.netconf.node.topology.rev150114.Netc<br/> onfNode=NetconfNode{getBetweenAttemptsTimeoutMillis=2000, getConnectionTimeoutMillis=20000, getCredentials=LoginPassword{getPassword=topsecret, getUsername=admin, augmentations={}}, getDefau<br/> ltRequestTimeoutMillis=60000, getHost=Host [_ipAddress=IpAddress [_ipv4Address=Ipv4Address <span class="error">&#91;_value=10.30.32.148&#93;</span>, _value=<span class="error">&#91;1, 0, ., 3, 0, ., 3, 2, ., 1, 4, 8&#93;</span>], _value=[1, 0, ., 3, 0, ., 3, 2<br/> , ., 1, 4, 8]], getKeepaliveDelay=0, getMaxConnectionAttempts=0, getPort=PortNumber <span class="error">&#91;_value=17830&#93;</span>, getSchemaCacheDirectory=schema, getSleepFactor=1.5, isReconnectOnChangedSchema=false, isTc<br/> pOnly=false}}}</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/45147/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/45147/</a></p> <p>Fixed with new netconf clustering topology implementation: <a href="https://git.opendaylight.org/gerrit/#/c/46660/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/46660/</a></p> Development External issue ID 5761 External issue URL ODL SR Target Milestone Rank 0|i01wzj: