https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/NETCONF-456 netconf <p>This doesn't happen everytime.</p> <p>The environment is Oxygen three node cluster.</p> <p>Node mount configuration:</p> { "netconf-node-topology:between-attempts-timeout-millis": 10000, "netconf-node-topology:connection-timeout-millis": 10000, "netconf-node-topology:default-request-timeout-millis": 10000, "netconf-node-topology:host": "192.168.10.26", "netconf-node-topology:keepalive-delay": 0, "netconf-node-topology:max-connection-attempts": 10, "netconf-node-topology:password": "HBfU1gKHExw89lMB3juq1g==", "netconf-node-topology:port": 2831, "netconf-node-topology:tcp-only": false, "netconf-node-topology:username": "HBfU1gKHExw89lMB3juq1g==", "node-id": "overcloud-novacompute-0.opnfv.org" } <p>Attached log corresponds to one of our nodes being stuck in connecting state:<br/> <span class="error">&#91;root@overcloud-controller-0 scripts&#93;</span># curl -XGET -u admin:admin <a href="http://$(grep" class="external-link" target="_blank" rel="nofollow noopener">http://$(grep</a> host /opt/opendaylight/etc/jetty.xml | grep -Eo '(<span class="error">&#91;0-9&#93;</span></p> {1,3}.){3}<span class="error">&#91;0-9&#93;</span>{1,3} <p>' | head -1):8081/restconf/operational/network-topology:network-topology/ | python -mjson.tool | grep netconf-node-topology:connection-status -A 4<br/> "netconf-node-topology:connection-status": "connecting",<br/> "netconf-node-topology:host": "192.168.10.26",<br/> "netconf-node-topology:port": 2831,<br/> "netconf-node-topology:unavailable-capabilities": {},<br/> "node-id": "overcloud-novacompute-0.opnfv.org"<br/> –<br/> "netconf-node-topology:connection-status": "connected",<br/> "netconf-node-topology:host": "192.168.10.22",<br/> "netconf-node-topology:port": 2831,<br/> "netconf-node-topology:unavailable-capabilities": {},<br/> "node-id": "overcloud-controller-1.opnfv.org"<br/> –<br/> "netconf-node-topology:connection-status": "connected",<br/> "netconf-node-topology:host": "192.168.10.14",<br/> "netconf-node-topology:port": 2831,<br/> "netconf-node-topology:unavailable-capabilities": {},<br/> "node-id": "overcloud-controller-0.opnfv.org"<br/> –<br/> "netconf-node-topology:connection-status": "connected",<br/> "netconf-node-topology:host": "192.168.10.20",<br/> "netconf-node-topology:port": 2831,<br/> "netconf-node-topology:unavailable-capabilities": {},<br/> "node-id": "overcloud-controller-2.opnfv.org"<br/> –<br/> "netconf-node-topology:connection-status": "connected",<br/> "netconf-node-topology:host": "192.168.10.24",<br/> "netconf-node-topology:port": 2831,<br/> "netconf-node-topology:unavailable-capabilities": {},<br/> "node-id": "overcloud-novacompute-1.opnfv.org"</p> <p>Operating System: Linux<br/> Platform: PC</p> NETCONF-456

Honeycomb mounting sometimes fails in Oxygen

Bug Resolved Done Tomas Cere Juraj Linkeš Wed, 23 Aug 2017 09:18:57 +0000 Fri, 15 Mar 2019 22:22:44 +0000 Mon, 25 Sep 2017 11:47:20 +0000 netconf 0 4 <p>Attachment compute0-karaf.zip has been added with description: karaf log</p> <p>First of all, I have to mention that this issue only with clustered netconf topology.</p> <p>This seems to be related to patch <a href="https://git.opendaylight.org/gerrit/#/c/52528/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/52528/</a>. This patch introduced netconf mountpoints credentials encryption. Now, I won't go into detail how does this credentials encryption exactly work, I have already described it in <a href="https://jira.opendaylight.org/browse/NETCONF-450" title="Failed to decrypt encoded data while mounting NETCONF" class="issue-link" data-issue-key="NETCONF-450"><del>NETCONF-450</del></a> and <a href="https://jira.opendaylight.org/browse/NETCONF-453" title="odl-netconf-topology creates two parallel connection for each configured device" class="issue-link" data-issue-key="NETCONF-453"><del>NETCONF-453</del></a>.</p> <p>The problem is this second DS write of netconf node config with encrypted credentials into netconf-topology. This triggers the reconfiguration of netconf node mountpoint. In the case that mountpoint has not been properly initialized, this reconfiguration can cause some problems. I think it's issue with some incorrect resource cleanup after disconnecting the old connection and closing the sal facade as a first step of reconfiguration process.</p> <p>Is this a blocker for nitrogen? Please confirm. Thanks!</p> <p>Yes, it is a blocker for Nitrogen.</p> <p>There is some discrepancy between stable/nitrogen and stable/carbon branches. We have to cherry-pick some patches to nitrogen (and master) so we will end up in the same state regarding the credentials encryption as in carbon.</p> <p>There was a consensus that we should not automatically encrypt mountpoint's credentials because it triggers the problematic second data tree change event. So the original patch that introduced it was reverted in carbon. </p> <p>Cherry-pick to nitrogen:<br/> <a href="https://git.opendaylight.org/gerrit/#/c/62680/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/62680/</a></p> <p>To allow users to encrypt mountpoint's credentials, new add-netconf-node RPC is introduced. This RPC does not issue problematic second write of the same netconf node to DS, so this approach does not suffer from the issues the original approach suffers from.</p> <p>Cherry-picks to nitrogen:<br/> <a href="https://git.opendaylight.org/gerrit/#/c/62681/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/62681/</a><br/> <a href="https://git.opendaylight.org/gerrit/#/c/62682/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/62682/</a></p> <p>The first patch actually introduces SSH public key authentication to netconf southbound. We need this patch to avoid complex cherry-pick of the second patch (that is actual add-netconf-node RPC), because it depends on some functionality the first patch introduces. Anyway, the public key authentication is present on Carbon and should be present also on Nitrogen.</p> <p>We also depends on some util stuff from AAA project and we need to port that util stuff to Nitrogen too.</p> <p>Cherry-pick to nitrogen:<br/> <a href="https://git.opendaylight.org/gerrit/#/c/62686/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/62686/</a></p> <p>With all of this in nitrogen, the bug should not be present anymore. There will be no automatic second writes to DS for each new netconf node, so there won't be no reconnects during mountpoints creation.</p> <p>However, we can still hit this when application or user will try to write the same node to netconf-topology multiple times in short time. I am going to open a new bug for that, since that is not blocker and does not happen automatically on our side with each new netconf node as this bug itself.</p> <p>The patches are in Nitrogen already. We have to cherry-pick them also to Oxygen.</p> <p>All cherry-picks should be in Oxygen now</p> Duplicate INTTEST-12 Development External issue ID 9033 External issue URL ODL SR Target Milestone Rank 0|i01ynj: