https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/MDSAL-456 mdsal <p>Before neon MRI upgrade, the output of oper inventory when no switch is connected was:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>GET /restconf/operational/opendaylight-inventory:nodes 200 OK {}</pre> </div></div> <p>After neon MRI upgrade, this did not change when the controller starts for the first time, however when switch connects and disconnects, the above changes to:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>GET /restconf/operational/opendaylight-inventory:nodes 404 Request could not be completed because the relevant data model content does not exist </pre> </div></div> <p> </p> <p>Because of the above we had to remove some OFP test.</p> MDSAL-456

Inconsistent list handling in Binding datastore operations

Bug Highest Resolved Done Robert Varga Luis Gomez Fri, 16 Nov 2018 19:07:40 +0000 Fri, 1 May 2020 12:20:17 +0000 Wed, 22 Apr 2020 08:22:33 +0000 3.0.8 4.0.2 6.0.0 0 2 <p>This actually explains, however it is weird it does not work the first time the controller boots up:<br/> <a href="https://wiki.opendaylight.org/view/Neon_platform_upgrade#Datastore_lifecycle" class="external-link" target="_blank" rel="nofollow noopener">https://wiki.opendaylight.org/view/Neon_platform_upgrade#Datastore_lifecycle</a></p> <p>I would say either we close this because the change impact is minimal or we move the issue to yangtools projects for further investigation.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=rovarga" class="user-hover" rel="rovarga">rovarga</a> any thoughts on this issue?</p> <p>As documented here: <a href="https://wiki.opendaylight.org/view/Neon_platform_upgrade#DataTree_removes_empty_lists.2C_choices_and_augmentations" class="external-link" target="_blank" rel="nofollow noopener">https://wiki.opendaylight.org/view/Neon_platform_upgrade#DataTree_removes_empty_lists.2C_choices_and_augmentations</a> the list will disappear, so the outcome is correct.</p> <p>We understand the change in behavior but we do not understand why this change does not apply the first time the controller boots up: if I boot the controller for the first time and the switches are not connected I get 200 OK + empty list, now if I connect a switch and disconnect the same, then I get 404, so it seems to me the API is not consistent or there is a bug during the boot up process.</p> <p>Ah, now I understand the inconsistency. I suspect this is due to how the topology is initialized. What are the transactions OFP executes during init?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=Avishnoi" class="user-hover" rel="Avishnoi">Avishnoi</a>, can you answer Roberts's question?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=rovarga" class="user-hover" rel="rovarga">rovarga</a> It's not topology, but inventory. During initialization it initialize the operational datastore with empty list</p> <p> </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">final</span> WriteTransaction tx = dataBroker.newWriteOnlyTransaction(); <span class="code-keyword">try</span> { tx.merge(LogicalDatastoreType.OPERATIONAL, InstanceIdentifier.create(Nodes.class), <span class="code-keyword">new</span> NodesBuilder() .setNode(Collections.emptyList()) .build()); tx.commit().get(); } <span class="code-keyword">catch</span> (ExecutionException | InterruptedException e) { LOG.error(<span class="code-quote">"Creation of node failed."</span>, e); <span class="code-keyword">throw</span> <span class="code-keyword">new</span> IllegalStateException(e); } </pre> </div></div> <p>After the controller starts and i fetch GET <a href="http://" class="external-link" target="_blank" rel="nofollow noopener">http://\</a><tt>controller-ip</tt>:8181/restconf/operational/opendaylight-inventory:nodes</p> <p>it shows me </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> {}</pre> </div></div> <p>than i connect the device and it shows me device list in following json for the same above url</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> { <span class="code-quote">"nodes"</span>: { <span class="code-quote">"node"</span>: ...... <span class="code-comment">//all nodes details </span> } }</pre> </div></div> <p>After devices are added to operational datastore it shows "nodes" list but in the previous case (Fresh restart), it was just showing empty list {} ( i thought it should have shown</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> { <span class="code-quote">"nodes"</span> :{} }</pre> </div></div> <p>Then i disconnect all the switches (it removes individual nodes)</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> @Override <span class="code-keyword">public</span> ListenableFuture&lt;?&gt; removeDeviceFromOperationalDS(@Nonnull <span class="code-keyword">final</span> KeyedInstanceIdentifier&lt;Node, NodeKey&gt; ii) { <span class="code-keyword">final</span> WriteTransaction delWtx = dataBroker.newWriteOnlyTransaction(); delWtx.delete(LogicalDatastoreType.OPERATIONAL, ii); <span class="code-keyword">return</span> delWtx.commit(); } </pre> </div></div> <p>and when i fetch the operational nodes again it throws error (404)</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> { <span class="code-quote">"errors"</span>: { <span class="code-quote">"error"</span>: [ { <span class="code-quote">"error-type"</span>: <span class="code-quote">"application"</span>, <span class="code-quote">"error-tag"</span>: <span class="code-quote">"data-missing"</span>, <span class="code-quote">"error-message"</span>: <span class="code-quote">"Request could not be completed because the relevant data model content does not exist"</span> } ] } } </pre> </div></div> <p> </p> <p>So if you do not do .setNode(Collection.emptyList()), things should be consistent. The reason why this is happening is that merge() will turned into a write() of nodes, which contains an empty list. We do not examine the contents of written data to eliminate empty lists etc.</p> <p>I think that's where the inconsistency in behavior is, so if user is adding element to that empty list and than removing all of it, which results in the same empty list, why does it throws error rather than responding with empty list?</p> <p>Since the list becomes empty, it is removed (along with any of its insignificant parents). The initial state of having the empty list present is wrong, hence the simplest solution is <a href="https://git.opendaylight.org/gerrit/82167" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/82167</a> .</p> <p>In my opinion, we have a API inconsistency here. Either the empty list/ container, should not be allowed while writing to the data store or the empty container/list should not be removed when all the elements are removed from it, but we are seeing different behavior for write and delete API's here. To make thing worst, these workarounds are something user will have to keep in mind while they program, where they don't have clear guidline on do's and don'ts. </p> <p>Well yes, but that inconsistency does not lie solely in yangtools, because:</p> <ul> <li>yangtools is providing DataTree API and an implementation</li> <li>you are talking to mdsal-binding-api, which is a completely unrelated API</li> <li>there is a stack of components (and since it's Operational store, which is internal, OFP is a privileged application, hence mitigations there are a fair game)</li> </ul> <p>And the end of the day, this boils down to OFP writing and empty List through Binding APIs, which translates do an empty map, which is retained by InMemoryDataTree.</p> <p>From yangtools perspective, it is an operational write, which we are expecting to be consistent – hence we trust the data to be minimaly-conformant. Unrestricted empty lists are okay from validation perspective, hence there is not much we are doing wrong. Fixing the issue at this level would require an additional check (incurred at every write) and possible NormalizedNode rebuild (if we find an empty list). If at all possible, we want to solve this problem at a different layer.</p> <p>Note that lists are addressable in DOM, but not addressable in Binding – hence Binding has nonnullFoo() bridges, as we are fully expecting people to treat empty and null lists the same (and anyway, that's an MD-SAL issue).</p> <p>That brings a possible solution: empty lists should be pruned completely when translated from binding. That obviously has impacts, so needs to be considered carefully. If you can provide an estimate of breakage this would bring, that would be appreciated.</p> <p>Whatever we do, the empty list merged by OFP will become a no-op and therefore, as a matter of efficiency, proposed OFP patch is an obvious simplification and is therefore the right thing to remove it.</p> <p>As for DO's and DON'T's, there are always going to be some. Some of them are "this is really a bad idea", and as genius shows, that will not dissuade people. Some of them are "yeah, okay, this really means nothing", which is the case here (once solved).</p> <p>Oh, and btw. the end result is better experience for applications – they have even less to worry about. Existing applications should simplify where possible, i.e. eliminate every single line of code, which is not required.</p> <p><a href="https://git.opendaylight.org/gerrit/82558" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/82558</a> implements the binding-level change: we do not emit empty lists. <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> can you give it a test-drive across all of CSIT?</p> <p>OK, I created distribution with multi-patch: <a href="https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/integration/distribution/karaf/0.11.0-SNAPSHOT/karaf-0.11.0-20190625.163317-26.zip" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/integration/distribution/karaf/0.11.0-SNAPSHOT/karaf-0.11.0-20190625.163317-26.zip</a></p> <p>And run few OFP tests and saw no problem. Should we run more tests?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> i think we need to revert <a href="https://git.opendaylight.org/gerrit/#/c/77908/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/77908/</a> and run the test again to confirm the fix?</p> <p>Yes, the revert patch + MDSAL change works here:</p> <p><a href="https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node-gate-clustering-bulkomatic-only-sodium/76/" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node-gate-clustering-bulkomatic-only-sodium/76/</a></p> <p>What is actually breaking is this: <a href="https://github.com/opendaylight/ovsdb/blob/master/hwvtepsouthbound/hwvtepsouthbound-impl/src/main/java/org/opendaylight/ovsdb/hwvtepsouthbound/transact/McastMacsRemoteRemoveCommand.java#L125" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/opendaylight/ovsdb/blob/master/hwvtepsouthbound/hwvtepsouthbound-impl/src/main/java/org/opendaylight/ovsdb/hwvtepsouthbound/transact/McastMacsRemoteRemoveCommand.java#L125</a></p> <p>I.e. the change here is that we get null vs. empty list, which ends up being compared as non-equal. This will be addressed in Aluminium as part of <a href="https://jira.opendaylight.org/browse/MDSAL-449" title="Empty lists/maps have no semantic meaning in Binding" class="issue-link" data-issue-key="MDSAL-449"><del>MDSAL-449</del></a>.</p> Blocks MDSAL-449 Relates YANGTOOLS-585 Development Rank 0|i03krr: