https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/YANGTOOLS-922 yangtools <p>Our current implementation is using LinkedHashMap to track children, which is not entirely efficient, as we are forcing copies on each modification.</p> <p>While a trivial optimization would be to switch to copying ImmutableMaps, this is not sufficient for the purposes of <a href="https://jira.opendaylight.org/browse/YANGTOOLS-921" title="Design insert/move operations on top of ordered (leaf-)lists" class="issue-link" data-issue-key="YANGTOOLS-921">YANGTOOLS-921</a>, as the lists involved can become quite large and we will require some sort of snapshot handling to support the operations.</p> YANGTOOLS-922

Improve ordered list tracking

Improvement High Confirmed Unresolved Unassigned Robert Varga Sun, 2 Dec 2018 17:55:58 +0000 Tue, 19 Sep 2023 16:20:40 +0000 14.0.0 data-impl 0 2 <p>This really boils down to defining an IndexedList interface, which aside from being a java.util.List provides:</p> <ol> <li>lookups by key, where the key may be a subset of value data</li> <li>insert operation</li> <li>move operation</li> <li>explicit mutable transitions, similar to ModifiableMapPhase and UnmodifiableMapPhase</li> </ol> <p>Most significant problem here is maintenance of key-based indices in face of an insert/move which ends up shifting positions of the nodes. While we could have trivial secondary Map&lt;Key, Integer&gt;, maintaining that map would probably kill performance. It would seem that the secondary index needs to be somehow integrated with individual entries – perhaps a skip list would do the trick?</p> <p> </p> <p>For the by-offset part we want something which behaves like a <a href="https://en.wikipedia.org/wiki/Rope_(data_structure)" class="external-link" target="_blank" rel="nofollow noopener">rope</a> generalized for not char but object storage. That should essentially take care of in-order traversal (via Iterable.iterator()) as well random access (via List.get(int)) rather easy to implement.</p> <p>An interesting problem here is going to be limiting fragmentation, such that we do not devolve to having a million two-item end nodes. This amounts to controlling the balance between the number of items stored and tree depth in a hypothetical </p> <p> </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">abstract</span> <span class="code-keyword">class </span>RopeNode; <span class="code-keyword">class </span>LeafNode <span class="code-keyword">extends</span> RopeNode { <span class="code-keyword">final</span> <span class="code-object">Object</span>[] values; <span class="code-comment">// erasure of &lt;V&gt; </span>} <span class="code-keyword">class </span>TrunkNode <span class="code-keyword">extends</span> RopeNode { <span class="code-keyword">final</span> <span class="code-object">int</span> leftWeight; <span class="code-keyword">final</span> RopeNode left; <span class="code-keyword">final</span> RopeNode right; } </pre> </div></div> <p>Maybe tracking depth and correlating it to something?</p> <p> </p> <p>In terms of supported operations, we need to support  <a href="https://datatracker.ietf.org/doc/html/rfc8072#section-2.5" class="external-link" target="_blank" rel="nofollow noopener">YANG patch operations</a> at minimum.</p> <p>I am not sure about the by-key part, as it seems like it needs to point somehow to a place in the rope.</p> <p>I think the key realization here is that each list is a sum of edits on it, similar to what <a href="https://en.wikipedia.org/wiki/Zipper_(data_structure)" class="external-link" target="_blank" rel="nofollow noopener">zipper</a> does.</p> <p>A tangential thought: Xanadu's Ent is <a href="https://web.archive.org/web/20071027094217/http://www.sunless-sea.net/wiki/Mark%20Miller_s%20Ent%20Class%20_a%20transcript_" class="external-link" target="_blank" rel="nofollow noopener">described</a> as</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>The way we get the ability to do this, in our structure we've got other kinds of nodes in this tree which we refer to as DspLoafs?. Don't worry about the derivation of the term right now. Actually Dsp- is for displacement. So this DspLoaf? might have in it for instance a plus three, what that means is take all the numbers that you encounter navigating downward from here and consider them to be offset by three, and these things get accumulated so over here if you've got a plus seven, and then over here, the distinction, we refer to as the split, is a four, then the split is really considered to be a 14, so what that allows us to do is when we have the insertion point here and you start typing, to basically rearrange the tree so that the distinction between these two parts of the document get represented as a distinction in the tree and then offset this part of the tree by this amount. This allows us to edit these large documents without bringing hardly any of them into core. Leaving most of them on disk untouched.</pre> </div></div> <p>Now this displacement calculus, along with a zipper (which would hold the effective offset?) sounds like one way to deal with edits.</p> <p>I wonder whether we could get away with an offset-based reference, where we have two classes of keys:</p> <ul> <li>keys which reference the first item in a rope leaf (i.e. end node) map to a linear offset (and hence can be quickly found)</li> <li>keys which reference any other item map to the key of the first item plus a linear delta</li> </ul> <p>Hence any item can be located at most through two lookups by key and one lookup by offset. This additional indirection should allow us to efficiently maintain the index, as we really need to reindex only when the shape of the leaf changes significantly, such as when a lead item is moved between leafs (or is removed). When an item is prepended into a leaf, we might get by without reindexing at the cost of additional by-key lookups.</p> <p>Perhaps we can use some sort of index/op counting to keep track of our worst-case lookup for a particular leaf and decide to reindex (which means N by-key replace operations where N is the number of items in the leaf).</p> <p> </p> <p>The modifiable phase essentially boils down to maintaining a zipper and collapsing it to a base rope/index structure.</p> <p>Does that mean we need to support access by integer offset (as list index)?</p> <p>The type name "target-resource-offset" suggest yes, but patching is only intended for configuration data, so lists always have keys (and leaf-lists are unique), and the examples use values such as "6" not as an offset, but as an actual string song name (key).</p> <p>If no, we need only access by key, which simplifies matters considerable as we no longer need to renumber offsets.</p> <p>Yes, we do need that – it is implied by java.util.List.</p> Blocks YANGTOOLS-921 Relates YANGTOOLS-920 Development Rank 0|i03l8v: