[CONTROLLER-1909] Deduplicate NodeIdentifierWithPredicate/LeafNode values when writing streams Created: 26/Aug/19  Updated: 21/Feb/20  Resolved: 21/Feb/20

Status: Resolved
Project: controller
Component/s: clustering
Affects Version/s: None
Fix Version/s: Magnesium

Type: Improvement Priority: Medium
Reporter: Robert Varga Assignee: Robert Varga
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
is blocked by CONTROLLER-1908 Deduplicate NodeIdentifierWithPredica... Resolved
Duplicate
duplicates CONTROLLER-1919 Add Magnesium streaming format Resolved

 Description   

This is a follow-up on CONTROLLER-1908, where we should understand where a LeafNode inside a MapNode corresponds to a key and thus has had its value emitted as part of the NodeIdentifierWithPredicates – these are expected to always match.

While CONTROLLER-1908 deals with the memory footprint of data restored from the stream, this issue is about not emitting duplicate values (or even QNames) for leaves that are covered in the predicates.

 

 Comments   
Comment by Robert Varga [ 26/Aug/19 ]

There are really two basic ways to approach this:

  • emit NodeIdentifierWithPredicates as usual, deduplicate leaf values (perhaps with dedicated 'key leaf' item type)
  • emit leaf values (i.e. the entire MapNode first and then write out the QNames of keys

or perhaps some hybrid approach.

Comment by Robert Varga [ 26/Aug/19 ]

For leaf-lists the, storing is quite clear – we do not need to emit values twice, no matter what – but they are not as common as MapNodes

Comment by Robert Varga [ 21/Feb/20 ]

This has been implemented as part of CONTROLLER-1919

Generated at Wed Feb 07 19:56:45 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.