Details
-
New Feature
-
Status: Resolved
-
Highest
-
Resolution: Done
-
None
-
None
Description
Current Neon-SR2 streaming format has a few deficiencies:
- it encodes MapEntry key values twice, once in the leaf itself and once in the entry's NodeIdentifierWithPredicates
- it is 'zero-happy', i.e. they do not recognize that integer codes and sizes are typically much smaller than the 4-byte range and thus much of the stream is just small ints encoded as 4 bytes
- while they are simple and straighforward, they also end up wasting bits – the four token families (codes, value types, node types, path argument types) each have at most 15 distinct values, and hence the bytes used to transmit them have 4 empty bits (or more)
Furthermore it cannot deal with Uint8/16/32/64 types, which is a must for the MRI window.
Attachments
Issue Links
- is blocked by
-
CONTROLLER-1889 Rework NormalizedNodePruner in terms of a forwarder
-
- Resolved
-
- is duplicated by
-
CONTROLLER-1909 Deduplicate NodeIdentifierWithPredicate/LeafNode values when writing streams
-
- Resolved
-
- relates to
-
CONTROLLER-2052 cds-access-api uses wrong ABI version
-
- Resolved
-
| # | Subject | Branch | Project | Status | CR | V |
|---|---|---|---|---|---|---|
| 84274,53 | Add LithiumSR1 input/output support | master | controller | Status: MERGED | +2 | +1 |
| 84539,40 | Switch default stream output to Sodium SR1 | master | controller | Status: MERGED | +2 | +1 |
| 84550,1 | Add encoding size asserts | master | controller | Status: MERGED | +2 | +1 |
| 84552,2 | Add a 100K-entry test | master | controller | Status: MERGED | +2 | +1 |
| 84557,13 | Add Magnesium encoding tokens | master | controller | Status: MERGED | +2 | +1 |
| 84559,3 | Move Lithium tokens to their own class | master | controller | Status: MERGED | +2 | +1 |
| 84560,3 | Rename NodeTypes to LithiumNode | master | controller | Status: MERGED | +2 | +1 |
| 84561,3 | Rename PathArgumentTypes to LithiumPathArgument | master | controller | Status: MERGED | +2 | +1 |
| 84562,3 | Rename ValueTypes to LithiumValue | master | controller | Status: MERGED | +2 | +1 |
| 84563,3 | Move Neon SR2 tokens into its own class | master | controller | Status: MERGED | +2 | +1 |
| 84662,2 | Add a 100K-entry test | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84663,1 | Add encoding size asserts | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84691,2 | Add an explicit namespace sharing test | master | controller | Status: MERGED | +2 | +1 |
| 84694,8 | Add Magnesium stream version | master | controller | Status: MERGED | +2 | +1 |
| 84695,23 | Switch default stream output to Magnesium | master | controller | Status: MERGED | +2 | +1 |
| 84696,2 | Add an explicit namespace sharing test | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84697,1 | Add an explicit namespace sharing test | stable/neon | controller | Status: ABANDONED | 0 | 0 |
| 84699,4 | Add cds-access-api SODIUM_SR1 version | master | controller | Status: MERGED | +2 | +1 |
| 84701,3 | Define DataStoreVersions.SODIUM_SR1_VERSION | master | controller | Status: MERGED | +2 | +1 |
| 84702,9 | Define PayloadVersion.SODIUM_SR1 | master | controller | Status: MERGED | +2 | +1 |
| 84703,10 | Define DataStoreVersions.MAGNESIUM_VERSION | master | controller | Status: MERGED | +2 | +1 |
| 84704,9 | Add cds-access-api MAGNESIUM version | master | controller | Status: MERGED | +2 | +1 |
| 84705,10 | Define PayloadVersion.MAGNESIUM | master | controller | Status: MERGED | +2 | +1 |
| 84707,3 | Use explicit versioning in MetadataShardDataTreeSnapshot | master | controller | Status: MERGED | +2 | +1 |
| 84708,1 | Switch ABIVersion/DataStoreVersions back to Neon SR2 | master | controller | Status: MERGED | +2 | +1 |
| 84709,4 | Switch current {ABI,DataStore,Payload}Version to Sodium SR1 | master | controller | Status: MERGED | +2 | +1 |
| 84710,7 | Switch current {ABI,DataStore,Payload}Version to Magnesium | master | controller | Status: MERGED | +2 | +1 |
| 84716,3 | Add uint adaptation to AbstractNormalizedNodePruner | master | controller | Status: MERGED | +2 | +1 |
| 84717,1 | Use explicit versioning in MetadataShardDataTreeSnapshot | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84718,2 | Use explicit versioning in MetadataShardDataTreeSnapshot | stable/neon | controller | Status: MERGED | +2 | +1 |
| 84723,1 | Move Lithium tokens to their own class | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84724,1 | Rename NodeTypes to LithiumNode | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84725,1 | Rename PathArgumentTypes to LithiumPathArgument | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84726,1 | Rename ValueTypes to LithiumValue | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84727,1 | Move Neon SR2 tokens into its own class | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84728,1 | Add Magnesium encoding tokens | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84729,1 | Add LithiumSR1 input/output support | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84730,1 | Add Magnesium stream version | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84731,1 | Add cds-access-api SODIUM_SR1 version | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84732,1 | Define DataStoreVersions.SODIUM_SR1_VERSION | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84733,1 | Switch ABIVersion/DataStoreVersions back to Neon SR2 | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84734,1 | Define PayloadVersion.SODIUM_SR1 | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84735,1 | Add cds-access-api MAGNESIUM version | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84736,1 | Define DataStoreVersions.MAGNESIUM_VERSION | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 84737,1 | Define PayloadVersion.MAGNESIUM | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 85098,2 | Use careful byte-masking/shifting in Mg Input | stable/sodium | controller | Status: MERGED | +2 | +1 |
| 85099,2 | Use careful byte-masking/shifting in Mg Input | master | controller | Status: MERGED | +2 | +1 |
| 88778,1 | Switch current {ABI,DataStore,Payload}Version to Sodium SR1 | stable/sodium | controller | Status: MERGED | +2 | +1 |