[YANGTOOLS-1020] Add runtime-generated Immutable maps Created: 27/Aug/19  Updated: 05/Jan/24

Status: Confirmed
Project: yangtools
Component/s: data-impl
Affects Version/s: None
Fix Version/s: 14.0.0

Type: New Feature Priority: Medium
Reporter: Robert Varga Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
blocks YANGTOOLS-1021 Add runtime-generated NormalizedNode ... Confirmed
Relates
relates to YANGTOOLS-490 Create SchemaContext-aware Normalized... Confirmed

 Description   

Heap dump analysis of an OFP-related use case is showing that even though we are using ImmutableOffsetMaps to hold children, those maps still account for 2.4M (10%) objects holding 55MiB (9%) shallow memory, with additional 55MiB (9%) retained by their Object[]s.

The Object array size ranges from 2 to 12 elements, which means they have 60% to 20% overhead.

We should be able to bring these overheads down by application of runtime code generation, where we'd generate the equivalent of ImmutableOffsetMap.Unordered, where the values are stored directly in fields of the generated class. This would eliminate the need for Object[]s – i.e. 7% of heap occupancy.

Code generation should be based on Byte-Buddy, i.e. generate bytecode.

The implementation should assume NodeIdentifier addressing, i.e. the keys are strictly based on QNames. These should be destructured during internal dispatch, so that we first compare QNameModule portion (which is typically unique, but there may be others, i.e. in case of augment), then we should perform a simple switch() on localName.

The bytecode should be roughly equivalent to:

// Hand-coded base class
public abstract class AbstractNodeIdentifierMap<V> extends AbstractMap<NodeIdentifier, V> implements Immutable {
  @Override
  public final V get(Object obj) {
    return obj instanceof NodeIdentifier nid ? get(nid.getType()) : null;
  }

  protected abstract V get(QName qname);
}

// Generated
final class NodeIdenfierMapXXYYZZ<V> extends AbstractNodeIdentifierMap<V> {
  // ... for each child QNameModule
  private static final QNameModule MODULE0;
  // ... also we may need constants for QNames for things like entrySet(), keySet(), etc.
  
  // actually something reasonably-derived from QName?
  private final V v0_0;
  private final V v0_1;

  @Override
  protected V get(QName qnamej) {
    final var module = qname.getModule();
    if (MODULE0.equals(module)) {
        return get0(qname.getLocalName());
    }
    // else ... if we have more QNameModule

    return null;
  }

  // peeled for each QNameModule
  private V get0(String localName) {
    return switch (localName) {
      case "some-name" -> field0_0;
      case "other-name" -> field0_1;
      default -> null;
    }
  }
}


 Comments   
Comment by Robert Varga [ 27/Aug/19 ]

Note that these maps will need to implement efficient expansion/contraction via ModifiableMapPhase, so they can re-done during DataTree operations.

Generated at Wed Feb 07 20:54:57 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.