https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/CONTROLLER-1871 controller <p>I'm looking at a Java Flight Recording obtained from (internal) scale lab testing, based on Oxygen SR3 code, and see a lot (cumulated as in millions, with many tens of GBs) of "TLAB Allocations" related to an Optional.of() Optional.ofNullable() from QName/QNameModule.getRevision() from NormalizedNodeOutputStreamWriter.</p> <p>Wondering if perhaps we could improve upon this in any way? Realizing this may not be trivial, and <em>generally speaking</em> Optional is much preferred over using a <tt>@Nullable</tt>, but wondering if this is an exceptional case where the Optional should perhaps be avoided?</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Optional java.util.Optional.of(<span class="code-object">Object</span>) 36282 Optional java.util.Optional.ofNullable(<span class="code-object">Object</span>) 31841 Optional org.opendaylight.yangtools.yang.common.QNameModule.getRevision() 11703 Optional org.opendaylight.yangtools.yang.common.QName.getRevision() 11703 void org.opendaylight.controller.cluster.datastore.node.utils.stream.NormalizedNodeOutputStreamWriter.writeQName(QName) 11703 void org.opendaylight.controller.cluster.datastore.node.utils.stream.AbstractNormalizedNodeDataOutput.startNode(QName, <span class="code-object">byte</span>) 8215 void org.opendaylight.controller.cluster.datastore.node.utils.stream.AbstractNormalizedNodeDataOutput.leafNode(YangInstanceIdentifier$NodeIdentifier, <span class="code-object">Object</span>) 5324 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessAsSimpleNode(NormalizedNode) 5324 NormalizedNodeWriter org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNode) 5324 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.writeChildren(Iterable) 4596 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNode) 3924 NormalizedNodeWriter org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNode) 3924 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.writeChildren(Iterable) 2301 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNode) 2284 NormalizedNodeWriter org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNode) 2284 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.writeChildren(Iterable) 1759 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter$OrderedNormalizedNodeWriter.writeMapEntryNode(MapEntryNode) 1485 <span class="code-object">boolean</span> org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNode) 1485 NormalizedNodeWriter org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNode) 1485 void org.opendaylight.controller.cluster.datastore.node.utils.stream.AbstractNormalizedNodeDataOutput.writeNormalizedNode(NormalizedNode) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeNode(NormalizedNodeDataOutput, DataTreeCandidateNode) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeChildren(NormalizedNodeDataOutput, Collection) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeNode(NormalizedNodeDataOutput, DataTreeCandidateNode) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeChildren(NormalizedNodeDataOutput, Collection) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeNode(NormalizedNodeDataOutput, DataTreeCandidateNode) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeChildren(NormalizedNodeDataOutput, Collection) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeNode(NormalizedNodeDataOutput, DataTreeCandidateNode) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeChildren(NormalizedNodeDataOutput, Collection) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeNode(NormalizedNodeDataOutput, DataTreeCandidateNode) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeChildren(NormalizedNodeDataOutput, Collection) 1023 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeNode(NormalizedNodeDataOutput, DataTreeCandidateNode) 1018 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeChildren(NormalizedNodeDataOutput, Collection) 1018 void org.opendaylight.controller.cluster.datastore.persisted.DataTreeCandidateInputOutput.writeDataTreeCandidate(DataOutput, DataTreeCandidate) 1018 CommitTransactionPayload org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload.create(TransactionIdentifier, DataTreeCandidate) 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree.startCommit(SimpleShardDataTreeCohort, DataTreeCandidate) 1018 void org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.commit(FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.CohortEntry.commit(FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.finishCommit(ActorRef, CohortEntry) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$3.onSuccess(DataTreeCandidate) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$3.onSuccess(<span class="code-object">Object</span>) 1018 void org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulPreCommit(DataTreeCandidateTip) 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree$1.onSuccess(<span class="code-object">Void</span>) 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree$1.onSuccess(<span class="code-object">Object</span>) 1018 void org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.doUserPreCommit(FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.userPreCommit(DataTreeCandidate, FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(SimpleShardDataTreeCohort) 1018 void org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(CohortEntry) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(<span class="code-object">Void</span>) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(<span class="code-object">Object</span>) 1018 void org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit() 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree$CommitEntry) 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree$$Lambda$863.1527042624.accept(<span class="code-object">Object</span>) 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(Queue, ShardDataTreeCohort$State, Consumer) 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction() 1018 void org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(SimpleShardDataTreeCohort) 1018 void org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(FutureCallback) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(CohortEntry) 1018 void org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ReadyLocalTransaction, ActorRef, Shard) 1018 void org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(ReadyLocalTransaction) 1018 void org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(<span class="code-object">Object</span>) 1018 void org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(<span class="code-object">Object</span>) 1018</pre> </div></div> CONTROLLER-1871

Excessive Optional.ofNullable() from QName/QNameModule.getRevision() from NormalizedNodeOutputStreamWriter

Bug Medium Resolved Won't Do Tom Pantelis Michael Vorburger Tue, 13 Nov 2018 11:26:35 +0000 Wed, 14 Nov 2018 13:01:07 +0000 Wed, 14 Nov 2018 13:01:07 +0000 Oxygen SR3 Neon Oxygen SR4 Fluorine SR2 0 3 <p>The Optional comes from QName so this should really be in yangtools. Perhaps QName could provide an @Nullable version of getRevision or store the revision as an Optional - of course that would increase the footprint but QName instances are reused so perhaps that wouldn't be significant (<a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=rovarga" class="user-hover" rel="rovarga">rovarga</a>).</p> <p>That said, the Optionals in this case are short-lived so really shouldn't have a significant affect on GC. </p> <p>We are using Optional for return values, which are immediately handled – making them immediate candidates for on-stack allocation and (if not) extremely easy to collect. They certainly do not end up leaking out of the current thread.</p> <p>Storing an Optional in QName is not really a good idea until we have Valhalla-capable JDK, as it adds 16 bytes of memory overhead for each QName.</p> <p>+1</p> <p>"many tens of GB" ... over what period of time?</p> <p>The scale test where this is from ran for only about 5h. IRL it's much worse.</p> <p>Well, according to <a href="https://shipilev.net/jvm-anatomy-park/4-tlab-allocation/" class="external-link" target="_blank" rel="nofollow noopener">https://shipilev.net/jvm-anatomy-park/4-tlab-allocation/</a> , TLAB can easily perform allocations to the tune of 2.5GB/s – making the overhead 'couple of seconds in a 5 hour run'.</p> <p>This is a reasonable use of API design. We will not go back to nullables.</p> Development Rank 0|i03ke7: