Details
-
Improvement
-
Status: Resolved
-
Medium
-
Resolution: Done
-
None
-
None
Description
After addressing CONTROLLER-1897, overall performance improved by about 18%, but the subsequent profiling is showing readQName() still accounting for 46% of CPU time spent.
There are two components to this cost:
- 30% is spent in readCodedString()
- 61% is spent in QNameFactory.create()
The sample (a 350MiB snapshot) invokes readQName() 1.7M times, with the overall result being 506 unique QNames – hence QNames are obviously a good candidate for the same coding we are using for Strings.
Implementing such coding will allow us:
- trim snapshot, as already-encoded QNames will result in one-third of reads, i.e. 1 read of 5 bytes instead of 3
- eliminate most of the QNameFactory.create() overhead, as repetitive QNames will be looked up in a local List instead of hash-based concurrent LoadingCache
Finally, this also enables us to cache NodeIdentifier instances, as NodeIdentifier is only a wrapper around a QName. Adding a secondary lookup table for caching these wrappers should allow us to lower the memory footprint of the deserialized data.
Attachments
Issue Links
- is blocked by
-
CONTROLLER-1897 Improve QNameFactory performance
-
- Resolved
-
- relates to
-
CONTROLLER-1888 Update journal/snapshot/datastore version
-
- Resolved
-
| # | Subject | Branch | Project | Status | CR | V |
|---|---|---|---|---|---|---|
| 82277,16 | Expose NormalizedNodeStream versioning | master | controller | Status: ABANDONED | 0 | 0 |
| 82278,6 | Separate lazy-versioned NormalizedNodeDataInput | master | controller | Status: MERGED | +2 | +1 |
| 82279,13 | Optimize NodeIdentifier reading | master | controller | Status: MERGED | +2 | +1 |
| 82304,7 | Add support for coded QNames/AugmentationIdentifiers | master | controller | Status: MERGED | +2 | +1 |
| 82338,3 | WIP: add a BigInteger/BigDecimal cache | master | controller | Status: ABANDONED | 0 | +1 |
| 82382,1 | Separate lazy-versioned NormalizedNodeDataInput | stable/neon | controller | Status: MERGED | -1 | +1 |
| 82383,1 | Add support for coded QNames/AugmentationIdentifiers | stable/neon | controller | Status: MERGED | -1 | +1 |
| 82398,2 | Optimize readQNameSet() | master | controller | Status: MERGED | +2 | +1 |
| 82399,2 | Take advantage of AugmentationIdentifier.create() | master | controller | Status: MERGED | +2 | +1 |
| 82409,1 | Optimize readQNameSet() | stable/neon | controller | Status: MERGED | +2 | +1 |
| 82426,5 | Add QNameModule coding | master | controller | Status: MERGED | +2 | +1 |
| 82461,1 | Optimize NodeIdentifier reading | stable/neon | controller | Status: MERGED | +2 | +1 |
| 82462,1 | Add QNameModule coding | stable/neon | controller | Status: MERGED | +2 | +1 |
| 82865,2 | Take advantage of AugmentationIdentifier.create() | stable/neon | controller | Status: MERGED | +2 | +1 |