[YANGTOOLS-629] Performance optimization potential in yang loading? Profiling SingleFeatureTest shows Hot Methods in e.g. StmtContextUtils & StatementDefinitionContext Created: 05/Jul/16 Updated: 10/Apr/22 Resolved: 24/Aug/16 |
|
| Status: | Resolved |
| Project: | yangtools |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | ||
| Reporter: | Michael Vorburger | Assignee: | Peter Kajsa |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Operating System: All |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| External issue ID: | 6158 | ||||||||
| Description |
|
I was curious today why that SingleFeatureTest was taking so long (e.g. approx. 15' on netvirt/vpnservice/features currently), and did some profiling using (Oracle, not OpenJDK) Java Mission Control of a local Maven build. I'll attach the zipped *.jfr file if anyone else wants to have a look. If I'm interpreting things correctly, which I may not, then a (to me) surprising fraction of the time spent in SingleFeatureTest, something like 20% (maybe I'm adding up things that are showing twice the same), is spent not in Karaf code loading features and all, nor in actual functional bundle initialization stuff (like netvirt or so) but in yangtools related stuff to load *.yang files. If this is just perfectly normal and to be expected, in the SingleFeatureTest use case, then please feel free to close this as WONTFIX. Just wondering if there isn't any Performance optimization potential in yang loading ... I've played with 1-2 things myself locally, related to more efficient collection use (I'll put in a Gerrit for that). My change does get the related code out of the Hot Methods in JMC, but overall still seems to have very little impact on the ca. 15' SingleFeatureTest of netvirt/vpnservice/features, unfortunately. After that change of mine, the Hot Methods in JMC seems to say that ca. 10% of the total time is in java.util.Collections$UnmodifiableCollection$1.hasNext() related to org.opendaylight.yangtools.yang.parser.spi.meta.StmtContextUtils.findFirstEffectiveSubstatement, and another 10% (which may be the same; I'm not 100% sure how to interpret) related to org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementDefinitionContext getPublicView() and org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase getPublicDefinition(), which doesn't really make any sense to me, as upon a closer look the call chain just seems to be a bunch of getters calling through, so I'm not sure what to make of that. Without fully understanding the internals, I'm wondering it it may be feasible to apply some caching somewhere (related to what jumps out as Hot Methods in JMC) ? |
| Comments |
| Comment by Michael Vorburger [ 05/Jul/16 ] |
|
> played with 1-2 things myself locally, related to more efficient collection |
| Comment by Michael Vorburger [ 05/Jul/16 ] |
|
Attachment SingleFeatureTest-Karaf-JavaFlightRecorder4817713825730748944.jfr.zip has been added with description: Java Mission Control flight recorder performance profiling of SingleFeatureTest |
| Comment by Michael Vorburger [ 05/Jul/16 ] |
|
> did some profiling using (Oracle, not OpenJDK) Java Mission Control |
| Comment by Robert Varga [ 06/Jul/16 ] |
|
Performance improvements are ceratinly welcome. I think this problem comes specifically from if-features implemention and will need to be analyzed a bit. I do not have JMC handy, will take a look at call sites for fFES() – but I doubt there is much we can do to cache that, as it really is filtering on dynamically interpreted context – e.g. cache invalidations are a problem. |
| Comment by Peter Kajsa [ 07/Jul/16 ] |
| Comment by Michael Vorburger [ 07/Jul/16 ] |
|
> https://git.opendaylight.org/gerrit/#/c/41471 I have just verified the impact of this on the SingleFeatureTest build time on netvirt/vpnservice/features, and can confirm that this approx. HALFS the time, 7:30 instead of the ca. 15' from above! This is AMAZING! GREAT. I cannot verify the proposed change itself of course, as my current understanding of yangtools internals is still too limited. I can also not confirm that this has no other side effects, as I haven't actually run any code generated by this with "real" usage - only built netvirt/vpnservice/features (SingleFeatureTest). |
| Comment by Michael Vorburger [ 24/Aug/16 ] |
|
FTR: YANGTOOLS-652 has something new that's a bit similar to this. |