https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/YANGTOOLS-660 yangtools <p>When I try to create SchemaContext including 500 complicated models (8 MB of yang sources), SchemaContext instance occupies 1.6 GB of memory. Profiling has shown, that one of the causes could be large number of duplicated objects - probably EnumMaps in class SubstatementContext.<br/> Would it be possible to somehow optimize it?</p> <p>Operating System: All<br/> Platform: All</p> YANGTOOLS-660

SchemaContext - excessive memory consumption

Bug Highest Resolved Done Robert Varga Andrej Mak Tue, 20 Sep 2016 13:03:53 +0000 Sun, 10 Apr 2022 18:35:51 +0000 Wed, 19 Oct 2016 17:07:44 +0000 0 5 <p>Can you share a heap dump and the list of models?</p> <p>Analysis of the provided memory dump shows quite clearly that we are retaining implementation-internal details and state in the end result.</p> <p>We retain BuildSchemaContext, SourceSpecificContext, SubstatementContext and similar, which should certainly not be retained once EffectiveSchemaContext has been built.</p> <p>It would seem that the problem is EffectiveStatementBase.unknownSubstatementsToBuild, introduced in <a href="https://git.opendaylight.org/gerrit/#/c/28300/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/28300/</a>.</p> <p>The approach taken to fix the recursion issue needs to be revised so that that field does not exist at all.</p> <p>master: <a href="https://git.opendaylight.org/gerrit/#/c/47038/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/47038/</a></p> <p>boron: <a href="https://git.opendaylight.org/gerrit/47041" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/47041</a></p> <p>Reopening bug since it needs to be fixed in Beryllium SR4.</p> <p>To better assess the impact of this bug and fix, could someone from your team please help us identify the following:<br/> Severity: Could you elaborate on the severity of this bug? Is this a BLOCKER such that we cannot release Beryllium without it? Is there a workaround such that we can write a release note?<br/> Testing: Could you also elaborate on the testing of this patch? How extensively has this patch been tested? Is it covered by any unit tests or system tests? <br/> Impact: Does this fix impact any dependent projects?</p> <p>Severity:<br/> This issue was introduced during fixing of BUG-4456 and while it fixed that issue, it inadvertently caused temporary objects to be retained indefinitely instead of being garbage collected, causing excessive memory overhead.</p> <p>The problem is triggered by presence of an 'extension' statement containing another extension. The only workaround is to not to load models containing such constructs, which is not practical.</p> <p>The issue could be exploited by a malicious SB device to cause a resource exhaustion on the controller. Given the SR4 is our last planned maintenance release, we should not be releasing it without this issue being fixed.</p> <p>Testing:<br/> BUG-4456 is covered by a unit test. The patch attached to this issue reverts the original fix, temporarily introducing a functionality regression (while fixing the memory leak). The patch attached to BUG-4456 reworks the original fix in a way, which does not leak memory. This is confirmed by the unit test passing again as expected.</p> <p>The combination of the two patches has been manually tested against the device which flushed the original bug report and confirmed to work (no regression on BUG-4456) and to have eliminated the memory leak (reported memory usage was ~240MB).</p> <p>We currently do not have system tests which would identify this sort of issues, but we are in process of introducing such a test suite in Carbon.</p> <p>Impact:<br/> No ODL projects use models which trigger this behavior as far as we can tell, hence they should not be impacted by this issue.</p> <p>beryllium: <a href="https://git.opendaylight.org/gerrit/47091" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/47091</a></p> <p>Has this bug been verified as fixed in the latest Beryllium SR4 Build 20161019?</p> Blocks NETCONF-256 YANGTOOLS-692 YANGTOOLS-693 Development Epic Link YANGTOOLS-652 External issue ID 6757 External issue URL Issue Type ODL SR Target Milestone Rank 0|i028lj: