[YANGTOOLS-1480] Fail to process deviation/augmentation from multiple modules Created: 31/Jan/23 Updated: 15/Mar/23 Resolved: 15/Mar/23 |
|
| Status: | Resolved |
| Project: | yangtools |
| Component/s: | parser |
| Affects Version/s: | 8.0.9, 7.0.18, 9.0.6, 10.0.3 |
| Fix Version/s: | 11.0.0, 8.0.10, 9.0.7, 10.0.5 |
| Type: | Bug | Priority: | High |
| Reporter: | Sangwook Ha | Assignee: | Robert Varga |
| Resolution: | Done | Votes: | 0 |
| Labels: | pt | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Description |
|
Parsing of data models with deviations/augmentations from multiple modules targeting a path defined using a grouping may fail. The following is such an example. foo.yang module foo {
namespace "urn:foo";
prefix "foo";
grouping foo-group {
container foo {
container cntr {
}
}
}
uses foo-group;
}
bar.yang module bar {
namespace "urn:bar";
prefix "bar";
import foo {
prefix foo;
}
deviation /foo:foo/foo:cntr {
deviate not-supported;
}
}
baz.yang module baz {
namespace "urn:baz";
prefix "baz";
import foo {
prefix foo;
}
augment /foo:foo {
container cntr {
leaf baz-leaf {
type string;
}
}
}
}
qux.yang module qux {
namespace "urn:qux";
prefix "qux";
import foo {
prefix foo;
}
import baz {
prefix baz;
}
deviation /foo:foo/baz:cntr/baz:baz-leaf {
deviate add {
default "baz";
}
}
}
Apparently there are a couple of code paths that end up with an error. In the first case parsing fails with a name collision for foo:cntr: Caused by: org.opendaylight.yangtools.yang.parser.spi.source.SourceException: Error in module 'foo': cannot add '(urn:foo)cntr'. Node name collision: '(urn:foo)cntr' already declared at /volumes/OpenDaylight/yangtools/parser/yang-parser-rfc7950/target/test-classes/bugs/YT1480/foo.yang:7:7 [at /volumes/OpenDaylight/yangtools/parser/yang-parser-rfc7950/target/test-classes/bugs/YT1480/foo.yang:7:7] at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.SchemaTreeNamespaceBehaviour.addTo(SchemaTreeNamespaceBehaviour.java:64) at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.SchemaTreeNamespaceBehaviour.addTo(SchemaTreeNamespaceBehaviour.java:27) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.SimpleNamespaceContext.addTo(SimpleNamespaceContext.java:44) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.NamespaceStorageSupport.addToNamespace(NamespaceStorageSupport.java:83) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.addToNs(StatementContextBase.java:268) at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.meta.AbstractSchemaTreeStatementSupport.onStatementAdded(AbstractSchemaTreeStatementSupport.java:115) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementDefinitionContext.onStatementAdded(StatementDefinitionContext.java:66) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.childCopyOf(StatementContextBase.java:812) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.childCopyOf(StatementContextBase.java:782) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.copyAsChildOfImpl(StatementContextBase.java:747) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.copyAsChildOf(StatementContextBase.java:729) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.InferredStatementContext.copySubstatement(InferredStatementContext.java:623) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.InferredStatementContext.requestSchemaTreeChild(InferredStatementContext.java:458) at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.SchemaTreeNamespaceBehaviour.requestFrom(SchemaTreeNamespaceBehaviour.java:70) at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.SchemaTreeNamespaceBehaviour.getFrom(SchemaTreeNamespaceBehaviour.java:47) at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.SchemaTreeNamespaceBehaviour.getFrom(SchemaTreeNamespaceBehaviour.java:27) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.NamespaceBehaviourWithListeners.getFrom(NamespaceBehaviourWithListeners.java:109) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ReactorStmtCtx.namespaceItem(ReactorStmtCtx.java:318) at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.meta.NamespaceStmtCtx.getFromNamespace(NamespaceStmtCtx.java:85) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.onNamespaceItemAddedAction(StatementContextBase.java:598) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ModifierImpl$AbstractPathPrerequisite.hookOnto(ModifierImpl.java:369) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ModifierImpl$AbstractPathPrerequisite.namespaceItemAdded(ModifierImpl.java:350) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase$1.onValueAdded(StatementContextBase.java:608) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.NamespaceBehaviourWithListeners.notifyListeners(NamespaceBehaviourWithListeners.java:88) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.SimpleNamespaceContext.addTo(SimpleNamespaceContext.java:47) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.NamespaceStorageSupport.addToNamespace(NamespaceStorageSupport.java:83) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.addToNs(StatementContextBase.java:268) at org.opendaylight.yangtools.yang.parser.spi@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.spi.meta.AbstractSchemaTreeStatementSupport.onStatementAdded(AbstractSchemaTreeStatementSupport.java:115) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementDefinitionContext.onStatementAdded(StatementDefinitionContext.java:66) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.childCopyOf(StatementContextBase.java:812) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.childCopyOf(StatementContextBase.java:782) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.copyAsChildOfImpl(StatementContextBase.java:747) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.copyAsChildOf(StatementContextBase.java:729) at org.opendaylight.yangtools.yang.parser.rfc7950@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.rfc7950.stmt.uses.UsesStatementSupport.copyFromSourceToTarget(UsesStatementSupport.java:199) at org.opendaylight.yangtools.yang.parser.rfc7950@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.rfc7950.stmt.uses.UsesStatementSupport$1.apply(UsesStatementSupport.java:106) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ModifierImpl.tryApply(ModifierImpl.java:146) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ModifierImpl$AddedToNamespace.phaseFinished(ModifierImpl.java:420) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.runPhaseListeners(StatementContextBase.java:578) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.onPhaseCompleted(StatementContextBase.java:523) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.doTryToCompletePhase(StatementContextBase.java:457) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ReactorStmtCtx.tryToCompletePhase(ReactorStmtCtx.java:449) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.completeChildren(StatementContextBase.java:466) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.StatementContextBase.doTryToCompletePhase(StatementContextBase.java:456) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ReactorStmtCtx.tryToCompletePhase(ReactorStmtCtx.java:449) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.SourceSpecificContext.tryToCompletePhase(SourceSpecificContext.java:299) at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.BuildGlobalContext.completePhaseActions(BuildGlobalContext.java:364) ... 116 more In the second case the parser complains that the target path (/foo:foo/baz:cntr/baz:baz-leaf) is not found: Caused by: org.opendaylight.yangtools.yang.parser.spi.meta.InferenceException: Deviation target 'Absolute{qnames=[(urn:foo)foo, (urn:baz)cntr, baz-leaf]}' not found. [at /volumes/OpenDaylight/yangtools/parser/yang-parser-rfc7950/target/test-classes/bugs/YT1480/qux.yang:13:3]
at org.opendaylight.yangtools.yang.parser.rfc7950@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.rfc7950.stmt.deviate.AbstractDeviateStatementSupport$1.prerequisiteFailed(AbstractDeviateStatementSupport.java:167)
at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.ModifierImpl.failModifier(ModifierImpl.java:86)
at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.SourceSpecificContext.failModifiers(SourceSpecificContext.java:375)
at org.opendaylight.yangtools.yang.parser.reactor@10.0.3-SNAPSHOT/org.opendaylight.yangtools.yang.parser.stmt.reactor.BuildGlobalContext.addSourceExceptions(BuildGlobalContext.java:307)
... 117 more
|
| Comments |
| Comment by Robert Varga [ 14/Feb/23 ] |
|
The second case is a dead ringer for |
| Comment by Robert Varga [ 13/Mar/23 ] |
|
Okay this is a case of us re-entering during uses statement inlining:
Thread [main] (Suspended (breakpoint at line 58 in SchemaTreeNamespaceBehaviour))
SchemaTreeNamespaceBehaviour<D,E>.addTo(NamespaceStorageNode, QName, StmtContext<QName,D,E>) line: 58
SchemaTreeNamespaceBehaviour<D,E>.addTo(NamespaceBehaviour$NamespaceStorageNode, Object, Object) line: 27
[3] SimpleNamespaceContext<K,V>.addTo(NamespaceStorageNode, K, V) line: 44
InferredStatementContext<A,D,E>(NamespaceStorageSupport).addToNamespace(ParserNamespace<K,V>, T, U) line: 83
InferredStatementContext<A,D,E>(StatementContextBase<A,D,E>).addToNs(ParserNamespace<K,V>, T, U) line: 268
ContainerStatementSupport(AbstractSchemaTreeStatementSupport<D,E>).onStatementAdded(Mutable<QName,D,E>) line: 115
StatementDefinitionContext<A,D,E>.onStatementAdded(Mutable<A,D,E>) line: 66
InferredStatementContext<A,D,E>(StatementContextBase<A,D,E>).childCopyOf(StatementContextBase<X,Y,Z>, CopyType, QNameModule) line: 812
InferredStatementContext<A,D,E>(StatementContextBase<A,D,E>).childCopyOf(StmtContext<?,?,?>, CopyType, QNameModule) line: 782
SubstatementContext<A,D,E>(StatementContextBase<A,D,E>).copyAsChildOfImpl(Mutable<?,?,?>, CopyType, QNameModule) line: 747
SubstatementContext<A,D,E>(StatementContextBase<A,D,E>).copyAsChildOf(Mutable<?,?,?>, CopyType, QNameModule) line: 729
InferredStatementContext<A,D,E>.copySubstatement(StmtContext<X,Y,Z>) line: 623
[2] InferredStatementContext<A,D,E>.requestSchemaTreeChild(QName) line: 458
SchemaTreeNamespaceBehaviour<D,E>.requestFrom(NamespaceStorageNode, QName) line: 70
SchemaTreeNamespaceBehaviour<D,E>.getFrom(NamespaceStorageNode, QName) line: 47
SchemaTreeNamespaceBehaviour<D,E>.getFrom(NamespaceBehaviour$NamespaceStorageNode, Object) line: 27
SimpleNamespaceContext<K,V>(NamespaceBehaviourWithListeners<K,V>).getFrom(NamespaceStorageNode, K) line: 109
InferredStatementContext<A,D,E>(ReactorStmtCtx<A,D,E>).namespaceItem(ParserNamespace<K,V>, T) line: 318
InferredStatementContext<A,D,E>(NamespaceStmtCtx).getFromNamespace(ParserNamespace<K,V>, T) line: 85
InferredStatementContext<A,D,E>(StatementContextBase<A,D,E>).onNamespaceItemAddedAction(ParserNamespace<K,V>, K, OnNamespaceItemAdded) line: 598
ModifierImpl$PhaseModificationInNamespacePath<C,K>(ModifierImpl$AbstractPathPrerequisite<C,K>).hookOnto(StatementContextBase<?,?,?>, ParserNamespace<?,?>, K) line: 377
ModifierImpl$PhaseModificationInNamespacePath<C,K>(ModifierImpl$AbstractPathPrerequisite<C,K>).namespaceItemAdded(StatementContextBase<?,?,?>, ParserNamespace<?,?>, Object, Object) line: 358
StatementContextBase$1.onValueAdded(Object) line: 608
SimpleNamespaceContext<K,V>(NamespaceBehaviourWithListeners<K,V>).notifyListeners(NamespaceStorageNode, Iterator<KeyedValueAddedListener<K>>, V) line: 88
[1] SimpleNamespaceContext<K,V>.addTo(NamespaceStorageNode, K, V) line: 47
RootStatementContext<A,D,E>(NamespaceStorageSupport).addToNamespace(ParserNamespace<K,V>, T, U) line: 83
RootStatementContext<A,D,E>(StatementContextBase<A,D,E>).addToNs(ParserNamespace<K,V>, T, U) line: 268
ContainerStatementSupport(AbstractSchemaTreeStatementSupport<D,E>).onStatementAdded(Mutable<QName,D,E>) line: 115
StatementDefinitionContext<A,D,E>.onStatementAdded(Mutable<A,D,E>) line: 66
[0] RootStatementContext<A,D,E>(StatementContextBase<A,D,E>).childCopyOf(StatementContextBase<X,Y,Z>, CopyType, QNameModule) line: 812
RootStatementContext<A,D,E>(StatementContextBase<A,D,E>).childCopyOf(StmtContext<?,?,?>, CopyType, QNameModule) line: 782
SubstatementContext<A,D,E>(StatementContextBase<A,D,E>).copyAsChildOfImpl(Mutable<?,?,?>, CopyType, QNameModule) line: 747
SubstatementContext<A,D,E>(StatementContextBase<A,D,E>).copyAsChildOf(Mutable<?,?,?>, CopyType, QNameModule) line: 729
UsesStatementSupport.copyFromSourceToTarget(StmtContext<?,?,?>, Mutable<?,?,?>, Mutable<QName,UsesStatement,UsesEffectiveStatement>) line: 199
UsesStatementSupport$1.apply(ModelActionBuilder$InferenceContext) line: 106
ModifierImpl.tryApply() line: 146
[...]
So we are:
But at some point we end up re-entering to requestSchemaTreeChild() even though we have the namespace populated and the substatement materialized: Thread [main] (Suspended (breakpoint at line 427 in InferredStatementContext)) InferredStatementContext<A,D,E>.requestSchemaTreeChild(QName) line: 427 SchemaTreeNamespaceBehaviour<D,E>.requestFrom(NamespaceStorageNode, QName) line: 70 SchemaTreeNamespaceBehaviour<D,E>.getFrom(NamespaceStorageNode, QName) line: 47 SchemaTreeNamespaceBehaviour<D,E>.getFrom(NamespaceBehaviour$NamespaceStorageNode, Object) line: 27 SimpleNamespaceContext<K,V>(NamespaceBehaviourWithListeners<K,V>).getFrom(NamespaceStorageNode, K) line: 109 InferredStatementContext<A,D,E>(ReactorStmtCtx<A,D,E>).namespaceItem(ParserNamespace<K,V>, T) line: 318 InferredStatementContext<A,D,E>(NamespaceStmtCtx).getFromNamespace(ParserNamespace<K,V>, T) line: 85 InferredStatementContext<A,D,E>(StatementContextBase<A,D,E>).onNamespaceItemAddedAction(ParserNamespace<K,V>, K, OnNamespaceItemAdded) line: 598 ModifierImpl$PhaseModificationInNamespacePath<C,K>(ModifierImpl$AbstractPathPrerequisite<C,K>).hookOnto(StatementContextBase<?,?,?>, ParserNamespace<?,?>, K) line: 377 ModifierImpl$PhaseModificationInNamespacePath<C,K>(ModifierImpl$AbstractPathPrerequisite<C,K>).namespaceItemAdded(StatementContextBase<?,?,?>, ParserNamespace<?,?>, Object, Object) line: 358 StatementContextBase$1.onValueAdded(Object) line: 608 SimpleNamespaceContext<K,V>(NamespaceBehaviourWithListeners<K,V>).notifyListeners(NamespaceStorageNode, Iterator<KeyedValueAddedListener<K>>, V) line: 88 SimpleNamespaceContext<K,V>.addTo(NamespaceStorageNode, K, V) line: 47 RootStatementContext<A,D,E>(NamespaceStorageSupport).addToNamespace(ParserNamespace<K,V>, T, U) line: 83 RootStatementContext<A,D,E>(StatementContextBase<A,D,E>).addToNs(ParserNamespace<K,V>, T, U) line: 268 ContainerStatementSupport(AbstractSchemaTreeStatementSupport<D,E>).onStatementAdded(Mutable<QName,D,E>) line: 115 StatementDefinitionContext<A,D,E>.onStatementAdded(Mutable<A,D,E>) line: 66 RootStatementContext<A,D,E>(StatementContextBase<A,D,E>).childCopyOf(StatementContextBase<X,Y,Z>, CopyType, QNameModule) line: 812 RootStatementContext<A,D,E>(StatementContextBase<A,D,E>).childCopyOf(StmtContext<?,?,?>, CopyType, QNameModule) line: 782 SubstatementContext<A,D,E>(StatementContextBase<A,D,E>).copyAsChildOfImpl(Mutable<?,?,?>, CopyType, QNameModule) line: 747 SubstatementContext<A,D,E>(StatementContextBase<A,D,E>).copyAsChildOf(Mutable<?,?,?>, CopyType, QNameModule) line: 729 UsesStatementSupport.copyFromSourceToTarget(StmtContext<?,?,?>, Mutable<?,?,?>, Mutable<QName,UsesStatement,UsesEffectiveStatement>) line: 199 UsesStatementSupport$1.apply(ModelActionBuilder$InferenceContext) line: 106 ModifierImpl.tryApply() line: 146 [...]
|
| Comment by Robert Varga [ 15/Mar/23 ] |
|
Alright, so I see what is going on: the problem is that we have both (foo)cntr and (baz)cntr meeting in InferredStatement.requestSchemaTree() – which looks uses (foo)cntr to materialize the statement and – we end up creating the same statement again (because it's name will be instantiated again, leading to the first reported error. |