[YANGTOOLS-992] Submodule importing extension from different file failed Created: 17/May/19  Updated: 31/Jan/20  Resolved: 31/Jan/20

Status: Resolved
Project: yangtools
Component/s: None
Affects Version/s: 2.0.16
Fix Version/s: 5.0.0, 3.0.10, 4.0.7

Type: Bug Priority: Medium
Reporter: Anna Bencúrová Assignee: Robert Varga
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File module1@2019-05-17.yang     File module1submodule1@2019-05-17.yang     File module2@2019-05-17.yang    

 Description   

If the yang submodule is importing external module and using its extension, the build is failing with exception (see bellow). If the import of external module is added to the parent module of the submodule, it works.

[ERROR] yang-to-sources: Unable to generate sources with org.opendaylight.mdsal.binding.maven.api.gen.plugin.CodeGeneratorImpl generator
java.lang.IllegalArgumentException: Failed to find prefix for statement (urn:example:module2?revision=2019-05-17)ext1
    at com.google.common.base.Preconditions.checkArgument (Preconditions.java:210)
    at org.opendaylight.yangtools.yang.model.export.ExportUtils.statementPrefix (ExportUtils.java:62)
    at org.opendaylight.yangtools.yang.model.export.YangTextSnippetIterator.pushStatement (YangTextSnippetIterator.java:199)
    at org.opendaylight.yangtools.yang.model.export.YangTextSnippetIterator.computeNext (YangTextSnippetIterator.java:160)
    at org.opendaylight.yangtools.yang.model.export.YangTextSnippetIterator.computeNext (YangTextSnippetIterator.java:42)
    at com.google.common.collect.AbstractIterator.tryToComputeNext (AbstractIterator.java:140)
    at com.google.common.collect.AbstractIterator.hasNext (AbstractIterator.java:135)
    at org.opendaylight.mdsal.binding.java.api.generator.BaseTemplate.appendYangSnippet (BaseTemplate.java:424)
    at org.opendaylight.mdsal.binding.java.api.generator.BaseTemplate.appendSnippet (BaseTemplate.java:391)
    at org.opendaylight.mdsal.binding.java.api.generator.BaseTemplate.formatDataForJavaDoc (BaseTemplate.java:342)
    at org.opendaylight.mdsal.binding.java.api.generator.InterfaceTemplate.body (InterfaceTemplate.java:73)
    at org.opendaylight.mdsal.binding.java.api.generator.BaseTemplate.generate (BaseTemplate.java:88)
    at org.opendaylight.mdsal.binding.java.api.generator.InterfaceGenerator.generate (InterfaceGenerator.java:34)
    at org.opendaylight.mdsal.binding.java.api.generator.GeneratorJavaFile$GeneratorStringSupplier.get (GeneratorJavaFile.java:54)
    at org.opendaylight.mdsal.binding.java.api.generator.GeneratorJavaFile$GeneratorStringSupplier.get (GeneratorJavaFile.java:43)
    at org.opendaylight.mdsal.binding.maven.api.gen.plugin.CodeGeneratorImpl.generateSources (CodeGeneratorImpl.java:113)
    at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesProcessor.generateSourcesWithOneGenerator (YangToSourcesProcessor.java:380)
    at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesProcessor.generateSources (YangToSourcesProcessor.java:330)
    at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesProcessor.conditionalExecute (YangToSourcesProcessor.java:157)
    at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesMojo.execute (YangToSourcesMojo.java:124)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR] yang-to-sources: One or more code generators failed, including failed list(generatorClass=exception) {org.opendaylight.mdsal.binding.maven.api.gen.plugin.CodeGeneratorImpl=java.lang.IllegalArgumentException}


 Comments   
Comment by Robert Varga [ 23/May/19 ]

The problem is the grouping in here and the fact it is inlined in the base module (see AbstractEffectiveModule) – MD-SAL codegen knows nothing about submodules, nor should it. The prefix resolver relies on the effective import map known to the module and it does not traverse additional imports (as inlined). So we have an effective statement, whose declared counterpart lives in a different module and there is not enough information retained, as the import statements are not inlined.

This only affects root module statements, hence it is possible to get around this by special-casing the case when we have submodules and match them up on identity. Ugly, but it should work.

Comment by Robert Varga [ 23/May/19 ]

DeclaredStatementFormatter is constructing the prefix map, and I think it should be able to discern when a statement is coming from submodule – but that is probably a costly operation.

Comment by Robert Varga [ 23/May/19 ]

The problem is also that the cross-relationship passed from MD-SAL is not limited to top-level nodes – it can be any nested node. I think the fix here should be:

  • index submodule imports (either in export or in actual EffectiveSubmoduleStatement)
  • fall back to checking per-module imports for a prefix match if we fail to find a prefix – this needs then be re-assured by finding the statement
  • that is really saying that there is a bloom filter for prefixes – where conflicts are not likely, but can happen
  • costly operation is validation, which involves a linear search of all of submodule's substatements (comparing on identity), so that needs to be avoided

The resulting lookup class is probably a good candidate for yang-model-util, but it cannot really be tested there, so keeping it in yang-model-export is good enough.

 

Comment by Robert Varga [ 23/May/19 ]

Another option is to include prefix mapping at each effective statement – but that is not really helpful, as we are working on declared statements.

Generated at Wed Feb 07 20:54:53 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.