[MDSAL-300] Binding V1 fails when action input is augmented Created: 04/Jan/18  Updated: 29/Jun/18  Resolved: 29/Jun/18

Status: Resolved
Project: mdsal
Component/s: Binding codegen
Affects Version/s: Nitrogen SR1, Carbon SR2
Fix Version/s: Fluorine

Type: Bug Priority: Medium
Reporter: Marek Gradzki Assignee: Robert Varga
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
is blocked by YANGTOOLS-841 SchemaContextUtil.findNodeInSchemaCon... Resolved
Cloners
clones YANGTOOLS-841 SchemaContextUtil.findNodeInSchemaCon... Resolved

 Description   

Yangtools fails to parse following augmentation present in
ietf-mpls@2017-07-02.yang:

augment
"/rt:routing-state/rt:ribs/rt:rib/rt:active-route/rt:input"

{ description "This augment adds the input parameter of the 'active-route' action."; uses nhlfe-contents; }

augment "/rt:routing-state/rt:ribs/rt:rib/rt:active-route/"
+ "rt:output/rt:route"

{ description "This augment adds the destination prefix to the reply of the 'active-route' action."; uses nhlfe-contents; }

where active-route is defined as(ietf-routing@2016-08-18.yang):

action active-route {
description
"Return the active RIB route that is used for the
destination address.

Address family specific modules MUST augment input
parameters with a leaf named 'destination-address'.";
output {

container route {
description
"The active RIB route for the specified destination.

If no route exists in the RIB for the destination
address, no output is returned.

Address family specific modules MUST augment this
container with appropriate route contents.";
container next-hop

{ description "Route's next-hop attribute."; uses next-hop-state-content; }

uses route-metadata;
}
}
}

with following exception:
[ERROR] Failed to parse YANG from source SourceSpecificContext [source=META-INF/yang/ietf-mpls@2017-07-02.yang, current=EFFECTIVE_MODEL, finished=FULL_DECLARATION]: Augment target 'Absolute

{path=[(urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)routing-state, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)ribs, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)rib, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)active-route, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)input]}

' not found [at META-INF/yang/ietf-mpls@2017-07-02.yang:366:2]
[ERROR] yang-to-sources: Unable to parse yang files from /home/m/hc2vpp/mpls/api/src/main/yang
org.opendaylight.yangtools.yang.parser.spi.meta.SomeModifiersUnresolvedException: Some of EFFECTIVE_MODEL modifiers for statements were not resolved.
[...]
Caused by: org.opendaylight.yangtools.yang.parser.spi.meta.InferenceException: Augment target 'Absolute

{path=[(urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)routing-state, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)ribs, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)rib, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)active-route, (urn:ietf:params:xml:ns:yang:ietf-routing?revision=2016-08-18)input]}

' not found [at META-INF/yang/ietf-mpls@2017-07-02.yang:366:2]
at org.opendaylight.yangtools.yang.parser.stmt.rfc6020.AugmentStatementImpl$Definition$1.prerequisiteFailed(AugmentStatementImpl.java:187)
at org.opendaylight.yangtools.yang.parser.stmt.reactor.ModifierImpl.failModifier(ModifierImpl.java:87)
at org.opendaylight.yangtools.yang.parser.stmt.reactor.SourceSpecificContext.failModifiers(SourceSpecificContext.java:341)
... 35 more



 Comments   
Comment by Robert Varga [ 04/Jan/18 ]

The general approach on how to solve this issue in binding V1 needs to be decided. There are multiple options ranging from ignoring the constructs to mapping actions to routed RPCs.

Comment by Robert Varga [ 08/Mar/18 ]

With the following models:

rw-rw-r-. 1 nite nite 16830 Mar  8 15:55 src/main/yang/ietf-inet-types@2013-07-15.yang
rw-rw-r-. 1 nite nite 24219 Mar  8 15:55 src/main/yang/ietf-interfaces@2014-05-08.yang
rw-rw-r-. 1 nite nite  9817 Mar  8 15:53 src/main/yang/ietf-mpls@2017-07-02.yang
rw-rw-r-. 1 nite nite 16836 Mar  8 15:54 src/main/yang/ietf-routing@2016-11-04.yang
rw-rw-r-. 1 nite nite 24480 Mar  8 15:54 src/main/yang/ietf-routing-types@2017-12-04.yang
rw-rw-r-. 1 nite nite 17933 Mar  8 15:55 src/main/yang/ietf-yang-types@2013-07-15.yang

 

I get the following codegen issue:

[ERROR] yang-to-sources: Unable to generate sources with org.opendaylight.mdsal.binding.maven.api.gen.plugin.CodeGeneratorImpl generator
java.lang.NullPointerException: Target type not yet generated: RPC input input
        at org.opendaylight.mdsal.binding.generator.impl.BindingGeneratorImpl.augmentationToGenTypes(BindingGeneratorImpl.java:872)
        at org.opendaylight.mdsal.binding.generator.impl.BindingGeneratorImpl.allAugmentsToGenTypes(BindingGeneratorImpl.java:416)
        at org.opendaylight.mdsal.binding.generator.impl.BindingGeneratorImpl.generateTypes(BindingGeneratorImpl.java:249)
        at org.opendaylight.mdsal.binding.maven.api.gen.plugin.CodeGeneratorImpl.generateSources(CodeGeneratorImpl.java:61)
        at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesProcessor.generateSourcesWithOneGenerator(YangToSourcesProcessor.java:379)
        at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesProcessor.generateSources(YangToSourcesProcessor.java:329)
        at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesProcessor.conditionalExecute(YangToSourcesProcessor.java:156)
        at org.opendaylight.yangtools.yang2sources.plugin.YangToSourcesMojo.execute(YangToSourcesMojo.java:123)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

with current mdsal + yangtools-2.0.2.

Comment by Robert Varga [ 08/Mar/18 ]

The root cause is that an augmentation class is being generated for an action input – and actions are not mapped not anything, hence the target type is not found.

Comment by Robert Varga [ 12/Mar/18 ]

Generating actions/notifications is not really a problem per se, but unfortunately they end up living in the same namespace as regular data nodes and groupings – which means we could end up regressing against current models (which compile just fine because of these elements being ignored).

An obvious solution would be similar to MDSAL-309, e.g. first determine all elements which need to be assigned into (hierarchical because of nested subclasses) namespace structure and then run legacy mapping, followed by bijective mapping (which needs to be augmented to take care of nested subclasses).

Note that mapping of identifiers should be slightly easier, as we are dealing with input identifiers, which are defined in https://tools.ietf.org/html/rfc7950#section-6.2 as:

 

   Identifiers are used to identify different kinds of YANG items by
   name.  Each identifier starts with an uppercase or lowercase ASCII
   letter or an underscore character, followed by zero or more ASCII
   letters, digits, underscore characters, hyphens, and dots.
   Implementations MUST support identifiers up to 64 characters in
   length and MAY support longer identifiers.  Identifiers are case
   sensitive.

And in section 14:

 

   identifier          = (ALPHA / "_")
                         *(ALPHA / DIGIT / "_" / "-" / ".")
   ALPHA               = %x41-5A / %x61-7A
                         ; A-Z / a-z
   DIGIT               = %x30-39
                         ; 0-9

 

Comment by Robert Varga [ 13/Mar/18 ]

https://git.opendaylight.org/gerrit/69414 adds a very simplistic encoding, which has just two escapes: "$$" for '.' and "$_" for '-', which I think is a reasonable way of mapping mapping these:

 

YANG Binding $ Binding _$
foo foo foo
true $true $true
_ $_ $_
__ $__ $__
bar-baz bar$_baz bar_baz
bar--baz bar$_$_baz bar__baz
bar.baz bar$$baz bar$baz
bar..baz bar$$$$baz bar$$baz
bar_baz bar_baz $bar_baz
bar__baz bar__baz $bar__baz
bar_.baz bar_$$baz $bar_$$baz
bar-.baz bar_$_$$baz bar_$baz
bar_-baz bar_$_baz $bar_$_baz

The code currently does 'Binding $'.

If we treat '' as a special character (and hence escape it), the much more appealing form 'Binding _$' can be used. In that scheme '$' is used as the initial escape meaning '.' – and since '.' cannot be at the start of YANG identifier, it is a padding escape on the first position indicating 'keep underscores as they are, dashes are escaped as '$' . Since underscore is less prevalent than dash in most YANG files, this escape should rarely be encountered with standard models. Vendor models use '_' in conjunction with all-capitals, which is probably a C language leak. Vendors should fix this by mapping underscore back to dash.

Comment by Robert Varga [ 13/Mar/18 ]

Alternatively we could try to find a valid Java identitifer part from (at least all of unicode is allowed). The trouble becomes typing it out.

Comment by Robert Varga [ 13/Mar/18 ]

At any rate, this does not help us that much, as there are conflicts possible with the old mapping, and hence we really will need a mapping name assignement phase (e.g. collect all candidates at the specified package/containing class name and choose an unambiguous mapping.

Comment by Robert Varga [ 19/Mar/18 ]

Perhaps we can get away with just adding a combined Service, just as we are doing with RPCs, to the package where the actions are define – e.g. for

container foo {
    action foo-bar;
    action bar-baz;
}

we would end up generating

package <prefix>.foo;

public interface FooActionService extends ActionService {
    ListenableFuture<RpcResult<FooBarOutput> fooBar(InstanceIdentifier<Foo> context, FooBarInput input);
    ListenableFuture<RpcResult<BarBazOutput> barBaz(InstanceIdentifier<Foo> context, BarBazInput input);
}
Comment by Robert Varga [ 25/Jun/18 ]

Actually we do not want to aggregate the service, as that is just creating headaches around identifier mapping. Since RFC7950 specifies we cannot overlap with containers and similar:

   o  All leafs, leaf-lists, lists, containers, choices, rpcs, actions,
      notifications, anydatas, and anyxmls defined (directly or through
      a "uses" statement) within a parent node or at the top level of
      the module or its submodules share the same identifier namespace.
      This namespace is scoped to the parent node or module, unless the
      parent node is a case node.  In that case, the namespace is scoped
      to the closest ancestor node that is not a case or choice node.

 

Generated at Wed Feb 07 20:09:27 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.