[YANGTOOLS-592] Beryllium yangtools gives error parsing files with Tail-F deref statement Created: 14/Mar/16  Updated: 10/Apr/22  Resolved: 20/Jul/16

Status: Resolved
Project: yangtools
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Giles Heron Assignee: Peter Kajsa
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: Zip Archive karaf.zip    
External issue ID: 5518

 Description   

Beryllium (0.4.1-SNAPSHOT) is unable to mount Tail-F NCS 3.4. Lithium-SR3 works fine.

The error I'm seeing appears to be related to ODL being unable to parse the Tail-f "deref" statement:

2016-03-14 13:50:07,992 | WARN | ssing-executor-1 | Utils | 53 - org.opendaylight.yangtools.yang-parser-impl - 0.8.1.SNAPSHOT | Argument " . = 'false' or
not (/cluster/remote-node
[not (username)]
[deref(authgroup)/../default-map[same-user or same-pass]])" is not valid XPath string at "tailf-ncs-cluster:320:8"
javax.xml.xpath.XPathExpressionException: javax.xml.transform.TransformerException: Could not find function: deref
...

2016-03-14 13:50:08,564 | ERROR | ssing-executor-1 | NetconfDevice | 158 - org.opendaylight.netconf.sal-netconf-connector - 1.3.1.SNAPSHOT | RemoteDevice

{ncs}

: Initialization in sal failed, disconnecting from device
java.lang.NullPointerException



 Comments   
Comment by Giles Heron [ 14/Mar/16 ]

Attachment karaf.zip has been added with description: zipped logs

Comment by Peter Kajsa [ 15/Mar/16 ]

Node-set deref(node-set nodes) function is not valid in Yang 1.0.
The function will be introduced in YANG 1.1 which is planned to Carbon release.

Comment by Giles Heron [ 15/Mar/16 ]

so why does this work ok in Lithium?

I'm ok with ODL not supporting it, but I'm not OK with ODL terminating the mount if it hits an issue with one model.

Comment by Peter Kajsa [ 15/Mar/16 ]

Yangtools logs the warning due to invalid xpath expression and whole execution is going on as usually and the RevisionAwareXpath is created normally.

Could you send me please the yang model which causes the issue, I will try to find out what we can do.

Comment by Peter Kajsa [ 17/Mar/16 ]

I tested obtained yang models and I believe the issue is not caused by invalid xpath expression and by the following warning log.

Therefore, in the next step I tested whole set of Tail-F models (but I am not sure I have the most recent version of all Tail-F yang models) and the yang parser reported many errors like refine or augment on unknown statements, etc.
The Augments when target node is an unknown statement were skipped only with warning LOG in Lithium, but in Beryllium they are reported as errors and I believe this is the root cause.

I have to discuss this issue, because the decision to report such augments as errors in Beryllium has been made intentionally.

Comment by Giles Heron [ 17/Mar/16 ]

ah, ok. and this is back to the fact that errors result in exceptions and the way we handle exceptions is to drop the netconf connection?

Comment by Peter Kajsa [ 18/Mar/16 ]

YangTools parser reports these errors via exceptions, because the parser is unable to perform such augmentation and in consequence it is unable to create consistent SchemaContext. But it is up to each ODL component (e.g. netconf etc.) how to handle these reported errors, whether it is an error state or not.

Comment by Giles Heron [ 18/Mar/16 ]

right - but as there's no context in the exception (IIRC the exception doesn't even tell you the name of the failed model) NETCONF has to assume the "worst case" and drop the mount. Which really isn't a great idea if the error only impacts some models. We'd much rather have any such models end up as "unavailable-capabilities" or similar.

Comment by Peter Kajsa [ 23/Mar/16 ]

We investigated the attached karaf log again and we find out the root cause of the issue. Yang file tailf-common-monitoring-ann breaks the rules of current yang statement parser grammar and therefore yang parser reported mismatched input in tailf-common-monitoring-ann (line 114763 of the log file).

I examined tailf-common-monitoring-ann and the issue is caused by the statements which contain characters like ['= in their argument and the argument is not enclosed in "" or '' (e.g. tailf:annotate-statement grouping[name='common-monitoring-objects']).

We will try to fix current yang statement grammar to support such statements and arguments. However, all yang models with "-ann" postfix should be excluded from export, because they break also other RFC6020 rules.

Comment by Peter Kajsa [ 28/Apr/16 ]

Yang lexer fix: https://git.opendaylight.org/gerrit/#/c/38190/
Please retest with this patch.

Comment by Giles Heron [ 28/Apr/16 ]

the patch fixes the issue with tailf-common-monitoring-ann (I applied it manually and re-built beryllium).

before the fix I saw logs such as:

2016-04-28 21:55:19,572 | WARN | sing-executor-16 | NetconfDevice | 158 - org.opendaylight.netconf.sal-netconf-connector - 1.3.2.Beryllium-SR2 | RemoteDevice

{nso}

: Unable to build schema context, missing source SourceIdentifier [name=tailf-common-monitoring-ann@0000-00-00], will reattempt without it

but now those have gone.

but I still see the deref issues (tho am not sure those are actually the problem - rather it looks like all the tailf:action statements are the problem) and I'm still unable to mount ncs-3.4 from Beryllium.

Comment by Giles Heron [ 28/Apr/16 ]

the patch fixes the issue with tailf-common-monitoring-ann (I applied it manually and re-built beryllium).

before the fix I saw logs such as:

2016-04-28 21:55:19,572 | WARN | sing-executor-16 | NetconfDevice | 158 - org.opendaylight.netconf.sal-netconf-connector - 1.3.2.Beryllium-SR2 | RemoteDevice

{nso}

: Unable to build schema context, missing source SourceIdentifier [name=tailf-common-monitoring-ann@0000-00-00], will reattempt without it

but now those have gone.

but I still see the deref issues (tho am not sure those are actually the problem - rather it looks like all the tailf:action statements are the problem) and I'm still unable to mount ncs-3.4 from Beryllium.

logs are at: https://www.dropbox.com/s/9mh568b11zkh8wv/karaf.log?dl=0

Comment by Peter Kajsa [ 20/Jul/16 ]

As I mentioned in my previous comments, I am sure the issue is not caused by invalid xpath expression due to deref function (please see the following test:
https://git.opendaylight.org/gerrit/#/c/42047/1), but augments to an unknown statement are the root cause e.g. tailf:action (in Lithium they were reported as warnings, but in Beryllium they are intentionaly reported as errors - please see my previous comments).

So I close this bug, because deref statement does definitely not cause this issue. If you want to handle augments to an unknown statement (e.g. tailf:action connect/input) in another way, please raise this question in community and open new bug with properly bug subject and description. Thanks.

Generated at Wed Feb 07 20:53:42 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.