[YANGTOOLS-586] Regex processing of yang models is broken. Models that worked on Li does not work on Be. Created: 20/Feb/16  Updated: 10/Apr/22  Resolved: 28/Mar/16

Status: Resolved
Project: yangtools
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Kevin Wang Assignee: Peter Kajsa
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: Text File karaf.log    
Issue Links:
Blocks
is blocked by YANGTOOLS-590 java.lang.RuntimeException: RemoteDev... Resolved
External issue ID: 5396

 Description   

Reproduce:

1. Fresh install Beryllium.
2. Mount a Vyatta 4.0 device with either the old yang:ext way or the new Beryllium way.
3. Do a GET on /restconf/config/opendaylight-inventory:nodes/node/vyatta/yang-ext:mount/
or on /restconf/operational/opendaylight-inventory:nodes/node/vyatta/yang-ext:mount/

You will see an exception thrown:

<p>Problem accessing /restconf/config/opendaylight-inventory:nodes/node/vyatta/yang-ext:mount/. Reason:
<pre> Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.RuntimeException: RemoteDevice

{vyatta}

: readOperationalData failed
at org.opendaylight.netconf.sal.connect.netconf.sal.tx.ReadOnlyTx.readWithTimeout(ReadOnlyTx.java:170)
at org.opendaylight.netconf.sal.connect.netconf.sal.tx.ReadOnlyTx.readConfigurationData(ReadOnlyTx.java:87)
at org.opendaylight.netconf.sal.connect.netconf.sal.tx.ReadOnlyTx.read(ReadOnlyTx.java:139)
at org.opendaylight.netconf.sal.restconf.impl.BrokerFacade.readDataViaTransaction(BrokerFacade.java:194)
at org.opendaylight.netconf.sal.restconf.impl.BrokerFacade.readConfigurationData(BrokerFacade.java:87)
at org.opendaylight.netconf.sal.restconf.impl.RestconfImpl.readConfigurationData(RestconfImpl.java:663)
at org.opendaylight.netconf.sal.restconf.impl.StatisticsRestconfServiceWrapper.readConfigurationData(StatisticsRestconfServiceWrapper.java:95)
at org.opendaylight.netconf.sal.rest.impl.RestconfCompositeWrapper.readConfigurationData(RestconfCompositeWrapper.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceObjectRule.accept(ResourceObjectRule.java:100)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1496)
at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:247)
at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:210)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:69)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:240)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:75)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
</pre>

  • This problem only exists on Vyatta 4.0 with Beryllium. It works on Vyatta 3.5, it works on Vyatta 4.0 and Lithium as well
  • Karaf log attached

 Comments   
Comment by Kevin Wang [ 20/Feb/16 ]

Attachment karaf.log has been added with description: karaf log

Comment by Tony Tkacik [ 22/Feb/16 ]

Could you verify that value "dp0s3" is valid for leaf /interfaces/dataplane/tagnode?

{path=[(urn:vyatta.com:mgmt:vyatta-interfaces:1?revision=2015-08-05)interfaces, (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-14)dataplane, (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-14)tagnode]

The underlaying exception is:

Failed to parse element [tagnode: null] as leaf AbsoluteSchemaPath

{path=[(urn:vyatta.com:mgmt:vyatta-interfaces:1?revision=2015-08-05)interfaces, (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-14)dataplane, (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-14)tagnode]}

at org.opendaylight.yangtools.yang.data.impl.schema.transform.dom.parser.LeafNodeDomParser.parseLeaf(LeafNodeDomParser.java:44)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.dom.parser.LeafNodeDomParser.parseLeaf(LeafNodeDomParser.java:22)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.LeafNodeBaseParser.parse(LeafNodeBaseParser.java:47)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.LeafNodeBaseParser.parse(LeafNodeBaseParser.java:28)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.NodeParserDispatcher$BaseNodeParserDispatcher.dispatchChildElement(NodeParserDispatcher.java:51)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.BaseDispatcherParser.parse(BaseDispatcherParser.java:147)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ListEntryNodeBaseParser.parse(ListEntryNodeBaseParser.java:61)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ListEntryNodeBaseParser.parse(ListEntryNodeBaseParser.java:28)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ListNodeBaseParser.parse(ListNodeBaseParser.java:46)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ListNodeBaseParser.parse(ListNodeBaseParser.java:26)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.NodeParserDispatcher$BaseNodeParserDispatcher.dispatchChildElement(NodeParserDispatcher.java:61)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.BaseDispatcherParser.parse(BaseDispatcherParser.java:177)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.BaseDispatcherParser.parse(BaseDispatcherParser.java:34)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.NodeParserDispatcher$BaseNodeParserDispatcher.dispatchChildElement(NodeParserDispatcher.java:66)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.BaseDispatcherParser.parse(BaseDispatcherParser.java:197)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ContainerNodeBaseParser.parse(ContainerNodeBaseParser.java:47)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ContainerNodeBaseParser.parse(ContainerNodeBaseParser.java:29)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.NodeParserDispatcher$BaseNodeParserDispatcher.dispatchChildElement(NodeParserDispatcher.java:49)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.BaseDispatcherParser.parse(BaseDispatcherParser.java:177)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ContainerNodeBaseParser.parse(ContainerNodeBaseParser.java:47)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.yangtools.yang.data.impl.schema.transform.base.parser.ContainerNodeBaseParser.parse(ContainerNodeBaseParser.java:29)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]
at org.opendaylight.netconf.sal.connect.netconf.schema.mapping.NetconfMessageTransformer.toRpcResult(NetconfMessageTransformer.java:294)[228:org.opendaylight.netconf.sal-netconf-connector:1.3.0.Beryllium-RC2]
... 33 more
Caused by: java.lang.IllegalArgumentException: Invalid value "dp0s3" for union type.
at org.opendaylight.yangtools.yang.data.impl.codec.UnionStringCodec.deserialize(UnionStringCodec.java:61)[82:org.opendaylight.yangtools.yang-data-impl:0.8.0.Beryllium-RC2]

Comment by Tony Tkacik [ 24/Feb/16 ]

Could you verify that value "dp0s3" is valid for leaf /interfaces/dataplane/tagnode?

{path=[(urn:vyatta.com:mgmt:vyatta-interfaces:1?revision=2015-08-05)interfaces, (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-14)dataplane, (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-14)tagnode]

Comment by Kevin Wang [ 02/Mar/16 ]

(In reply to Tony Tkacik from comment #2)
> Could you verify that value "dp0s3" is valid for leaf
> /interfaces/dataplane/tagnode?
>
> {path=[(urn:vyatta.com:mgmt:vyatta-interfaces:1?revision=2015-08-
> 05)interfaces,
> (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-
> 14)dataplane,
> (urn:vyatta.com:mgmt:vyatta-interfaces-dataplane:1?revision=2015-08-
> 14)tagnode]

Hi Tony,

What do you mean by "valid" here? I can see the dp0s3 interface working properly on the device. Could you tell me how I can verify this? Thanks

Comment by Balaji Varadaraju [ 04/Mar/16 ]

Hi Tony,
Yes, this is a valid interface on the device. We tested the same device on both Lithium and Be. On Li, it works fine. However on Be we get this error while doing a Get call. We tried get at different levels ( right at the child, its parent and at mount point) . So this looks like a regression. Do you need more information?

Thanks,
bala

Comment by Balaji Varadaraju [ 15/Mar/16 ]

Hi Tony,
The interface name is dp0s3. This name is failing the regex pattern in the YANG file and hence the error. The regex test passes fine in Li.
Here is the regex syntax for this field which looks fine and I think it should match the second one. Once I comment out regex rules in the yang files and relax the validation it works fine. So the issue seems to be in regex validation. Has anything changed in regex validation in Be?

typedef dataplane-ifname {
type union {
// Onboard device
type string

{ pattern "dp[0-9]+o[0-9]+(d[0-9]+)?"; configd:pattern-help "<dpFoN>"; }

// hotplug slot index
type string

{ pattern "dp[0-9]+s[0-9]+(f[0-9]+)?(d[0-9]+)?"; configd:pattern-help "<dpFsN>"; }

// PCI device address
type string {
pattern "dp[0-9](P[0-9])?p[0-9]

{1,3}s[0-9]{1,3}

(f[0-9])?(d[0-9])?";
configd:pattern-help "<dpFpNsS>";
}
// Biosdevname compatiable names. VMware only
type string

{ pattern "dp[0-9]+p[0-9]+p[0-9]+"; configd:pattern-help "<dpFpNpS>"; }

}
}

Comment by Ryan Goulding [ 16/Mar/16 ]

Hi Tony,

Any ideas on what changes in yang parser impl from Li->Be may have caused this change in behavior? Any input is appreciated.

Thanks,
Ryan

Comment by Ryan Goulding [ 21/Mar/16 ]

It appears if we comment out the patterns, then we are at least able to GET the endpoint. Was there a change the the pattern parsing code? If so, which files should I be looking at w/in the yangtools project?

Thanks,
Ryan

Comment by Tomas Cere [ 22/Mar/16 ]

Seems more like an issue with pattern definitions in yangtools, moving to yangtools.

Comment by Balaji Varadaraju [ 22/Mar/16 ]

Regex expressions in question are listed in my post above. Looks like a regression.

Comment by Peter Kajsa [ 22/Mar/16 ]

I am trying to replicate this bug and I am hitting exactly the same exception as it is in the karaf log (i.e. java.lang.IllegalArgumentException: Failed to parse element [...: null] as leaf ...), but only if the input string does not match any of the regex expressions in the union.

If the input string matches one of the regex expressions, everything work fine.

Could you send me please the input xml and yang models which cause the issue ?

Comment by Balaji Varadaraju [ 22/Mar/16 ]

HI Peter,
I'll see if I can gather those details Meanwhile its complaining on "dp0s3". It says this string does not match the regex. Do you think that should be the case? dp0s3 is the name of the data plane interface.

Do you think this information will help you debug further?

Thanks,
Bala

Comment by Peter Kajsa [ 22/Mar/16 ]

Yes, thank you. The bug has been replicated with this input string. I am investigating the bug further.

Comment by Peter Kajsa [ 23/Mar/16 ]

fix master: https://git.opendaylight.org/gerrit/#/c/36587/
fix stable/beryllium: https://git.opendaylight.org/gerrit/#/c/36589/

Comment by Balaji Varadaraju [ 23/Mar/16 ]

Peter, thanks a lot for the quick turn around on this. We'll test it once merged to Be.

Generated at Wed Feb 07 20:53:41 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.