[YANGTOOLS-1107] XML document: Lists interleaving not supported

Relates — Wed, 10 Jun 2020 18:31:09 +0000

Hi, I believe I found a new bug related to XML document parsing when your YANG model has multiple lists in the same container causing them to MAY have interleaves between their elements.

Using netopeer2+sysrepo as NETCONF server.

1. Consider the follow YANG model:

module test2 {
    namespace "testenv:idea:test2";
    prefix "tt";
    container parent {
        config true;
        list user {
            key name;
            leaf name {
                type string;
            }
        }
        list admin {
            key name;
            leaf name {
                type string;
            }
        }
    }
}

2. Do the following operations in the NETCONF server in this order (In my case I used 3 RPCs edit-config, but should have the same result for YANG patch or RESTCONF POST, not the main point of the issue).

a. create parent/user/name="Bob"

b. create parent/admin/name="John"

c. create parent/user/name="Fred"

3. That results in the following XML returned by netopeer2+sysrepo to a NETCONF GET operation. Notice that elements of "admin" and "user" are interleaved.

<parent xmlns="testenv:idea:test2">
  <user>
    <name>Bob</name>
  </user>
  <admin>
    <name>John</name>
  </admin>
  <user>
    <name>Freud</name>
  </user>
</parent>

This is a valid XML according to RFC7950, last paragraph of section 7.8.5:

The XML elements representing list entries MAY be interleaved with elements for siblings of the list, unless the list defines RPC or action input or output parameters.

However, Opendaylight fails to parse this XML after a RESTCONF GET to mount-point test2:parent, returning (full log in the bottom of the issue):

<errors xmlns="urn:ietf:params:xml:ns:yang:ietf-restconf">
    <error>
        <error-message>Request could not be completed because the relevant data model content does not exist</error-message>
        <error-tag>data-missing</error-tag>
        <error-type>protocol</error-type>
    </error>
</errors>

If you change the insertion order to:

a. create parent/user/name="Bob"

b. create parent/user/name="Fred"

c. create parent/admin/name="John"

The resulting XML does not have interleaves and the RESTCONF GET works as expected returning:

<parent xmlns="testenv:idea:test2">
    <admin>
        <name>John</name>
    </admin>
    <user>
        <name>Freud</name>
    </user>
    <user>
        <name>Bob</name>
    </user>
</parent>

Full error log confirms the issue parsing interleaving XML documents:

ERROR [nioEventLoopGroupCloseable-3-1] Cannot parse anyxml.ERROR [nioEventLoopGroupCloseable-3-1] Cannot parse anyxml.javax.xml.stream.XMLStreamException: ParseError at [row,col]:[-1,-1]Message: Duplicate namespace "testenv:idea:test2" element "user" in XML input at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.read(XmlParserStream.java:471) ~[314:org.opendaylight.yangtools.yang-data-codec-xml:4.0.6] at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.read(XmlParserStream.java:486) ~[314:org.opendaylight.yangtools.yang-data-codec-xml:4.0.6] at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.parse(XmlParserStream.java:270) ~[314:org.opendaylight.yangtools.yang-data-codec-xml:4.0.6] at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.traverse(XmlParserStream.java:297) ~[314:org.opendaylight.yangtools.yang-data-codec-xml:4.0.6] at org.opendaylight.netconf.util.NetconfUtil.transformDOMSourceToNormalizedNode(NetconfUtil.java:175) ~[288:org.opendaylight.netconf.util:1.8.0] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfRpcStructureTransformer.selectFromDataStructure(NetconfRpcStructureTransformer.java:46) ~[285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfBaseOps.lambda$extractData$0(NetconfBaseOps.java:226) ~[285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:242) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:232) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:726) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.SettableFuture.set(SettableFuture.java:48) [76:com.google.guava:27.1.0.jre] at org.opendaylight.netconf.sal.connect.netconf.sal.NetconfDeviceRpc$1.onSuccess(NetconfDeviceRpc.java:60) [285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at org.opendaylight.netconf.sal.connect.netconf.sal.NetconfDeviceRpc$1.onSuccess(NetconfDeviceRpc.java:57) [285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1076) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958) [76:com.google.guava:27.1.0.jre] at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:726) [76:com.google.guava:27.1.0.jre] at org.opendaylight.netconf.sal.connect.netconf.listener.UncancellableFuture.set(UncancellableFuture.java:45) [285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at org.opendaylight.netconf.sal.connect.netconf.listener.NetconfDeviceCommunicator.processMessage(NetconfDeviceCommunicator.java:337) [285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at org.opendaylight.netconf.sal.connect.netconf.listener.NetconfDeviceCommunicator.onMessage(NetconfDeviceCommunicator.java:269) [285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at org.opendaylight.netconf.sal.connect.netconf.listener.NetconfDeviceCommunicator.onMessage(NetconfDeviceCommunicator.java:48) [285:org.opendaylight.netconf.sal-netconf-connector:1.11.0] at org.opendaylight.netconf.nettyutil.AbstractNetconfSession.handleMessage(AbstractNetconfSession.java:64) [280:org.opendaylight.netconf.netty-util:1.8.0] at org.opendaylight.netconf.nettyutil.AbstractNetconfSession.channelRead0(AbstractNetconfSession.java:187) [280:org.opendaylight.netconf.netty-util:1.8.0] at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [105:io.netty.transport:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) [105:io.netty.transport:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [105:io.netty.transport:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) [105:io.netty.transport:4.1.45.Final] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) [100:io.netty.codec:4.1.45.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) [100:io.netty.codec:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) [105:io.netty.transport:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [105:io.netty.transport:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) [105:io.netty.transport:4.1.45.Final] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) [100:io.netty.codec:4.1.45.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) [100:io.netty.codec:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) [105:io.netty.transport:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:59) [105:io.netty.transport:4.1.45.Final] at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:368) [105:io.netty.transport:4.1.45.Final] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [102:io.netty.common:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [102:io.netty.common:4.1.45.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [105:io.netty.transport:4.1.45.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [102:io.netty.common:4.1.45.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [102:io.netty.common:4.1.45.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [102:io.netty.common:4.1.45.Final] at java.lang.Thread.run(Thread.java:834) [?:?]

OpenDaylight JIRA

[YANGTOOLS-1107] XML document: Lists interleaving not supported