[CONTROLLER-1202] netconf client can not parse RPC message begin with blank characters. Created: 13/Mar/15  Updated: 09/Jun/15  Resolved: 09/Jun/15

Status: Resolved
Project: controller
Component/s: netconf
Affects Version/s: Helium
Fix Version/s: None

Type: Bug
Reporter: Bo Li Assignee: Gwenael Lambrouin
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


External issue ID: 2838

 Description   

I have a netconf server which sent the RPC-reply begins and ends with new line character 0x0a as following message. sax exception will be thrown. I try to trim the message in the NetconfXMLToMessageDecoder.decode(), the exception disappears.

0a 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 2e 30 22 20 65 6e 63 6f 64 69 6e 67 3d 22 55 54 46 2d 38 22 3f 3e
......
3c 2f 72 70 63 2d 72 65 70 6c 79 3e 0a

Caused by: org.xml.sax.SAXParseException: The processing instruction target matching "[xX][mM][lL]" is not allowed.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)[:]
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)[:]
at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)[:2.4.0]
at org.opendaylight.controller.netconf.util.xml.XmlUtil.readXmlToDocument(XmlUtil.java:99)[191:org.opendaylight.controller.netconf-util:0.2.7.Helium-SR2]
at org.opendaylight.controller.netconf.nettyutil.handler.NetconfXMLToMessageDecoder.decode(NetconfXMLToMessageDecoder.java:29)[199:org.opendaylight.controller.netconf-netty-util:0.2.7.Helium-SR2]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249)[203:io.netty.codec:4.0.23.Final]
... 15 more



 Comments   
Comment by Gwenael Lambrouin [ 20/Mar/15 ]

I proposed a patch to fix this problem here:
https://git.opendaylight.org/gerrit/16927

Comment by Maros Marsalek [ 23/Mar/15 ]

Please take bugs if you work on them (so we can keep track of the bugs). I assigned it to you for you this time.

Also if you provide a fix and push into gerrit, set me as a reviewer just so I can see it in gerrit web UI.

Thanks and Thanks for the patches

Comment by Maros Marsalek [ 23/Mar/15 ]

As mentioned in gerrit, such XML is invalid and its expectad to fail during parsing. Have you tried finding a fix on the side of remote devices so that the devices do not return invalid XML ?

Comment by Bo Li [ 24/Mar/15 ]

I have test the netconf server with Netopeer,Mg-soft,Seguesoft netconf client. Both of them can communicate correctly.

I don't think blank character before <?xml means a invalid xml format. it should be a noise in transport. it is too strict for this check in ODL.

I think it should be fixed from both side..

Comment by Maros Marsalek [ 24/Mar/15 ]

I believe that it is invalid XML according to the XML specification. Thats why the standard java parser fails with exception. So before merging a fix into ODL codebase, the netconf devices should be checked.

I am not against making it less strict in ODL, but it should be verified in the devices.

Btw. I commented on proposed fix. Could you or Gwenael take a look at those in the meantime ?

Thanks

Comment by Gwenael Lambrouin [ 27/Mar/15 ]

I made some unit tests of NetconfXMLToMessageDecoder.decode(), and I observed the following behaviour:

  • when the message to decode does NOT start with a XML declaration (reminder: a "XML declaration" looks like "<?xml version="1.0" encoding="UTF-8"?>": leading white spaces are accepted by the parser.
  • when the message to decode starts with a XML declaration: leading white spaces are rejected by the parser.

This looks inconsistent, but in my understanding it is correct with respect to the XML 1.0 specifications (see http://www.w3.org/TR/REC-xml/#sec-well-formed and http://www.w3.org/TR/REC-xml/#NT-prolog). So according to XML, I think that the ODL code is correct.

I also looked at RFC 4742 (NETCONF over SSH). The examples are all nicely formatted with new lines and they include XML declarations. Unfortunately, there is no example with two consecutive messages. But in the spirit of that RFC, I think it is correct to have a new line just before an XML declaration at the beginning of a NETCONF message . This is really useful in interactive SSH sessions with a human being because it improves readability.

Also, I made some tests with another network device that does not include XML declarations (a fairly recent Juniper router): the device starts its messages with a newline. This is accepted by ODL because there is not XML declaration.

So I am now convinced that in the spirit of NETCONF, it is correct to accept messages coming with a leading newline and an XML declaration. So I think ODL should accept such messages.

Now, I see three ways to fix the issue in ODL:
1. do the minimum to support the interoperability problems observed today: remove at most one line break at the beginning of the message: either LF (the case observed by Bo when he reported the bug) or CRLF (the case I observed with Cisco routers).
2. accept any number of XML white spaces (space, tab, CR, LF) at the beginning of the message (this is (almost) my initial proposal)
3. accept anything before the first '<' character at the beginning of the message (this is Maros's proposal)

Solution 1 is enough today, and it looks fine in the spirit of RFC 4742. However, the problem may resurface with a device that would eg start a messge with two line breaks.

Solution 3 works fine and solves the problem. However, it looks excessive to me, because it would make ODL accept messages such as "slkdjf lsdkjf d<hello/>". While network device vendors do not use ODL as a reference NETCONF client to test their devices, this is not really a problem. But I think ODL should not do that.

For me solution 2 provides a good balance between interop and strictness, and I still prefer this solution. But tell me what you think about it.

In the meantime, I uploaded a new patchset that implements solution 3 (Maros code suggestion) and adds some unit tests:
https://git.opendaylight.org/gerrit/16927

Generated at Wed Feb 07 19:54:56 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.