[CONTROLLER-185] BI Broker fails to serve remote rpc call Created: 24/Feb/14  Updated: 25/Jul/23  Resolved: 29/Jul/14

Status: Verified
Project: controller
Component/s: mdsal
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Abhishek Kumar Assignee: Maros Marsalek
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issue Links:
Blocks
blocks CONTROLLER-294 Milestone: Fix and demostrate Remote ... Resolved
External issue ID: 461
Priority: Normal

 Description   

The call fails at checkArgument() below. “inputContainer” is null.

public RpcResult<CompositeNode> invokeRpc(QName rpc, CompositeNode input) {
CompositeNode inputContainer = input.getFirstCompositeByName(QName.create(rpc,"input"));
checkArgument(inputContainer != null, "Rpc payload must contain input element”);

Debugging deeper, I found that MutableCompositeNodeTOImpl could not find “children” in nodeMap. The key “QName" in the NodeMap is different than the key “children” used to lookup.
public List<CompositeNode> getCompositesByName(QName children) {
List<Node<?>> toFilter = getNodeMap().get(children);

The QName sent by remote rpc router looks like:
{
namespace: "urn:opendaylight:flow:service"
localname: "input"
prefix: ""
formattedRevision: null
revision: null
}

While the key for flow service in NodeMap looks like:
{
namespace: "urn:opendaylight:flow:service"
localname: "input"
prefix: "flow"
formattedRevision: "2013-08-19"
revision: "Mon Aug 19 00:00:00 GMT-08:00 2013"
}

My RESTConf payload is:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<flow xmlns="urn:opendaylight:flow:inventory">
<priority>5</priority>
<flow-name>Foo</flow-name>
<match>
<ethernet-match>
<ethernet-type>
<type>2048</type>
</ethernet-type>
</ethernet-match>
<ipv4-destination>10.0.10.2/24</ipv4-destination>
</match>
<id>4</id>
<table_id>4</table_id>
<instructions>
<instruction>
<order>0</order>
<apply-actions>
<action>
<order>0</order>
<dec-nw-ttl/>
</action>
</apply-actions>
</instruction>
</instructions>
</flow>

The payload that I get in remote rpc router is
<add-flow xmlns="urn:opendaylight:flow:service">
<input>
<transaction-uri>BA-7</transaction-uri>
<table_id>4</table_id>
<priority>5</priority>
<node>
/(urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:1}

]
</node>
<match>
<ipv4-destination>10.0.10.2/24</ipv4-destination>
<ethernet-match>
<ethernet-type>
<type>2048</type>
</ethernet-type>
</ethernet-match>
</match>
<instructions>
<instruction>
<order>0</order>
<apply-actions>
<action>
<order>0</order>
<dec-nw-ttl/>
</action>
</apply-actions>
</instruction>
</instructions>
<flow-table>
/(urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:1}

]/(urn:opendaylight:flow:inventory?revision=2013-08-19)table[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=4}

]
</flow-table>
<flow-ref>
/(urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:1}

]/(urn:opendaylight:flow:inventory?revision=2013-08-19)table[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=4}

]/(urn:opendaylight:flow:inventory?revision=2013-08-19)flow[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=4}

]
</flow-ref>
<flow-name>Foo</flow-name>
</input>
</add-flow>



 Comments   
Comment by Tony Tkacik [ 27/Feb/14 ]

The contract of MD-SAL requires QNames always contain information according to the schema: this means namespace, revision, local-name.

If this is not present, MD-SAL could not distinguish between two different revisions and thus do the correct processing.

ZeroMQ should send QNames with revision, also all data upstreamed to MD-SAL
should contain revision information. Otherwise MD-SAL is not able to process that.

If you are using w3c Document as payload and XmlDocumentUtils to convert it to yang data - consider using SchemaService from ProviderContext to learn the schema, and then use corresponding methods
from XmlDocumentUtils which takes DataSchemaNode or SchemaContext as argument for normalization (and verification)of XmlDocument.

Comment by Tony Tkacik [ 27/Feb/14 ]

Abhishek please provide additional information which serialization / deserialization method are you using.

Comment by Abhishek Kumar [ 27/Feb/14 ]

Tony,

Ser/DSer code is in this class:
org.opendaylight.controller.sal.connector.remoterpc.util.XmlUtils

Let me know if this is not appropriate.

Copy/Pasting the code:

public static String compositeNodeToXml(CompositeNode cNode){
if (cNode == null) return new String();

Document domTree = NodeUtils.buildShadowDomTree(cNode);
StringWriter writer = new StringWriter();
try

{ TransformerFactory tf = TransformerFactory.newInstance(); Transformer transformer = tf.newTransformer(); transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes"); transformer.transform(new DOMSource(domTree), new StreamResult(writer)); }

catch (TransformerException e)

{ _logger.error("Error during translation of Document to OutputStream", e); }

return writer.toString();
}

public static CompositeNode xmlToCompositeNode(String xml){
if (xml==null || xml.length()==0) return null;

Node<?> dataTree;
try

{ dataTree = XmlTreeBuilder.buildDataTree(new ByteArrayInputStream(xml.getBytes())); }

catch (XMLStreamException e)

{ _logger.error("Error during building data tree from XML", e); return null; }

if (dataTree == null)

{ _logger.error("data tree is null"); return null; }

if (dataTree instanceof SimpleNode)

{ _logger.error("RPC XML was resolved as SimpleNode"); return null; }

return (CompositeNode) dataTree;
}

Comment by Vaishali Mithbaokar [ 07/May/14 ]

Hi Tony,

We followed your suggestions in the CONTROLLER-185 notes to use XMLDocumentUtil methods for serialization, deserialization. We first get the ProviderSession, then get SchemaService from it, and get SchemaContext from the SchemaService.
We tried using following methods

public static Document toDocument(CompositeNode data, DataNodeContainer schema, XmlCodecProvider codecProvider) (for converting CompositeNode to XML Document, and that works fine

But other way round we used following method, as that was the only suitable option

public static Node<?> toDomNode(Element xmlElement, Optional<DataSchemaNode> schema,
Optional<XmlCodecProvider> codecProvider) {

But it seems this method at
checkQName(xmlElement, schema.getQName());

Since the qName of both parameters are not equal. Following is the code which calls respective methods of XMLDocumentUtils, see the red highlighted lines. We need your guidance on how to proceed further, or is there any other method from XMLDocumentUtils that you had in mind?
We saw that toCompositeNodeWithSchema is private and can't be used.

/*

  • Copyright (c) 2013 Cisco Systems, Inc. and others. All rights reserved.
    *
  • This program and the accompanying materials are made available under the
  • terms of the Eclipse Public License v1.0 which accompanies this distribution,
  • and is available at http://www.eclipse.org/legal/epl-v10.html
    */

package org.opendaylight.controller.sal.connector.remoterpc.util;

import java.io.IOException;
import javax.xml.parsers.ParserConfigurationException;
import javax.activation.UnsupportedDataTypeException;
import org.opendaylight.yangtools.yang.data.api.CompositeNode;
import org.opendaylight.yangtools.yang.data.api.Node;
import org.opendaylight.yangtools.yang.data.api.SimpleNode;
import org.opendaylight.yangtools.yang.data.impl.NodeUtils;
import org.opendaylight.yangtools.yang.data.impl.XmlTreeBuilder;
import org.opendaylight.yangtools.yang.data.impl.codec.xml.XmlDocumentUtils;
import org.opendaylight.yangtools.yang.model.api.SchemaContext;
import org.opendaylight.yangtools.yang.model.api.DataSchemaNode;
import org.opendaylight.controller.netconf.util.xml.XmlUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.w3c.dom.Document;
import org.xml.sax.InputSource;
import javax.xml.stream.XMLStreamException;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import java.io.ByteArrayInputStream;
import java.io.StringWriter;
import org.xml.sax.SAXException;
import java.io.StringReader;
import com.google.common.base.Optional;
public class XmlUtils {

private static final Logger _logger = LoggerFactory.getLogger(XmlUtils.class);

public static String compositeNodeToXml(CompositeNode cNode, SchemaContext schemaContext){
if (cNode == null) return new String();

//Document domTree = NodeUtils.buildShadowDomTree(cNode);
Document domTree = null;
try

{ domTree = XmlDocumentUtils.toDocument(cNode, schemaContext, XmlDocumentUtils.defaultValueCodecProvider()); }

catch (UnsupportedDataTypeException e)

{ _logger.error("Error during translation of CompositeNode to Document", e); }

StringWriter writer = new StringWriter();
try

{ TransformerFactory tf = TransformerFactory.newInstance(); Transformer transformer = tf.newTransformer(); transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes"); transformer.transform(new DOMSource(domTree), new StreamResult(writer)); }

catch (TransformerException e)

{ _logger.error("Error during translation of Document to OutputStream", e); }

_logger.debug("compositeNodeToXml " + writer.toString());

return writer.toString();

}

public static CompositeNode xmlToCompositeNode(String xml, SchemaContext schemaContext){
if (xml==null || xml.length()==0) return null;

Node<?> dataTree = null;
try

{ Document doc = null; doc = XmlUtil.readXmlToDocument(xml); _logger.debug("vaishali xmlToCompositeNode Document is " + xml ); dataTree = XmlDocumentUtils.toDomNode(doc.getDocumentElement(), Optional.<DataSchemaNode>of(schemaContext), Optional.of(XmlDocumentUtils.defaultValueCodecProvider())); }

catch (SAXException e)

{ _logger.error("Error during building data tree from XML", e); } catch (IOException e) { _logger.error("Error during building data tree from XML", e); }

_logger.debug("xmlToCompositeNode " + dataTree.toString());
return (CompositeNode) dataTree;
}
}

Thanks,
Vaishali

Comment by Tony Tkacik [ 20/May/14 ]

Using SchemaContext only payload is assumed to be data tree, not RPC one.

You may want to search SchemaContext for definition of that RPC
and then getInput/getOutput from Rpc Definition and pass that as an
argument to xml serialization/deserialization - so the XmlDocumentUtils
knows that your data needs to comform to that specific RPC definition
not global data tree definition.

Comment by Tony Tkacik [ 02/Jul/14 ]

Offending component was removed.

Comment by Harman Singh [ 21/Jul/14 ]

Reopening the defect. I am trying to use it in RPC clustering.

Tony,

The comments you mentioned, i tried to use them like following

public static String inputCompositeNodeToXml(CompositeNode cNode, SchemaContext schemaContext){
if (cNode == null) return new String();

//Document domTree = NodeUtils.buildShadowDomTree(cNode);
Document domTree = null;
try {
Set<RpcDefinition> rpcs = schemaContext.getOperations();
for(RpcDefinition rpc : rpcs) {
if(rpc.getQName().equals(cNode.getNodeType()))

{ domTree = XmlDocumentUtils.toDocument(cNode, rpc.getInput(), XmlDocumentUtils.defaultValueCodecProvider()); break; }

}

} catch (UnsupportedDataTypeException e)

{ LOG.error("Error during translation of CompositeNode to Document", e); }

return domTransformer(domTree);
}

public static CompositeNode inputXmlToCompositeNode(QName rpc, String xml, SchemaContext schemaContext){
if (xml==null || xml.length()==0) return null;

Node<?> dataTree = null;
try {

Document doc = XmlUtil.readXmlToDocument(xml);
LOG.debug("xmlToCompositeNode Document is " + xml );
Set<RpcDefinition> rpcs = schemaContext.getOperations();
for(RpcDefinition rpcDef : rpcs) {
if(rpcDef.getQName().equals(rpc))

{ dataTree = XmlDocumentUtils.toDomNode(doc.getDocumentElement(), Optional.<DataSchemaNode>of(rpcDef.getInput()), Optional.of(XmlDocumentUtils.defaultValueCodecProvider())); break; }

}
} catch (SAXException e)

{ LOG.error("Error during building data tree from XML", e); } catch (IOException e) { LOG.error("Error during building data tree from XML", e); }

LOG.debug("xmlToCompositeNode " + dataTree.toString());
return (CompositeNode) dataTree;
}

It works fine while node to xml conversation and give me output, something like this

<add-flow xmlns="urn:opendaylight:flow:service"><input><flow-name>Foo</flow-name><flow-ref>(urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:249485023638597}

]/(urn:opendaylight:flow:inventory?revision=2013-08-19)table[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=3}

]/(urn:opendaylight:flow:inventory?revision=2013-08-19)flow[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=155}

]</flow-ref><flow-table>(urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:249485023638597}

]/(urn:opendaylight:flow:inventory?revision=2013-08-19)table[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=3}

]</flow-table><instructions><instruction><apply-actions><action><dec-nw-ttl/><order>0</order></action></apply-actions><order>0</order></instruction></instructions><match><ethernet-match><ethernet-type><type>2048</type></ethernet-type></ethernet-match><ipv4-destination>10.0.0.2/24</ipv4-destination></match><node>(urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:249485023638597}

]</node><priority>2</priority><table_id>3</table_id><transaction-uri>DOM-1</transaction-uri></input></add-flow>

But it still throws illegalArgument Exception when converting xml to DomNode on the second node.

Comment by Maros Marsalek [ 28/Jul/14 ]

Hi,

I have added a test to transform the input for an RPC using XmlDocumentUtils (I also tried the xml you provided):

https://git.opendaylight.org/gerrit/#/c/9380/

It worked fine so could you please provide the exact data you are transforming ? And could you also provide your code (as a gerrit draft or give me a pointer) that I can reproduce and debug the issue ?

Comment by Harman Singh [ 28/Jul/14 ]

Actually, CompositeNode to XML transformation works fine for me also, its the other way around, which does not work fine. When i try to convert XML to composite node, it does create the Node, but the values inside instance identifier are not converted based on type, they are always String.

Say in Add flow RPC, i have table id in flow-ref instanceidentifier, which should get converted to Short type, but it always gives me String

Comment by Maros Marsalek [ 29/Jul/14 ]

I see.

I have fixed the issue in the commit I posted (there is also a test to prove it):

https://git.opendaylight.org/gerrit/#/c/9380/

Could you please verify ? Setting state to: waiting for review

There was a problem when using derived type of instance identifier, but it is fixed now.

Comment by Harman Singh [ 29/Jul/14 ]

Thanks for your fix, this works now for normal use case.

there was one special case for which it won't work, I have a fix for that, i'll try to put patch for same.

Generated at Wed Feb 07 19:52:24 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.