[NETCONF-1130] (devices) POST returns 500 on data already exists Created: 10/Aug/23  Updated: 22/Jan/24  Resolved: 22/Jan/24

Status: Resolved
Project: netconf
Component/s: restconf-nb
Affects Version/s: 5.0.7, 4.0.8, 6.0.1
Fix Version/s: 7.0.0, 5.0.10, 6.0.6

Type: Bug Priority: Medium
Reporter: Ivan Hrasko Assignee: Yaroslav Lastivka
Resolution: Done Votes: 0
Labels: pt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File karaf.log    
Issue Links:
Blocks
is blocked by YANGTOOLS-1541 Expose YangInstanceIdentifier codec f... Resolved
is blocked by YANGTOOLS-1543 XmlStringInstanceIdentifierCodec is u... Resolved
Duplicate
duplicates NETCONF-782 RestconfDocumentedExceptionMapper una... Resolved
Relates
relates to YANGTOOLS-1542 Improve YangInstanceIdentifier serial... Resolved

 Description   

Successfully create a resource of your choice on device using POST request. Then invoke the same request again, for example:

POST 

http://localhost:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount
{
  "toaster:toaster": {
    "toasterManufacturer": "pantheon",
    "toasterModelNumber": 25,
    "toasterStatus": "up",
    "darknessFactor": 1000
  }
} 

We get the error response (500 Internal Server Error):

<html><head>
    <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1" />
    <title>Error 500 Internal Server Error</title>
</head><body>
    <h2>HTTP ERROR 500 Internal Server Error</h2>
    <table>
        <tr>
            <th>URI:</th>
            <td>/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount
            </td>
        </tr>
        <tr>
            <th>STATUS:</th>
            <td>500</td>
        </tr>
        <tr>
            <th>MESSAGE:</th>
            <td>Internal Server Error</td>
        </tr>
        <tr>
            <th>SERVLET:</th>
            <td>org.glassfish.jersey.servlet.ServletContainer</td>
        </tr>
    </table></body></html>

According to RFC 8040 we should get 409 Conflict. Something similar to:

<errors xmlns="urn:ietf:params:xml:ns:yang:ietf-restconf">
    <error>
        <error-type>protocol</error-type>
        <error-tag>data-exists</error-tag>
        <error-path
        xmlns:rc="urn:ietf:params:xml:ns:yang:ietf-restconf"
        xmlns:jbox="https://example.com/ns/example-jukebox">\
        /rc:restconf/rc:data/jbox:jukebox
        </error-path>
        <error-message>
        Data already exists; cannot create new resource
        </error-message>
    </error>
</errors>


 Comments   
Comment by Ivan Hrasko [ 11/Aug/23 ]

It seems that we do not check if data already exists for default value of insert (last): https://datatracker.ietf.org/doc/html/rfc8040#section-4.8.5.

See PostDataTransactionUtil#submitData.

In addition, we need unit test coverage for this scenario.

Comment by Peter Suna [ 08/Sep/23 ]

The issue arises only when the device produces an error. The device generates the correct response with an appropriate NetconfDocumentedException error. This exception is then mapped to a RestconfDocumentedException in the TransactionUtil#decodeException method, along with the YangInstanceIdentifier path of the device schema, for example: "/(http://netconfcentral.org/ns/toaster2?revision=2009-11-20)toaster/darknessFactor". 

Subsequently, this RestconfDocumentedException is mapped to a JSON/XML Response in the RestconfDocumentedExceptionMapper, using the SchemaContext of the controller.  Notably, this SchemaContext does not include the models of the device.

The path of the device model is added to the "error-path" leaf and validated by the Controller SchemaContext. This validation process results in a Mapper exception, leading to a 500 error, as seen in the following log snippet:

2023-09-05T12:38:46,308 | ERROR | qtp1936609051-682 | ServerRuntime$Responder          | 172 - org.glassfish.jersey.core.jersey-server - 2.40.0 | An exception has been thrown from an exception mapper class org.opendaylight.restconf.nb.rfc8040.jersey.providers.errors.RestconfDocumentedExceptionMapper.
java.lang.IllegalArgumentException: Invalid input /(http://netconfcentral.org/ns/toaster2?revision=2009-11-20)toaster/darknessFactor: schema for argument (http://netconfcentral.org/ns/toaster2?revision=2009-11-20)toaster (after ) not found
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:463) ~[bundleFile:?]
	at org.opendaylight.yangtools.yang.data.util.AbstractStringInstanceIdentifierCodec.serializeImpl(AbstractStringInstanceIdentifierCodec.java:53) ~[bundleFile:?]

 

The simplest solution for this issue would be removing validation for "error-path", similar to what is done for the "error-info" leaf.

A more complex solution would involve retaining the validation and adding the Device schema context to the RestconfDocumentedExceptionMapper. Here are suggested steps for this solution:

  • RestconfDocumentedExceptionMapper will have DOMMountPointService in his constructor.
  • RestconfDocumentedException will have option to add UriInfo in the constructor.
  • Then we can create Inference stack for the device in method RestconfDocumentedExceptionMapper#toResponse as follows: 
if (exception.getUriInfo() != null) {
    final var identifierContext = ParserIdentifier.toInstanceIdentifier(
        exception.getUriInfo().getPathParameters(false).getFirst("identifier"),
        databindProvider.currentContext().modelContext(), mountPointService);
    final var deviceInference = identifierContext.inference();

} 
  • Use this device Inference for creating a NormalizedNodeStreamWriter:
    jsonNodeStreamWriter = JSONNormalizedNodeStreamWriter.createExclusiveWriter(databindContext.jsonCodecs(),
    inference, IETF_RESTCONF_URI, jsonWriter);
    
    
    xmlNodeStreamWriter = XMLStreamNormalizedNodeStreamWriter.create(xmlWriter, inference); 
Comment by Robert Varga [ 12/Sep/23 ]

So actually we should be taking a similar approach to what we do in JSONDataTreeCandidateFormatter (and XML as well). There we talk directly to a JsonWriter and then move to JsonDataTreeCandidateSerializer – which again emits some things directly through JsonWriter and emits other things through a nested JSONNormalizedNodeStreamWriter.

In this particular case, we want to emit pretty much everything directly, except for the YangInstanceIdentifier.
In DTCL formatters the EffectiveModelContext is always local, so we get away with just emitting startLeafNode()/scalarValue() for emitting the YangInstanceIdentifier (see NETCONF-1152).
Here we do not have that luxury and need help from yang-data-codec-

{json,xml}

: their codecs need to expose a YangInstanceIdentifier codec which just emits the encoding into an open element (in XML case) or value(String):

   XmlCodecFactory codecFactory; // corresponding to device context
   XMLStreamWriter xmlWriter; // bound to OutputStream
   xmlWriter.writeStartElement("error-path");
   codecFactory.instanceIdentifierSerializer().serializeTo(xmlWriter, path);
   xmlWriter.writeEndElement();

The resulting document contract is tied to the request URI – e.g. users reading need to understand they used yang-ext:mount and interpret things accordingly.

Comment by Sangwook Ha [ 14/Sep/23 ]

 I believe this is a duplicate of NETCONF-782.

Comment by Robert Varga [ 14/Sep/23 ]

Yes, it is a duplicate, but its solution is somewhat different and more in kind with eating our own dogfood.

Comment by Robert Varga [ 14/Sep/23 ]

https://git.opendaylight.org/gerrit/c/netconf/+/107877 fixes a related bug: in those two places we just do a straight-up YangInstanceIdentifier.toString().

Comment by Robert Varga [ 22/Sep/23 ]

Reported stack trace should have a tad different structure.

Generated at Wed Feb 07 20:16:45 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.