[NETCONF-1051] SSE with sub identifier does not work Created: 05/Jun/23  Updated: 11/Oct/23  Resolved: 11/Oct/23

Status: Resolved
Project: netconf
Component/s: restconf-nb
Affects Version/s: 4.0.5, 5.0.4
Fix Version/s: 7.0.0, 5.0.8, 6.0.3

Type: Bug Priority: High
Reporter: Olivier Dugeon Assignee: Sangwook Ha
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When you create a stream with SSE on a sub node of the data store, stream events are not reported. For example subscribe to PCEP Topology within the network topology with:
 

{
  "input": {
    "path": "/network-topology:network-topology/topology[topology-id='pcep-topology']",
    "sal-remote-augment:datastore": "OPERATIONAL",
    "sal-remote-augment:scope": "SUBTREE",
    "notification-output-type": "JSON"
  }
}

 
will not produce any events while subscribing to top "network-topology:network-topology" produces event.

Note that adding optional 'odl-leaf-nodes-only=true' when you get the stream url will work. SSE events are produce as expected.



 Comments   
Comment by Olivier Dugeon [ 05/Jun/23 ]

To complement, when subscribing to such node event in the Data Store (here graph:graph-topology/graph[name='ted://example-linkstate-topology']), karaf.log show some Java Exception:

2023-06-05T15:01:24,365 | ERROR | opendaylight-cluster-data-notification-dispatcher-48 | ListenerAdapter                  | 267 - org.opendaylight.netconf.restconf-nb - 5.0.4 | Failed to process notification DefaultDataTreeCandidate{rootPath=/(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph-topology/graph/graph[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name=ted://example-linkstate-topology}], rootNode=ChildNode{mod = ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name=ted://example-linkstate-topology}], operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=180506016}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=180506016}], operation=WRITE, modificationType=WRITE}}}}}

<skip too long debug message which show the full content of the modified node in the graph>

 

java.lang.IllegalArgumentException: List item is not appropriate
    at com.google.common.base.Preconditions.checkArgument(Preconditions.java:145) ~[bundleFile:?]
    at org.opendaylight.yangtools.yang.data.util.NormalizedNodeStreamWriterStack.startListItem(NormalizedNodeStreamWriterStack.java:207) ~[bundleFile:?]
    at org.opendaylight.yangtools.yang.data.codec.gson.JSONNormalizedNodeStreamWriter.startMapEntryNode(JSONNormalizedNodeStreamWriter.java:375) ~[bundleFile:?]
    at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter$OrderedNormalizedNodeWriter.writeMapEntryNode(NormalizedNodeWriter.java:241) ~[bundleFile:?]
    at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:202) ~[bundleFile:?]
    at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:102) ~[bundleFile:?]
    at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JsonDataTreeCandidateSerializer.serializeData(JsonDataTreeCandidateSerializer.java:47) ~[bundleFile:?]
    at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serializeData(AbstractWebsocketSerializer.java:99) ~[bundleFile:?]
    at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serialize(AbstractWebsocketSerializer.java:48) ~[bundleFile:?]
    at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:70) ~[bundleFile:?]
    at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:23) ~[bundleFile:?]
    at org.opendaylight.restconf.nb.rfc8040.streams.listeners.EventFormatter.eventData(EventFormatter.java:78) ~[bundleFile:?]
    at org.opendaylight.restconf.nb.rfc8040.streams.listeners.ListenerAdapter.onDataTreeChanged(ListenerAdapter.java:78) ~[bundleFile:?]
    at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataTreeChanged(DataTreeChangeListenerActor.java:84) ~[bundleFile:?]
    at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:45) ~[bundleFile:?]
    at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?]
    at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?]
    at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?]
    at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?]
    at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) ~[bundleFile:?]
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ~[bundleFile:?]
    at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?]
    at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?]
    at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[bundleFile:?]
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) ~[bundleFile:?]
    at akka.actor.ActorCell.invoke(ActorCell.scala:547) ~[bundleFile:?]
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[bundleFile:?]
    at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[bundleFile:?]
    at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[bundleFile:?]
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?]
 

 

Note also that the log file continuously grow while SSE socket is open and there is events to be reported with a risk of file system saturation. 

Comment by Robert Varga [ 05/Jun/23 ]

Looks like JsonDataTreeCandidateSerializer does not correctly initialize the writer stack: this is coming from:

 

    public void startListItem(final PathArgument name) throws IOException {
        final Object schema = getParent();
        checkArgument(schema instanceof ListSchemaNode, "List item is not appropriate");
        schemaStack.push((ListSchemaNode) schema);
    }

needs some further investigation

 

Comment by Olivier Dugeon [ 19/Jun/23 ]

Hello Robert,

After doing many try, I got a different java error telling that the data tree child is not present in the parent schema while it is:

11:30:21.478 INFO [CommitFutures-5] GraphModel: Edge(r1 - r2) has been published in operational datastore 
11:30:21.478 ERROR [opendaylight-cluster-data-notification-dispatcher-38] Failed to process notification DefaultDataTreeCandidate{rootPath=/(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph-topology/graph/graph[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name=LB042}]/edge/edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], rootNode=ChildNode{mod = ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], operation=WRITE, modificationType=WRITE}, oldMeta = SimpleContainerNode{version=org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2045e89c, data=ImmutableMapEntryNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-vertex-id, body=2}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-vertex-id, body=1}, ImmutableContainerNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-attributes, body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)metric, body=50}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-address, body=10.194.77.180}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)adj-sid, body=15000}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)te-metric, body=50}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-address, body=10.194.77.181}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-resv-link-bandwidth, body=176258176.0}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-link-bandwidth, body=176258176.0}]}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id, body=1}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name, body=r1 - r2}]}}, newMeta = SimpleContainerNode{version=org.opendaylight.yangtools.yang.data.tree.impl.node.Version@73d13f2d, data=ImmutableMapEntryNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-vertex-id, body=2}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-vertex-id, body=1}, ImmutableContainerNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-attributes, body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)metric, body=20}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-address, body=10.194.77.180}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)adj-sid, body=15000}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)te-metric, body=50}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-address, body=10.194.77.181}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-resv-link-bandwidth, body=176258176.0}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-link-bandwidth, body=176258176.0}]}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id, body=1}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name, body=r1 - r2}]}}}}
java.lang.IllegalArgumentException: Data tree child (urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id not present in schema parent (urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph
        at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.notPresent(SchemaInferenceStack.java:965) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.lambda$pushData$6(SchemaInferenceStack.java:771) ~[bundleFile:?]
        at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
        at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.pushData(SchemaInferenceStack.java:771) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.pushData(SchemaInferenceStack.java:765) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.pushData(SchemaInferenceStack.java:759) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.enterDataTree(SchemaInferenceStack.java:473) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.util.NormalizedNodeStreamWriterStack.enterDataTree(NormalizedNodeStreamWriterStack.java:188) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.util.NormalizedNodeStreamWriterStack.startLeafNode(NormalizedNodeStreamWriterStack.java:212) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.codec.gson.JSONNormalizedNodeStreamWriter.startLeafNode(JSONNormalizedNodeStreamWriter.java:319) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessAsSimpleNode(NormalizedNodeWriter.java:143) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:106) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter$OrderedNormalizedNodeWriter.writeMapEntryNode(NormalizedNodeWriter.java:248) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:202) ~[bundleFile:?]
        at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:102) ~[bundleFile:?]
        at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JsonDataTreeCandidateSerializer.serializeData(JsonDataTreeCandidateSerializer.java:47) ~[bundleFile:?]
        at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serializeData(AbstractWebsocketSerializer.java:215) ~[bundleFile:?]
        at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serialize(AbstractWebsocketSerializer.java:65) ~[bundleFile:?]
        at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:70) ~[bundleFile:?]
        at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:23) ~[bundleFile:?]
        at org.opendaylight.restconf.nb.rfc8040.streams.listeners.EventFormatter.eventData(EventFormatter.java:78) ~[bundleFile:?]
        at org.opendaylight.restconf.nb.rfc8040.streams.listeners.ListenerAdapter.onDataTreeChanged(ListenerAdapter.java:78) ~[bundleFile:?]
        at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataTreeChanged(DataTreeChangeListenerActor.java:84) ~[bundleFile:?]
        at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:45) ~[bundleFile:?]
        at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?]
        at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?]
        at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?]
        at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?]
        at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?]
        at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) ~[bundleFile:?]
        at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ~[bundleFile:?]
        at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?]
        at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?]
        at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[bundleFile:?]
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) ~[bundleFile:?]
        at akka.actor.ActorCell.invoke(ActorCell.scala:547) ~[bundleFile:?]
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[bundleFile:?]
        at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[bundleFile:?]
        at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[bundleFile:?]
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?]
        at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[?:?]
        at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[?:?]
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[?:?]
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[?:?]

I'm using this path to produce this error: 

"path": "/graph:graph-topology/graph[name='LB042']/edge[edge-id='1']"

which is correctly handle during the subscription and events are correctly send with 'odl-leaf-nodes-only' query parameter

Comment by Sangwook Ha [ 20/Jun/23 ]

We bumped into the same issue, so I just submitted a patch for review: https://git.opendaylight.org/gerrit/c/netconf/+/106565

odd22 Can you test if the patch works for you?

Comment by Olivier Dugeon [ 21/Jun/23 ]

Hi Sangwook

Thanks a lot for your patch. I tested it and it works fine completely resolved this issue (both reported errors). I added my review (+1).

 

I'll will submit today a new PR on top of yours to add new 'child nodes only' query parameter to get in compact form only modified child and not the complete Data Tree from the root node.

 

Olivier

Generated at Wed Feb 07 20:16:33 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.