[NETCONF-1051] SSE with sub identifier does not work Created: 05/Jun/23 Updated: 11/Oct/23 Resolved: 11/Oct/23 |
|
| Status: | Resolved |
| Project: | netconf |
| Component/s: | restconf-nb |
| Affects Version/s: | 4.0.5, 5.0.4 |
| Fix Version/s: | 7.0.0, 5.0.8, 6.0.3 |
| Type: | Bug | Priority: | High |
| Reporter: | Olivier Dugeon | Assignee: | Sangwook Ha |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
When you create a stream with SSE on a sub node of the data store, stream events are not reported. For example subscribe to PCEP Topology within the network topology with:
{
"input": {
"path": "/network-topology:network-topology/topology[topology-id='pcep-topology']",
"sal-remote-augment:datastore": "OPERATIONAL",
"sal-remote-augment:scope": "SUBTREE",
"notification-output-type": "JSON"
}
}
Note that adding optional 'odl-leaf-nodes-only=true' when you get the stream url will work. SSE events are produce as expected. |
| Comments |
| Comment by Olivier Dugeon [ 05/Jun/23 ] |
|
To complement, when subscribing to such node event in the Data Store (here graph:graph-topology/graph[name='ted://example-linkstate-topology']), karaf.log show some Java Exception: 2023-06-05T15:01:24,365 | ERROR | opendaylight-cluster-data-notification-dispatcher-48 | ListenerAdapter | 267 - org.opendaylight.netconf.restconf-nb - 5.0.4 | Failed to process notification DefaultDataTreeCandidate{rootPath=/(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph-topology/graph/graph[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name=ted://example-linkstate-topology}], rootNode=ChildNode{mod = ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name=ted://example-linkstate-topology}], operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=180506016}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=180506016}], operation=WRITE, modificationType=WRITE}}}}}
<skip too long debug message which show the full content of the modified node in the graph>
java.lang.IllegalArgumentException: List item is not appropriate at com.google.common.base.Preconditions.checkArgument(Preconditions.java:145) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.util.NormalizedNodeStreamWriterStack.startListItem(NormalizedNodeStreamWriterStack.java:207) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.codec.gson.JSONNormalizedNodeStreamWriter.startMapEntryNode(JSONNormalizedNodeStreamWriter.java:375) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter$OrderedNormalizedNodeWriter.writeMapEntryNode(NormalizedNodeWriter.java:241) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:202) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:102) ~[bundleFile:?] at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JsonDataTreeCandidateSerializer.serializeData(JsonDataTreeCandidateSerializer.java:47) ~[bundleFile:?] at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serializeData(AbstractWebsocketSerializer.java:99) ~[bundleFile:?] at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serialize(AbstractWebsocketSerializer.java:48) ~[bundleFile:?] at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:70) ~[bundleFile:?] at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:23) ~[bundleFile:?] at org.opendaylight.restconf.nb.rfc8040.streams.listeners.EventFormatter.eventData(EventFormatter.java:78) ~[bundleFile:?] at org.opendaylight.restconf.nb.rfc8040.streams.listeners.ListenerAdapter.onDataTreeChanged(ListenerAdapter.java:78) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataTreeChanged(DataTreeChangeListenerActor.java:84) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:45) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) ~[bundleFile:?] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ~[bundleFile:?] at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?] at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[bundleFile:?] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) ~[bundleFile:?] at akka.actor.ActorCell.invoke(ActorCell.scala:547) ~[bundleFile:?] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[bundleFile:?] at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[bundleFile:?] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[bundleFile:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?]
Note also that the log file continuously grow while SSE socket is open and there is events to be reported with a risk of file system saturation. |
| Comment by Robert Varga [ 05/Jun/23 ] |
|
Looks like JsonDataTreeCandidateSerializer does not correctly initialize the writer stack: this is coming from:
public void startListItem(final PathArgument name) throws IOException { final Object schema = getParent(); checkArgument(schema instanceof ListSchemaNode, "List item is not appropriate"); schemaStack.push((ListSchemaNode) schema); } needs some further investigation
|
| Comment by Olivier Dugeon [ 19/Jun/23 ] |
|
Hello Robert, After doing many try, I got a different java error telling that the data tree child is not present in the parent schema while it is: 11:30:21.478 INFO [CommitFutures-5] GraphModel: Edge(r1 - r2) has been published in operational datastore
11:30:21.478 ERROR [opendaylight-cluster-data-notification-dispatcher-38] Failed to process notification DefaultDataTreeCandidate{rootPath=/(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph-topology/graph/graph[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name=LB042}]/edge/edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], rootNode=ChildNode{mod = ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], operation=WRITE, modificationType=WRITE}, oldMeta = SimpleContainerNode{version=org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2045e89c, data=ImmutableMapEntryNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-vertex-id, body=2}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-vertex-id, body=1}, ImmutableContainerNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-attributes, body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)metric, body=50}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-address, body=10.194.77.180}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)adj-sid, body=15000}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)te-metric, body=50}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-address, body=10.194.77.181}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-resv-link-bandwidth, body=176258176.0}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-link-bandwidth, body=176258176.0}]}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id, body=1}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name, body=r1 - r2}]}}, newMeta = SimpleContainerNode{version=org.opendaylight.yangtools.yang.data.tree.impl.node.Version@73d13f2d, data=ImmutableMapEntryNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge[{(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id=1}], body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-vertex-id, body=2}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-vertex-id, body=1}, ImmutableContainerNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-attributes, body=[ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)metric, body=20}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)remote-address, body=10.194.77.180}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)adj-sid, body=15000}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)te-metric, body=50}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)local-address, body=10.194.77.181}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-resv-link-bandwidth, body=176258176.0}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)max-link-bandwidth, body=176258176.0}]}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id, body=1}, ImmutableLeafNode{identifier=(urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)name, body=r1 - r2}]}}}}
java.lang.IllegalArgumentException: Data tree child (urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)edge-id not present in schema parent (urn:opendaylight:params:xml:ns:yang:graph?revision=2022-07-20)graph
at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.notPresent(SchemaInferenceStack.java:965) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.lambda$pushData$6(SchemaInferenceStack.java:771) ~[bundleFile:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.pushData(SchemaInferenceStack.java:771) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.pushData(SchemaInferenceStack.java:765) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.pushData(SchemaInferenceStack.java:759) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.model.util.SchemaInferenceStack.enterDataTree(SchemaInferenceStack.java:473) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.util.NormalizedNodeStreamWriterStack.enterDataTree(NormalizedNodeStreamWriterStack.java:188) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.util.NormalizedNodeStreamWriterStack.startLeafNode(NormalizedNodeStreamWriterStack.java:212) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.codec.gson.JSONNormalizedNodeStreamWriter.startLeafNode(JSONNormalizedNodeStreamWriter.java:319) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessAsSimpleNode(NormalizedNodeWriter.java:143) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:106) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter$OrderedNormalizedNodeWriter.writeMapEntryNode(NormalizedNodeWriter.java:248) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:202) ~[bundleFile:?]
at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:102) ~[bundleFile:?]
at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JsonDataTreeCandidateSerializer.serializeData(JsonDataTreeCandidateSerializer.java:47) ~[bundleFile:?]
at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serializeData(AbstractWebsocketSerializer.java:215) ~[bundleFile:?]
at org.opendaylight.restconf.nb.rfc8040.streams.listeners.AbstractWebsocketSerializer.serialize(AbstractWebsocketSerializer.java:65) ~[bundleFile:?]
at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:70) ~[bundleFile:?]
at org.opendaylight.restconf.nb.rfc8040.streams.listeners.JSONDataTreeCandidateFormatter.createText(JSONDataTreeCandidateFormatter.java:23) ~[bundleFile:?]
at org.opendaylight.restconf.nb.rfc8040.streams.listeners.EventFormatter.eventData(EventFormatter.java:78) ~[bundleFile:?]
at org.opendaylight.restconf.nb.rfc8040.streams.listeners.ListenerAdapter.onDataTreeChanged(ListenerAdapter.java:78) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataTreeChanged(DataTreeChangeListenerActor.java:84) ~[bundleFile:?]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:45) ~[bundleFile:?]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) ~[bundleFile:?]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ~[bundleFile:?]
at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?]
at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[bundleFile:?]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) ~[bundleFile:?]
at akka.actor.ActorCell.invoke(ActorCell.scala:547) ~[bundleFile:?]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[bundleFile:?]
at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[bundleFile:?]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[bundleFile:?]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?]
at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[?:?]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[?:?]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[?:?]
I'm using this path to produce this error: "path": "/graph:graph-topology/graph[name='LB042']/edge[edge-id='1']" which is correctly handle during the subscription and events are correctly send with 'odl-leaf-nodes-only' query parameter |
| Comment by Sangwook Ha [ 20/Jun/23 ] |
|
We bumped into the same issue, so I just submitted a patch for review: https://git.opendaylight.org/gerrit/c/netconf/+/106565 odd22 Can you test if the patch works for you? |
| Comment by Olivier Dugeon [ 21/Jun/23 ] |
|
Hi Sangwook Thanks a lot for your patch. I tested it and it works fine completely resolved this issue (both reported errors). I added my review (+1).
I'll will submit today a new PR on top of yours to add new 'child nodes only' query parameter to get in compact form only modified child and not the complete Data Tree from the root node.
Olivier |