[NETCONF-689] Extend Websocket streams for data-less notifications Created: 22/May/20  Updated: 26/Oct/21  Resolved: 30/Sep/20

Status: Resolved
Project: netconf
Component/s: restconf-nb
Affects Version/s: None
Fix Version/s: 1.13.0

Type: Improvement Priority: Medium
Reporter: Nikhil Soni Assignee: Nikhil Soni
Resolution: Done Votes: 0
Labels: restconf
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to NETCONF-823 Announce 'odl-skip-notification-data'... Resolved

 Description   

In one of our customers, the use case was to listen to changes in a particular yangtree. however the data underneath was quite huge, thereby causing the heap running over and the controller crashing in response to lot of changes in the subtree. Hence, we had to fallback to a mechanism to detect just a change in the yangtree from a northbound application and on detection of the change, do a bulk get of the data. that was more effective.

 

In supporting this usecase, the current implementation needed an extension. Currently,  WebSocket stream notification contains notification metadata with data.  However, there is no way to avoid collecting data during notification processing in the websocket server. Hence, we decided to add  a new filter named "odl-skip-notification-data", similar to "odl-leaf-nodes-only" in the subscription API.

 

1. Create Stream:

POST http://\controller-ip:8181/restconf/operations/sal-remote:create-data-change-event-subscription
{
  "input": {
    "path": "/network-topology:network-topology",
    "sal-remote-augment:datastore": "CONFIGURATION",
    "sal-remote-augment:scope": "SUBTREE",
    "sal-remote-augment:notification-output-type": "JSON"
  }
}

Response:
{
  "output": {
    "stream-name": "data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE"
  }
}

 

2. Subscribe:

GET http://{{controller-ip}}:8181/restconf/streams/stream/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE?&odl-skip-notification-data=true

Response:
{
    "location": "ws://localhost:8185/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE"
}

3. Get Notification:

GET  ws://localhost:8185/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE

4. Write to topology or any respective data tree:

5. Websocket JSON Response -

{
   "notification":{
      "xmlns":"urn:ietf:params:xml:ns:netconf:notification:1.0",
      "data-changed-notification":{
         "xmlns":"urn:opendaylight:params:xml:ns:yang:controller:md:sal:remote",
         "data-change-event":{
            "path":"/network-topology:network-topology",
            "operation":"updated"
         }
      },
      "eventTime":"2020-05-05T20:22:23.293+05:30"
   }
}


 Comments   
Comment by Tejas Nevrekar [ 29/May/20 ]

Further to the description, we looked at the RFC to see if there is any standard way. But it does not currently define a way to only notify with no data. However, it does allow a query param to be vendor specific - https://tools.ietf.org/html/rfc8040#section-4.8.

If vendors define additional query parameters, they SHOULD use a
 prefix (such as the enterprise or organization name) for query
 parameter names in order to avoid collisions with other parameters.
Comment by Jamo Luhrsen [ 05/Jun/20 ]

rovarga, tcere, could you comment on this use case and take a look at the review submitted please?

Comment by Robert Varga [ 08/Jun/20 ]

"however the data underneath was quite huge, thereby causing the heap running over and the controller crashing in response to lot of changes" – this looks like the core of the issue, shouldn't that be fixed?

Note that the proposed notify-get mechanics is subject to races and it is not explained anywhere.

Comment by Tejas Nevrekar [ 23/Jul/20 ]

rovarga I agree that the ideal solution is to fix the heap running over. But this patch at least gives a reasonable workaround for consumers to continue to use websockets in such a loaded system. Also I concur we need to open a ticket to fully fix the issue. nikhil.soni.lumina can you add that with the appropriate summary and link it here so that it mentions the details of the overrun that we observe and in what conditions?

Comment by Nikhil Soni [ 28/Jul/20 ]

rovarga
Usecase - If a client what to listen via websocket on any changes happens in the leaf, an the leaf is inside List which contains hundreds leaf object.
If any change happens in the list then websocket returns all list data instead of one particular changed leaf which can be huge data.

Topology Example data -
{
"topology": [
{
"topology-id": "1",
"default": "true",
"topology-name": "1",
"node": [

{ id:1}

.....thousands of nodes
]
}
in above example if any change happens in one node which is inside in "node" list then all thousands of node with data come in websocket which creates heap memory issue in client side.

Generated at Wed Feb 07 20:15:39 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.