[NETCONF-689] Extend Websocket streams for data-less notifications Created: 22/May/20 Updated: 26/Oct/21 Resolved: 30/Sep/20 |
|
| Status: | Resolved |
| Project: | netconf |
| Component/s: | restconf-nb |
| Affects Version/s: | None |
| Fix Version/s: | 1.13.0 |
| Type: | Improvement | Priority: | Medium |
| Reporter: | Nikhil Soni | Assignee: | Nikhil Soni |
| Resolution: | Done | Votes: | 0 |
| Labels: | restconf | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
In one of our customers, the use case was to listen to changes in a particular yangtree. however the data underneath was quite huge, thereby causing the heap running over and the controller crashing in response to lot of changes in the subtree. Hence, we had to fallback to a mechanism to detect just a change in the yangtree from a northbound application and on detection of the change, do a bulk get of the data. that was more effective.
In supporting this usecase, the current implementation needed an extension. Currently, WebSocket stream notification contains notification metadata with data. However, there is no way to avoid collecting data during notification processing in the websocket server. Hence, we decided to add a new filter named "odl-skip-notification-data", similar to "odl-leaf-nodes-only" in the subscription API.
1. Create Stream: POST http://\controller-ip:8181/restconf/operations/sal-remote:create-data-change-event-subscription
{
"input": {
"path": "/network-topology:network-topology",
"sal-remote-augment:datastore": "CONFIGURATION",
"sal-remote-augment:scope": "SUBTREE",
"sal-remote-augment:notification-output-type": "JSON"
}
}
Response:
{
"output": {
"stream-name": "data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE"
}
}
2. Subscribe: GET http://{{controller-ip}}:8181/restconf/streams/stream/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE?&odl-skip-notification-data=true
Response:
{
"location": "ws://localhost:8185/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE"
}
3. Get Notification: GET ws://localhost:8185/data-change-event-subscription/network-topology:network-topology/datastore=CONFIGURATION/scope=SUBTREE 4. Write to topology or any respective data tree: 5. Websocket JSON Response - {
"notification":{
"xmlns":"urn:ietf:params:xml:ns:netconf:notification:1.0",
"data-changed-notification":{
"xmlns":"urn:opendaylight:params:xml:ns:yang:controller:md:sal:remote",
"data-change-event":{
"path":"/network-topology:network-topology",
"operation":"updated"
}
},
"eventTime":"2020-05-05T20:22:23.293+05:30"
}
}
|
| Comments |
| Comment by Tejas Nevrekar [ 29/May/20 ] |
|
Further to the description, we looked at the RFC to see if there is any standard way. But it does not currently define a way to only notify with no data. However, it does allow a query param to be vendor specific - https://tools.ietf.org/html/rfc8040#section-4.8. If vendors define additional query parameters, they SHOULD use a prefix (such as the enterprise or organization name) for query parameter names in order to avoid collisions with other parameters. |
| Comment by Jamo Luhrsen [ 05/Jun/20 ] |
|
rovarga, tcere, could you comment on this use case and take a look at the review submitted please? |
| Comment by Robert Varga [ 08/Jun/20 ] |
|
"however the data underneath was quite huge, thereby causing the heap running over and the controller crashing in response to lot of changes" – this looks like the core of the issue, shouldn't that be fixed? Note that the proposed notify-get mechanics is subject to races and it is not explained anywhere. |
| Comment by Tejas Nevrekar [ 23/Jul/20 ] |
|
rovarga I agree that the ideal solution is to fix the heap running over. But this patch at least gives a reasonable workaround for consumers to continue to use websockets in such a loaded system. Also I concur we need to open a ticket to fully fix the issue. nikhil.soni.lumina can you add that with the appropriate summary and link it here so that it mentions the details of the overrun that we observe and in what conditions? |
| Comment by Nikhil Soni [ 28/Jul/20 ] |
|
rovarga Topology Example data - .....thousands of nodes |