[NETCONF-503] Listener for stream was not found

Fri, 26 Jan 2018 01:56:12 +0000

I am currently integrating OpenStack Pike with ODL Carbon( https://nexus.opendaylight.org/content/repositories/opendaylight-carbon-epel-7-x86_64-devel/org/opendaylight/integration-packaging/opendaylight/6.3.0-0.1.20180108snap438.el7.noarch/opendaylight-6.3.0-0.1.20180108snap438.el7.noarch.rpm).

When creating VM virtual interface, networking-odl through below error:

2018-01-18 11:42:14.174 33 DEBUG networking_odl.common.client [-] Sending METHOD (post) URL (http://192.168.11.20:8087/restconf/operations/sal-remote:create-data-change-event-subscription) JSON ({

"input":

{ "path": "/neutron:neutron/neutron:ports", "sal-remote-augment:notification-output-type": "JSON", "sal-remote-augment:datastore": "OPERATIONAL", "sal-remote-augment:scope": "SUBTREE" }

}) request /usr/lib/python2.7/site-packages/networking_odl/common/client.py:89

2018-01-18 11:42:14.186 33 DEBUG networking_odl.common.client [-] Got response:

({"output":{"stream-name":"data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE"}}) _check_response /usr/lib/python2.7/site-packages/networking_odl/common/client.py:143

2018-01-18 11:42:14.187 33 DEBUG networking_odl.common.client [-] Sending METHOD (get) URL (http://192.168.11.20:8087/restconf/streams/stream/data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE?odl-leaf-nodes-only=true) JSON (None) request /usr/lib/python2.7/site-packages/networking_odl/common/client.py:89

2018-01-18 11:42:14.193 33 DEBUG networking_odl.common.websocket_client [-] websocket stream URL: ws://192.168.11.20:8185/data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE _subscribe_websocket /usr/lib/python2.7/site-packages/networking_odl/common/websocket_client.py:210

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client [-] webSocket bad status exception: WebSocketBadStatusException: Handshake status 500

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client Traceback (most recent call last):

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/networking_odl/common/websocket_client.py", line 238, in _socket_create_connection

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client timeout=self.timeout)

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 104, in create_connection

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client websock.connect(url, **options)

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 269, in connect

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client self.handshake_response = handshake(self.sock, *addrs, **options)

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 67, in handshake

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client status, resp = _get_resp_headers(sock)

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 123, in _get_resp_headers

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client raise WebSocketBadStatusException("Handshake status %d", status)

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client WebSocketBadStatusException: Handshake status 500

2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client

And there are a lot of Listener for stream was not found error logs in ODL like below:

2018-01-18 03:51:44,998 | ERROR | tLoopGroup-11-70 | WebSocketServerHandler | 280 - org.opendaylight.netconf.sal-rest-connector - 1.5.3.SNAPSHOT | Listener for stream with name 'data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE' was not found.

It is a intermittent issue. The environment is a five-nodes OpenStack+ODL cluster which is deployed on a POD for OPNFV production CI(run every day, the latest failure is at http://artifacts.opnfv.org/logs/functest/zte-pod3/2018-01-23_21-02-30/functest.log). Three nodes of them act as controllers for both ODL and OpenStack. The other two nodes are act as compute nodes. All the five nodes have br-ex bridge that connect to physnet. The OVSs actively connect to each ODL controller.

I have collected all the three controller node's config files and logs at https://github.com/huzhijiang/odldebug0119

OpenDaylight JIRA

[NETCONF-503] Listener for stream was not found