-
Bug
-
Resolution: Won't Do
-
Medium
-
None
-
None
-
None
-
The environment is a five-nodes OpenStack+ODL cluster which is deployed on a POD for OPNFV production CI(run every day, the latest failure is at http://artifacts.opnfv.org/logs/functest/zte-pod3/2018-01-23_21-02-30/functest.log). Three nodes of them act as controllers for both ODL and OpenStack. The other two nodes are act as compute nodes. All the five nodes have br-ex bridge that connect to physnet. The OVSs actively connect to each ODL controller.
I have collected all the three controller node's config files and logs at https://github.com/huzhijiang/odldebug0119
The environment is a five-nodes OpenStack+ODL cluster which is deployed on a POD for OPNFV production CI(run every day, the latest failure is at http://artifacts.opnfv.org/logs/functest/zte-pod3/2018-01-23_21-02-30/functest.log ). Three nodes of them act as controllers for both ODL and OpenStack. The other two nodes are act as compute nodes. All the five nodes have br-ex bridge that connect to physnet. The OVSs actively connect to each ODL controller. I have collected all the three controller node's config files and logs at https://github.com/huzhijiang/odldebug0119
I am currently integrating OpenStack Pike with ODL Carbon( https://nexus.opendaylight.org/content/repositories/opendaylight-carbon-epel-7-x86_64-devel/org/opendaylight/integration-packaging/opendaylight/6.3.0-0.1.20180108snap438.el7.noarch/opendaylight-6.3.0-0.1.20180108snap438.el7.noarch.rpm).
When creating VM virtual interface, networking-odl through below error:
2018-01-18 11:42:14.174 33 DEBUG networking_odl.common.client [-] Sending METHOD (post) URL (http://192.168.11.20:8087/restconf/operations/sal-remote:create-data-change-event-subscription) JSON ({
"input":
{ "path": "/neutron:neutron/neutron:ports", "sal-remote-augment:notification-output-type": "JSON", "sal-remote-augment:datastore": "OPERATIONAL", "sal-remote-augment:scope": "SUBTREE" }}) request /usr/lib/python2.7/site-packages/networking_odl/common/client.py:89
2018-01-18 11:42:14.186 33 DEBUG networking_odl.common.client [-] Got response:
({"output":{"stream-name":"data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE"}}) _check_response /usr/lib/python2.7/site-packages/networking_odl/common/client.py:143
2018-01-18 11:42:14.187 33 DEBUG networking_odl.common.client [-] Sending METHOD (get) URL (http://192.168.11.20:8087/restconf/streams/stream/data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE?odl-leaf-nodes-only=true) JSON (None) request /usr/lib/python2.7/site-packages/networking_odl/common/client.py:89
2018-01-18 11:42:14.193 33 DEBUG networking_odl.common.websocket_client [-] websocket stream URL: ws://192.168.11.20:8185/data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE _subscribe_websocket /usr/lib/python2.7/site-packages/networking_odl/common/websocket_client.py:210
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client [-] webSocket bad status exception: WebSocketBadStatusException: Handshake status 500
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client Traceback (most recent call last):
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/networking_odl/common/websocket_client.py", line 238, in _socket_create_connection
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client timeout=self.timeout)
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 104, in create_connection
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client websock.connect(url, **options)
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 269, in connect
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client self.handshake_response = handshake(self.sock, *addrs, **options)
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 67, in handshake
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client status, resp = _get_resp_headers(sock)
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 123, in _get_resp_headers
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client raise WebSocketBadStatusException("Handshake status %d", status)
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client WebSocketBadStatusException: Handshake status 500
2018-01-18 11:42:14.196 33 ERROR networking_odl.common.websocket_client
And there are a lot of Listener for stream was not found error logs in ODL like below:
2018-01-18 03:51:44,998 | ERROR | tLoopGroup-11-70 | WebSocketServerHandler | 280 - org.opendaylight.netconf.sal-rest-connector - 1.5.3.SNAPSHOT | Listener for stream with name 'data-change-event-subscription/neutron:neutron/neutron:ports/datastore=OPERATIONAL/scope=SUBTREE' was not found.
It is a intermittent issue. The environment is a five-nodes OpenStack+ODL cluster which is deployed on a POD for OPNFV production CI(run every day, the latest failure is at http://artifacts.opnfv.org/logs/functest/zte-pod3/2018-01-23_21-02-30/functest.log). Three nodes of them act as controllers for both ODL and OpenStack. The other two nodes are act as compute nodes. All the five nodes have br-ex bridge that connect to physnet. The OVSs actively connect to each ODL controller.
I have collected all the three controller node's config files and logs at https://github.com/huzhijiang/odldebug0119
