Uploaded image for project: 'genius'
  1. genius
  2. GENIUS-166

Genius CSIT Intermittent RESTCONF ReadTimeOut Errors for POST/DELETE requests

XMLWordPrintable

    • High

      Genius Fluorine CSIT is randomly hitting some RESTCONF ReadTimeOuts on some of the DELETE/POST requests recently.

      https://jenkins.opendaylight.org/sandbox/job/jamo-genius-csit-1node-gate-all-fluorine/

      Documentation: Send a DELETE request on the session object found using the
      Start / End / Elapsed: 20180531 04:34:26.956 / 20180531 04:34:31.569 / 00:00:04.613
      04:34:27.959 WARN Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='10.30.170.29', port=8181): Read timed out. (read timeout=1.0)",)': /restconf/config/itm:transport-zones/transport-zone/TZA/  
      04:34:29.162 WARN Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='10.30.170.29', port=8181): Read timed out. (read timeout=1.0)",)': /restconf/config/itm:transport-zones/transport-zone/TZA/  
      04:34:30.565 WARN Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='10.30.170.29', port=8181): Read timed out. (read timeout=1.0)",)': /restconf/config/itm:transport-zones/transport-zone/TZA/  
      04:34:31.568 FAIL ConnectionError: HTTPConnectionPool(host='10.30.170.29', port=8181): Max retries exceeded with url: /restconf/config/itm:transport-zones/transport-zone/TZA/ (Caused by ReadTimeoutError("HTTPConnectionPool(host='10.30.170.29', port=8181): Read timed out. (read timeout=1.0)",))

      We see the below exception in karaf log on all failing runs, but not sure whether that is the reason for the failure though:

           2018-05-30T15:08:06,198 | WARN  | opendaylight-cluster-data-shard-dispatcher-88 | ShardDataTree 240 - org.opendaylight.controller.sal-distributed-datastore -| 1.8.0.SNAPSHOT | member-1-shard-default-config: Current transaction member-1-datastore-config-fe-0-txn-1477-0 has timed out after 19233 ms in state CAN_COMMIT_COMPLETE

           2018-05-30T15:08:06,198 | WARN  | opendaylight-cluster-data-shard-dispatcher-65 | ShardDataTree | 240 - org.opendaylight.controller.sal-distributed-datastore - 1.8.0.SNAPSHOT | member-1-shard-inventory-config: Current transaction member-1-datastore-config-fe-0-txn-1478-0 has timed out after 19234 ms in state READY

           2018-05-30T15:08:06,199 | ERROR | opendaylight-cluster-data-shard-dispatcher-88 | Shard | 232 - org.opendaylight.controller.sal-clustering-commons - | 1.8.0.SNAPSHOT | member-1-shard-inventory-config: Cannot canCommit transaction member-1-datastore-config-fe-0-txn-1478-0 - no cohort entry found

      2018-05-30T15:08:06,199 | ERROR | opendaylight-cluster-data-shard-dispatcher-65 | Shard                            | 232 - org.opendaylight.controller.sal-clustering-commons - 1.8.0.SNAPSHOT | member-1-shard-default-config: Cannot commit transaction member-1-datastore-config-fe-0-txn-1477-0 - no cohort entry found

            tpantelis Tom Pantelis
            k.faseela Faseela K
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: