Uploaded image for project: 'netvirt'
  1. netvirt
  2. NETVIRT-377

openflow connection broken in 3 node CSIT cluster setup

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Resolution: Done
    • Boron
    • None
    • None
    • None
    • Operating System: All
      Platform: All

    • 7398

    Description

      In below Run cluster related cases are failed and VM's not getting IP address.
      In openstack node observed ODL3 controller connection is not established properly.

      https://logs.opendaylight.org/releng/jenkins092/netvirt-legacy-csit-3node-openstack-mitaka-upstream-boron/39/archives/log.html.gz#s1-s3-s2

      Manager "tcp:10.29.12.207:6640"
      is_connected: true
      Manager "tcp:10.29.13.140:6640"
      is_connected: true
      Manager "tcp:10.29.12.15:6640"
      is_connected: true
      Bridge br-int
      Controller "tcp:10.29.13.140:6653"
      Controller "tcp:10.29.12.15:6653"
      is_connected: true
      Controller "tcp:10.29.12.207:6653"
      is_connected: true
      fail_mode: secure
      Port patch-ext
      Interface patch-ext
      type: patch
      options:

      {peer=patch-int}

      controller "10.29.13.140" port 6640 connection status is true but 6653 connection status is not displayed.

      Next Run all the Test cases are passed this issue is inconsistent.

      There are many ERROR in ODL3 controller logs.
      uploaded all controller logs.

      2016-12-13 20:02:11,373 | INFO | entLoopGroup-7-7 | ConnectionAdapterImpl | 296 - org.opendaylight.openflowjava.openflow-protocol-impl - 0.8.2.SNAPSHOT | Hello received / branch
      2016-12-13 20:02:11,378 | WARN | lt-dispatcher-16 | ReliableDeliverySupervisor | 199 - com.typesafe.akka.slf4j - 2.4.7 | Association with remote system [akka.tcp://opendaylight-cluster-data@10.29.12.207:2550] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://opendaylight-cluster-data@10.29.12.207:2550]] Caused by: [Connection refused: /10.29.12.207:2550]
      2016-12-13 20:02:11,380 | ERROR | on-dispatcher-80 | OutboundQueueProviderImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | No queue present, failing request
      2016-12-13 20:02:11,381 | INFO | entLoopGroup-7-7 | SystemNotificationsListenerImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | ConnectionEvent: Connection closed by device, Device:/10.29.13.11:50964, NodeId:null
      2016-12-13 20:02:11,387 | ERROR | on-dispatcher-82 | OutboundQueueProviderImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | No queue present, failing request
      2016-12-13 20:02:11,387 | WARN | on-dispatcher-82 | RpcContextImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Xid cannot be reserved for new RequestContext, node:openflow:119508379114414
      2016-12-13 20:02:11,395 | ERROR | on-dispatcher-69 | OutboundQueueProviderImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | No queue present, failing request
      2016-12-13 20:02:11,395 | WARN | on-dispatcher-69 | RpcContextImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Xid cannot be reserved for new RequestContext, node:openflow:270479877144834
      2016-12-13 20:02:11,396 | WARN | on-dispatcher-80 | RpcContextImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Xid cannot be reserved for new RequestContext, node:openflow:119508379114414
      2016-12-13 20:02:11,397 | ERROR | on-dispatcher-80 | OutboundQueueProviderImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | No queue present, failing request
      2016-12-13 20:02:11,397 | WARN | on-dispatcher-80 | RpcContextImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Xid cannot be reserved for new RequestContext, node:openflow:119508379114414
      2016-12-13 20:02:11,405 | WARN | entLoopGroup-7-8 | StatisticsGatheringUtils | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Can't write to transaction, transaction chain probably closed.
      2016-12-13 20:02:11,405 | WARN | entLoopGroup-7-5 | StatisticsGatheringUtils | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Can't write to transaction, transaction chain probably closed.
      2016-12-13 20:02:11,406 | WARN | entLoopGroup-7-8 | StatisticsManagerImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Statistics gathering for single node openflow:270479877144834 was not successful: Device connection is closed for Node : Uri [_value=openflow:270479877144834].
      2016-12-13 20:02:11,406 | WARN | entLoopGroup-7-5 | StatisticsManagerImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Statistics gathering for single node openflow:167998386060623 was not successful: Device connection is closed for Node : Uri [_value=openflow:167998386060623].
      2016-12-13 20:02:11,406 | INFO | entLoopGroup-7-5 | ConnectionAdapterImpl | 296 - org.opendaylight.openflowjava.openflow-protocol-impl - 0.8.2.SNAPSHOT | Hello received / branch
      2016-12-13 20:02:11,406 | INFO | entLoopGroup-7-8 | ConnectionAdapterImpl | 296 - org.opendaylight.openflowjava.openflow-protocol-impl - 0.8.2.SNAPSHOT | Hello received / branch
      2016-12-13 20:02:11,410 | ERROR | on-dispatcher-68 | OutboundQueueProviderImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | No queue present, failing request
      2016-12-13 20:02:11,410 | WARN | on-dispatcher-68 | RpcContextImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Xid cannot be reserved for new RequestContext, node:openflow:167998386060623
      2016-12-13 20:02:11,420 | WARN | entLoopGroup-7-3 | StatisticsGatheringUtils | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Can't write to transaction, transaction chain probably closed.
      2016-12-13 20:02:11,420 | WARN | entLoopGroup-7-3 | StatisticsManagerImpl | 301 - org.opendaylight.openflowplugin.impl - 0.3.2.SNAPSHOT | Statistics gathering for single node openflow:119508379114414 was not successful: Device connection is closed for Node : Uri [_value=openflow:119508379114414].

      Attachments

        1. example2.tar.gz
          564 kB
          bertrand.low@hcl.com
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Unassigned Unassigned
            balakrishnan balakrishnan k
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: