Uploaded image for project: 'OpenFlowPlugin'
  1. OpenFlowPlugin
  2. OPNFLWPLUG-1007

Future (eventually) failed: addFlow -and- transmitPacket

    XMLWordPrintable

Details

    Description

      It's not clear (yet) what kind of problem (if any) this Exception poses, but the netvirt
      clustering CSIT jobs are unstable. The following Exception stood out as something
      scary. Potentially, missing flows could be one reason some of the openstack
      instances do not have proper connectivity and test cases fail.

       

      2018-04-20T06:25:17,932 | ERROR | opendaylight-cluster-data-notification-dispatcher-94 | DefaultConfigPusher              | 374 - org.opendaylight.openflowplugin.applications.of-switch-config-pusher - 0.6.1 | Future (eventually) failed: addFlow
org.opendaylight.controller.md.sal.dom.api.DOMRpcImplementationNotAvailableException: No implementation of RPC AbsoluteSchemaPath{path=[(urn:opendaylight:module:config?revision=2014-10-15)set-config]} available
        at org.opendaylight.controller.md.sal.dom.broker.impl.RoutedDOMRpcRoutingTableEntry.invokeRpc(RoutedDOMRpcRoutingTableEntry.java:85) [218:org.opendaylight.controller.sal-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRoutingTable.invokeRpc(DOMRpcRoutingTable.java:178) [218:org.opendaylight.controller.sal-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRouter.invokeRpc(DOMRpcRouter.java:102) [218:org.opendaylight.controller.sal-broker-impl:1.7.1]
        at Proxyaec100e4_e760_48b8_b4f7_2f391b651b90.invokeRpc(Unknown Source) [?:?]
        at Proxyc11c9ab5_3bc0_4a35_92fb_52a7d17ea7ca.invokeRpc(Unknown Source) [?:?]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke0(RpcServiceAdapter.java:68) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.access$000(RpcServiceAdapter.java:46) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter$RpcInvocationStrategy.invoke(RpcServiceAdapter.java:165) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke(RpcServiceAdapter.java:99) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at com.sun.proxy.$Proxy97.setConfig(Unknown Source) [386:org.opendaylight.openflowplugin.model.flow-service:0.6.1]
        at org.opendaylight.openflowplugin.openflow.ofswitch.config.DefaultConfigPusher.onDataTreeChanged(DefaultConfigPusher.java:84) [374:org.opendaylight.openflowplugin.applications.of-switch-config-pusher:0.6.1]
        at org.opendaylight.controller.md.sal.binding.impl.BindingDOMDataTreeChangeListenerAdapter.onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataChanged(DataTreeChangeListenerActor.java:67) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1]
        at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:41) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1]
        at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:38) [221:org.opendaylight.controller.sal-clustering-commons:1.7.1]
        at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.Actor.aroundReceive(Actor.scala:517) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.Actor.aroundReceive$(Actor.scala:515) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:590) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.ActorCell.invoke(ActorCell.scala:559) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.Mailbox.run(Mailbox.scala:224) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [42:com.typesafe.akka.actor:2.5.11]
2018-04-20T06:25:17,933 | INFO  | ForkJoinPool-1-worker-7 | FlowBasedEgressServicesStateBindHelper | 252 - org.opendaylight.genius.interfacemanager-impl - 0.4.1 | bindServicesOnInterfaceType Egress - WIP
2018-04-20T06:25:17,934 | ERROR | opendaylight-cluster-data-notification-dispatcher-103 | LLDPSpeaker                      | 373 - org.opendaylight.openflowplugin.applications.lldp-speaker - 0.6.1 | Future (eventually) failed: transmitPacket
org.opendaylight.controller.md.sal.dom.api.DOMRpcImplementationNotAvailableException: Action Local{type=AbsoluteSchemaPath{path=[(urn:opendaylight:packet:service?revision=2013-07-09)transmit-packet]}, contextReference=/(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:275486970454348}]} has no instance matching org.opendaylight.controller.md.sal.binding.impl.LazySerializedContainerNode$WithContextRef@4a0d2b2
        at org.opendaylight.controller.blueprint.ext.ActionProviderBean.lambda$registerFallback$0(ActionProviderBean.java:133) ~[195:org.opendaylight.controller.blueprint:0.8.1]
        at org.opendaylight.controller.md.sal.dom.broker.impl.RoutedDOMRpcRoutingTableEntry.invokeRpc(RoutedDOMRpcRoutingTableEntry.java:71) [218:org.opendaylight.controller.sal-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRoutingTable.invokeRpc(DOMRpcRoutingTable.java:178) [218:org.opendaylight.controller.sal-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRouter.invokeRpc(DOMRpcRouter.java:102) [218:org.opendaylight.controller.sal-broker-impl:1.7.1]
        at Proxyaec100e4_e760_48b8_b4f7_2f391b651b90.invokeRpc(Unknown Source) [?:?]
        at Proxyc11c9ab5_3bc0_4a35_92fb_52a7d17ea7ca.invokeRpc(Unknown Source) [?:?]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke0(RpcServiceAdapter.java:68) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.access$000(RpcServiceAdapter.java:46) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter$RpcInvocationStrategy.invoke(RpcServiceAdapter.java:165) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke(RpcServiceAdapter.java:99) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at com.sun.proxy.$Proxy133.transmitPacket(Unknown Source) [386:org.opendaylight.openflowplugin.model.flow-service:0.6.1]
        at org.opendaylight.openflowplugin.applications.lldpspeaker.LLDPSpeaker.nodeConnectorAdded(LLDPSpeaker.java:190) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1]
        at org.opendaylight.openflowplugin.applications.lldpspeaker.NodeConnectorInventoryEventTranslator.notifyNodeConnectorAppeared(NodeConnectorInventoryEventTranslator.java:179) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1]
        at org.opendaylight.openflowplugin.applications.lldpspeaker.NodeConnectorInventoryEventTranslator.processAddedConnector(NodeConnectorInventoryEventTranslator.java:124) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1]
        at org.opendaylight.openflowplugin.applications.lldpspeaker.NodeConnectorInventoryEventTranslator.onDataTreeChanged(NodeConnectorInventoryEventTranslator.java:102) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1]
        at org.opendaylight.controller.md.sal.binding.impl.BindingDOMDataTreeChangeListenerAdapter.onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1]
        at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataChanged(DataTreeChangeListenerActor.java:67) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1]
        at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:41) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1]
        at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:38) [221:org.opendaylight.controller.sal-clustering-commons:1.7.1]
        at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.Actor.aroundReceive(Actor.scala:517) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.Actor.aroundReceive$(Actor.scala:515) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:590) [42:com.typesafe.akka.actor:2.5.11]
        at akka.actor.ActorCell.invoke(ActorCell.scala:559) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.Mailbox.run(Mailbox.scala:224) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [42:com.typesafe.akka.actor:2.5.11]
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [42:com.typesafe.akka.actor:2.5.11]

      The above trace was taken from this karaf log from this job. The other two ODL karaf logs are here and here.

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. CONTROLLER-1788 Failed to execute RPC (urn:opendaylight:packet:service?revision=2013-07-09)transmit-packet due to org.opendaylight.controller.md.sal.dom.api.DOMRpcImplementationNotAvailableException

          • Medium - Has the potential to affect progress.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. OPNFLWPLUG-1024 Break out the odl-openflowplugin-app-topology feature.

          • Medium - Has the potential to affect progress.
          • Resolved

          Sub-task - The sub-task of the issue NETVIRT-1031 Review controller-max-backoff and controller-inactivity-probe values

          • Medium - Has the potential to affect progress.
          • Resolved
          # Subject Branch Project Status CR V
          51763,8 Set ovsdb inactivity_timeout to 30s master integration/test Status: MERGED +2 +1
          73868,10 OPNFLWPLUG-1007 : Future (eventually) failed: addFlow -and- transmitPacket master openflowplugin Status: MERGED +2 +1
          73898,5 Set ovsdb monitor manager status to false master integration/test Status: MERGED +2 +1
          74786,2 OPNFLWPLUG-1007 : Future (eventually) failed: addFlow -and- transmitPacket stable/oxygen openflowplugin Status: MERGED +2 +1

          Activity

            People

              gobinath Gobinath Suganthan
              jluhrsen Jamo Luhrsen
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: