-
New Feature
-
Resolution: Done
-
High
-
None
It's not clear (yet) what kind of problem (if any) this Exception poses, but the netvirt
clustering CSIT jobs are unstable. The following Exception stood out as something
scary. Potentially, missing flows could be one reason some of the openstack
instances do not have proper connectivity and test cases fail.
2018-04-20T06:25:17,932 | ERROR | opendaylight-cluster-data-notification-dispatcher-94 | DefaultConfigPusher | 374 - org.opendaylight.openflowplugin.applications.of-switch-config-pusher - 0.6.1 | Future (eventually) failed: addFlow org.opendaylight.controller.md.sal.dom.api.DOMRpcImplementationNotAvailableException: No implementation of RPC AbsoluteSchemaPath{path=[(urn:opendaylight:module:config?revision=2014-10-15)set-config]} available at org.opendaylight.controller.md.sal.dom.broker.impl.RoutedDOMRpcRoutingTableEntry.invokeRpc(RoutedDOMRpcRoutingTableEntry.java:85) [218:org.opendaylight.controller.sal-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRoutingTable.invokeRpc(DOMRpcRoutingTable.java:178) [218:org.opendaylight.controller.sal-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRouter.invokeRpc(DOMRpcRouter.java:102) [218:org.opendaylight.controller.sal-broker-impl:1.7.1] at Proxyaec100e4_e760_48b8_b4f7_2f391b651b90.invokeRpc(Unknown Source) [?:?] at Proxyc11c9ab5_3bc0_4a35_92fb_52a7d17ea7ca.invokeRpc(Unknown Source) [?:?] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke0(RpcServiceAdapter.java:68) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.access$000(RpcServiceAdapter.java:46) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter$RpcInvocationStrategy.invoke(RpcServiceAdapter.java:165) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke(RpcServiceAdapter.java:99) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at com.sun.proxy.$Proxy97.setConfig(Unknown Source) [386:org.opendaylight.openflowplugin.model.flow-service:0.6.1] at org.opendaylight.openflowplugin.openflow.ofswitch.config.DefaultConfigPusher.onDataTreeChanged(DefaultConfigPusher.java:84) [374:org.opendaylight.openflowplugin.applications.of-switch-config-pusher:0.6.1] at org.opendaylight.controller.md.sal.binding.impl.BindingDOMDataTreeChangeListenerAdapter.onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataChanged(DataTreeChangeListenerActor.java:67) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1] at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:41) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1] at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:38) [221:org.opendaylight.controller.sal-clustering-commons:1.7.1] at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.Actor.aroundReceive(Actor.scala:517) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.Actor.aroundReceive$(Actor.scala:515) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:590) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.ActorCell.invoke(ActorCell.scala:559) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.Mailbox.run(Mailbox.scala:224) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [42:com.typesafe.akka.actor:2.5.11] 2018-04-20T06:25:17,933 | INFO | ForkJoinPool-1-worker-7 | FlowBasedEgressServicesStateBindHelper | 252 - org.opendaylight.genius.interfacemanager-impl - 0.4.1 | bindServicesOnInterfaceType Egress - WIP 2018-04-20T06:25:17,934 | ERROR | opendaylight-cluster-data-notification-dispatcher-103 | LLDPSpeaker | 373 - org.opendaylight.openflowplugin.applications.lldp-speaker - 0.6.1 | Future (eventually) failed: transmitPacket org.opendaylight.controller.md.sal.dom.api.DOMRpcImplementationNotAvailableException: Action Local{type=AbsoluteSchemaPath{path=[(urn:opendaylight:packet:service?revision=2013-07-09)transmit-packet]}, contextReference=/(urn:opendaylight:inventory?revision=2013-08-19)nodes/node/node[{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:275486970454348}]} has no instance matching org.opendaylight.controller.md.sal.binding.impl.LazySerializedContainerNode$WithContextRef@4a0d2b2 at org.opendaylight.controller.blueprint.ext.ActionProviderBean.lambda$registerFallback$0(ActionProviderBean.java:133) ~[195:org.opendaylight.controller.blueprint:0.8.1] at org.opendaylight.controller.md.sal.dom.broker.impl.RoutedDOMRpcRoutingTableEntry.invokeRpc(RoutedDOMRpcRoutingTableEntry.java:71) [218:org.opendaylight.controller.sal-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRoutingTable.invokeRpc(DOMRpcRoutingTable.java:178) [218:org.opendaylight.controller.sal-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRouter.invokeRpc(DOMRpcRouter.java:102) [218:org.opendaylight.controller.sal-broker-impl:1.7.1] at Proxyaec100e4_e760_48b8_b4f7_2f391b651b90.invokeRpc(Unknown Source) [?:?] at Proxyc11c9ab5_3bc0_4a35_92fb_52a7d17ea7ca.invokeRpc(Unknown Source) [?:?] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke0(RpcServiceAdapter.java:68) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.access$000(RpcServiceAdapter.java:46) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter$RpcInvocationStrategy.invoke(RpcServiceAdapter.java:165) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke(RpcServiceAdapter.java:99) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at com.sun.proxy.$Proxy133.transmitPacket(Unknown Source) [386:org.opendaylight.openflowplugin.model.flow-service:0.6.1] at org.opendaylight.openflowplugin.applications.lldpspeaker.LLDPSpeaker.nodeConnectorAdded(LLDPSpeaker.java:190) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1] at org.opendaylight.openflowplugin.applications.lldpspeaker.NodeConnectorInventoryEventTranslator.notifyNodeConnectorAppeared(NodeConnectorInventoryEventTranslator.java:179) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1] at org.opendaylight.openflowplugin.applications.lldpspeaker.NodeConnectorInventoryEventTranslator.processAddedConnector(NodeConnectorInventoryEventTranslator.java:124) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1] at org.opendaylight.openflowplugin.applications.lldpspeaker.NodeConnectorInventoryEventTranslator.onDataTreeChanged(NodeConnectorInventoryEventTranslator.java:102) [373:org.opendaylight.openflowplugin.applications.lldp-speaker:0.6.1] at org.opendaylight.controller.md.sal.binding.impl.BindingDOMDataTreeChangeListenerAdapter.onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41) [215:org.opendaylight.controller.sal-binding-broker-impl:1.7.1] at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataChanged(DataTreeChangeListenerActor.java:67) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1] at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:41) [228:org.opendaylight.controller.sal-distributed-datastore:1.7.1] at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:38) [221:org.opendaylight.controller.sal-clustering-commons:1.7.1] at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.Actor.aroundReceive(Actor.scala:517) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.Actor.aroundReceive$(Actor.scala:515) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:590) [42:com.typesafe.akka.actor:2.5.11] at akka.actor.ActorCell.invoke(ActorCell.scala:559) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.Mailbox.run(Mailbox.scala:224) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [42:com.typesafe.akka.actor:2.5.11]
The above trace was taken from this karaf log from this job. The other two ODL karaf logs are here and here.
- duplicates
-
CONTROLLER-1788 Failed to execute RPC (urn:opendaylight:packet:service?revision=2013-07-09)transmit-packet due to org.opendaylight.controller.md.sal.dom.api.DOMRpcImplementationNotAvailableException
- Resolved
- relates to
-
OPNFLWPLUG-1024 Break out the odl-openflowplugin-app-topology feature.
- Resolved
-
NETVIRT-1031 Review controller-max-backoff and controller-inactivity-probe values
- Resolved