Uploaded image for project: 'OpenFlowPlugin'
  1. OpenFlowPlugin
  2. OPNFLWPLUG-599

He: FlowCapableInventoryProvider thread dies

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Resolution: Done
    • None
    • None
    • General
    • None
    • Operating System: Linux
      Platform: All

    • 5020

    Description

      The thread that polls from the queue (Named “FlowCapableInventoryProvider”) died at our system, because of an exception in one of the handlers.
      The run() loop should be protected from exceptions.

      The Exception that made the thread die:
      Exception in thread "FlowCapableInventoryProvider" java.lang.IllegalArgumentException: Instance identifier references (urn:opendaylight:inventory?revision=2013-08-19)node-connector[

      {(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:4:3}

      ] but data identifier is ImmutableMapEntryNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)node-connector[

      {(urn:opendaylight:inventory?revision=2013-08-19)id=3}

      ], value=[ImmutableLeafNode{nodeIdentifier=(urn:opendaylight:inventory?revision=2013-08-19)id, value=3, attributes={}}], attributes={}}

      This caused deadlocks in the system, because inventory change operations are now stuck.

      This is the thread that polls from the Notifications Queue, it is stuck because the FlowCapableInventoryProvider is dead:
      "pool-31-thread-1" prio=10 tid=0x00007fd9d4f02000 nid=0x17e7 waiting on condition [0x00007fda80f70000]
      java.lang.Thread.State: WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <0x000000072b8d69c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(Unknown Source)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
        at java.util.concurrent.LinkedBlockingDeque.putLast(Unknown Source)
        at java.util.concurrent.LinkedBlockingDeque.put(Unknown Source)
        at org.opendaylight.openflowplugin.applications.inventory.manager.FlowCapableInventoryProvider.enqueue(FlowCapableInventoryProvider.java:60)
        at org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter.onNodeRemoved(NodeChangeCommiter.java:125)
      • locked <0x000000072b8d6a40> (a org.opendaylight.openflowplugin.applications.inventory.manager.NodeChangeCommiter)
        at org.opendaylight.yangtools.yang.binding.util.NotificationListenerInvoker.invokeNotification(NotificationListenerInvoker.java:91)
        at org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationListenerAdapter.onNotification(BindingDOMNotificationListenerAdapter.java:44)
        at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouterEvent.deliverNotification(DOMNotificationRouterEvent.java:50)
        at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:68)
        at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter$1.onEvent(DOMNotificationRouter.java:65)
        at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

      So the Notifications Queue is full, and then we can’t put new Notifications:
      "nioEventLoopGroup-11-83" prio=10 tid=0x00007fd698061800 nid=0xb7bf runnable [0x00007fd5a6997000]
      java.lang.Thread.State: TIMED_WAITING (parking)
      at sun.misc.Unsafe.park(Native Method)
      at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
      at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
      at com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
      at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
      at org.opendaylight.controller.md.sal.dom.broker.impl.DOMNotificationRouter.putNotification(DOMNotificationRouter.java:200)
      at org.opendaylight.controller.md.sal.binding.impl.BindingDOMNotificationPublishServiceAdapter.putNotification(BindingDOMNotificationPublishServiceAdapter.java:51)
      at org.opendaylight.controller.md.sal.binding.compat.HeliumNotificationProviderServiceAdapter.publish(HeliumNotificationProviderServiceAdapter.java:33)
      at org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:115)
      at org.opendaylight.openflowplugin.openflow.md.core.sal.OFRpcTaskUtil$1FutureCallbackImpl.onSuccess(OFRpcTaskUtil.java:101)
      ….

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            narayanan.bhalaji@ericsson.com Bhalaji Narayanan
            guy.sela@hpe.com Guy Sela
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: