[GBP-4] GBP fails to launch intermittently when launched from Karaf Created: 22/Sep/14  Updated: 19/Oct/17  Resolved: 25/Sep/14

Status: Verified
Project: groupbasedpolicy
Component/s: General
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Keith Burns Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

External issue ID: 2019

 Description   

Requires reset of mininet and stop/start karaf.

 Comments   
Comment by Thomas Bachman [ 22/Sep/14 ]

When it works, I see this in the logs

2014-09-20 05:26:14,857 | INFO | config-pusher | InMemoryDataTree | 97 - org.opendaylight.yangtools.yang-data-impl - 0.6.2.SNAPSHOT | Attempting to install schema contexts
2014-09-20 05:26:14,865 | INFO | config-pusher | ThreadExecutorStatsMXBeanImpl | 109 - org.opendaylight.controller.sal-common-util - 1.1.0.SNAPSHOT | Executor com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService@5d2f96c0 is not supported
2014-09-20 05:26:14,865 | INFO | config-pusher | InMemoryDataTree | 97 - org.opendaylight.yangtools.yang-data-impl - 0.6.2.SNAPSHOT | Attempting to install schema contexts
2014-09-20 05:26:14,866 | INFO | config-pusher | ThreadExecutorStatsMXBeanImpl | 109 - org.opendaylight.controller.sal-common-util - 1.1.0.SNAPSHOT | Executor com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService@455d19f5 is not supported
2014-09-20 05:26:14,902 | INFO | config-pusher | dedBackwardsCompatibleDataBroker | 163 - org.opendaylight.controller.sal-binding-broker-impl - 1.1.0.SNAPSHOT | ForwardedBackwardsCompatibleBroker started.
2014-09-20 05:26:14,911 | WARN | config-pusher | BindingIndependentConnector | 163 - org.opendaylight.controller.sal-binding-broker-impl - 1.1.0.SNAPSHOT | Unable to start Notification forwarding. Reason: DOM Notification Service is not initialized properly!
2014-09-20 05:26:14,912 | INFO | config-pusher | RootBindingAwareBroker | 163 - org.opendaylight.controller.sal-binding-broker-impl - 1.1.0.SNAPSHOT | Starting Binding Aware Broker: binding-broker-impl
2014-09-20 05:26:14,937 | INFO | pool-17-thread-2 | FRMActivator | 220 - org.opendaylight.controller.md.forwardingrules-manager - 1.1.0.SNAPSHOT | FRMActivator initialization.
2014-09-20 05:26:15,095 | INFO | pool-17-thread-1 | LLDPDiscoveryProvider | 217 - org.opendaylight.controller.md.topology-lldp-discovery - 1.1.0.SNAPSHOT | LLDPDiscoveryListener Started.
2014-09-20 05:26:15,097 | INFO | er [_value=8185] | WebSocketServer | 198 - org.opendaylight.controller.sal-rest-connector - 1.1.0.SNAPSHOT | Web socket server started at port 8185.

when it’s stuck, it only gets to the first line, and I never see the config-pusher line:

2014-09-20 05:26:14,865 | INFO | config-pusher | ThreadExecutorStatsMXBeanImpl | 109 - org.opendaylight.controller.sal-common-util - 1.1.0.SNAPSHOT | Executor com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService@5d2f96c0 is not supported

Comment by Tony Tkacik [ 23/Sep/14 ]

Are you able to provide jstack or stack dump of JVM when that bug occured?

Comment by Tony Tkacik [ 23/Sep/14 ]

To do that you need to know PID of your Karaf (e.g. ps aux | grep java )
and run jstack $PID

Comment by Thomas Bachman [ 23/Sep/14 ]

Here's what I have:

alagalah@odlgbp1:~/testing/distribution-karaf-0.2.0-SNAPSHOT/data$ jstack 20032
2014-09-23 02:41:22
Full thread dump OpenJDK 64-Bit Server VM (24.65-b04 mixed mode):

"Attach Listener" daemon prio=10 tid=0x00007fe00403d800 nid=0x4f2e waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"odl-stat-ds-oper-thread-0" prio=10 tid=0x00007fdffc7f2800 nid=0x4f0b waiting on condition [0x00007fdfe14cd000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x000000078950b610> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
    at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
    at org.opendaylight.controller.md.statistics.manager.impl.StatisticsManagerImpl.run(StatisticsManagerImpl.java:190)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"odl-stat-rpc-oper-thread-0" prio=10 tid=0x00007fdffc7f1800 nid=0x4f0a waiting on condition [0x00007fdfe15ce000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x000000078950bc98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at org.opendaylight.controller.md.statistics.manager.impl.StatRpcMsgManagerImpl.run(StatRpcMsgManagerImpl.java:141)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"FlowCapableTopologyExporter-flow:1" daemon prio=10 tid=0x00007fdff4049000 nid=0x4f09 waiting on condition [0x00007fdfe16cf000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x000000078950be68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at org.opendaylight.md.controller.topology.manager.OperationProcessor.run(OperationProcessor.java:51)
    at java.lang.Thread.run(Thread.java:745)

"WriteTxCommit-0" daemon prio=10 tid=0x00007fdff4046000 nid=0x4f08 waiting on condition [0x00007fdfe17d0000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000781a471f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
    at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"FlowCapableInventoryProvider" daemon prio=10 tid=0x00007fe008b0a800 nid=0x4f07 waiting on condition [0x00007fdfe18d1000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x000000078950c9c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:489)
    at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:678)
    at org.opendaylight.controller.md.inventory.manager.FlowCapableInventoryProvider.run(FlowCapableInventoryProvider.java:96)
    at java.lang.Thread.run(Thread.java:745)

"Timer-0" prio=10 tid=0x00007fe00003f800 nid=0x4f06 in Object.wait() [0x00007fdfe19d2000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x000000078950d5a0> (a java.util.TaskQueue)
    at java.util.TimerThread.mainLoop(Timer.java:552)
  • locked <0x000000078950d5a0> (a java.util.TaskQueue)
    at java.util.TimerThread.run(Timer.java:505)

"nioEventLoopGroup-5-1" prio=10 tid=0x00007fe01412c800 nid=0x4ef9 runnable [0x00007fdfe222f000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)

  • locked <0x0000000781e0b978> (a io.netty.channel.nio.SelectedSelectionKeySet)
  • locked <0x0000000781e0da68> (a java.util.Collections$UnmodifiableSet)
  • locked <0x0000000781e0b8d0> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
    at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)

"Web socket server on port PortNumber [_value=8185]" daemon prio=10 tid=0x00007fe008e6b800 nid=0x4ef8 in Object.wait() [0x00007fdfe3bfa000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x0000000781e0b7e8> (a io.netty.channel.AbstractChannel$CloseFuture)
    at java.lang.Object.wait(Object.java:503)
    at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254)
  • locked <0x0000000781e0b7e8> (a io.netty.channel.AbstractChannel$CloseFuture)
    at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:129)
    at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:28)
    at io.netty.util.concurrent.DefaultPromise.sync(DefaultPromise.java:218)
    at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:117)
    at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:28)
    at org.opendaylight.controller.sal.streams.websockets.WebSocketServer.run(WebSocketServer.java:83)
    at java.lang.Thread.run(Thread.java:745)

"HashSessionScavenger-0" daemon prio=10 tid=0x00007fe00c037800 nid=0x4eed in Object.wait() [0x00007fdfe2330000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x0000000781ff9da8> (a java.util.TaskQueue)
    at java.util.TimerThread.mainLoop(Timer.java:552)
  • locked <0x0000000781ff9da8> (a java.util.TaskQueue)
    at java.util.TimerThread.run(Timer.java:505)

"ServletEventDispatcher: 3" daemon prio=10 tid=0x00007fe00c02b000 nid=0x4eec waiting on condition [0x00007fdfe2431000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007817c2800> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"ServletEventDispatcher: 2" daemon prio=10 tid=0x00007fe00c025800 nid=0x4eeb waiting on condition [0x00007fdfe2532000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007817c2800> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"ServletEventDispatcher: 1" daemon prio=10 tid=0x00007fe00c028000 nid=0x4eea waiting on condition [0x00007fdfe2936000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007817c2800> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"pool-10-thread-1" prio=10 tid=0x00007fe01c0a9800 nid=0x4ee9 waiting on condition [0x00007fdfe2633000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007818a71c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"WebEventExecutor: 1" daemon prio=10 tid=0x00007fe01c95a000 nid=0x4ee8 waiting on condition [0x00007fdfe2835000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007817b78f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"com.google.common.base.internal.Finalizer" daemon prio=10 tid=0x00007fe01c7c7000 nid=0x4ed9 in Object.wait() [0x00007fdfe2734000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x00000007811d90d0> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  • locked <0x00000007811d90d0> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
    at com.google.common.base.internal.Finalizer.run(Finalizer.java:132)
    at java.lang.Thread.run(Thread.java:745)

"com.google.common.base.internal.Finalizer" daemon prio=10 tid=0x00007fe01c12e800 nid=0x4ed5 in Object.wait() [0x00007fdfe2a37000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x0000000781439930> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  • locked <0x0000000781439930> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
    at com.google.common.base.internal.Finalizer.run(Finalizer.java:132)
    at java.lang.Thread.run(Thread.java:745)

"ConfigFeatureListener - ConfigPusher" daemon prio=10 tid=0x00007fe0085b4000 nid=0x4ece runnable [0x00007fdfe2b38000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007816f4968> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
    at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
    at org.opendaylight.controller.configpusherfeature.internal.ConfigPushingRunnable.run(ConfigPushingRunnable.java:46)
    at java.lang.Thread.run(Thread.java:745)

"config-pusher" daemon prio=10 tid=0x00007fe01c6a0000 nid=0x4ecd waiting on condition [0x00007fdfe2d3a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000781749a48> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at org.opendaylight.controller.netconf.persist.impl.ConfigPusherImpl.process(ConfigPusherImpl.java:74)
    at org.opendaylight.controller.netconf.persist.impl.osgi.ConfigPersisterActivator$InnerCustomizer$1.run(ConfigPersisterActivator.java:178)
    at java.lang.Thread.run(Thread.java:745)

"nioEventLoopGroup-2-1" prio=10 tid=0x00007fe01c64b800 nid=0x4ecc runnable [0x00007fdfe2c39000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)

  • locked <0x000000078173cc68> (a io.netty.channel.nio.SelectedSelectionKeySet)
  • locked <0x000000078173dcd8> (a java.util.Collections$UnmodifiableSet)
  • locked <0x000000078173cbd0> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
    at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)

"config-bundle-tracker-0" prio=10 tid=0x00007fe01c12c800 nid=0x4ec4 waiting on condition [0x00007fdfe2e3b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007817012f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-74" prio=10 tid=0x00007fe0140a0800 nid=0x4ec3 waiting on condition [0x00007fdfe2f3c000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000781374198> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
    at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-73" prio=10 tid=0x00007fe01409e800 nid=0x4ec2 waiting on condition [0x00007fdfe303d000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000781374198> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
    at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-72" prio=10 tid=0x00007fe01409c000 nid=0x4ec1 waiting on condition [0x00007fdfe313e000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000781374198> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
    at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-71" prio=10 tid=0x00007fe01409a000 nid=0x4ec0 waiting on condition [0x00007fdfe323f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000781374198> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
    at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-70 Acceptor1 SelectChannelConnector@0.0.0.0:8181" prio=10 tid=0x00007fe014098000 nid=0x4ebf runnable [0x00007fdfe3340000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)

  • locked <0x0000000781373e08> (a java.lang.Object)
    at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
    at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-69 Acceptor0 SelectChannelConnector@0.0.0.0:8181" prio=10 tid=0x00007fe014096000 nid=0x4ebe waiting for monitor entry [0x00007fdfe3441000]
java.lang.Thread.State: BLOCKED (on object monitor)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:225)

  • waiting to lock <0x0000000781373e08> (a java.lang.Object)
    at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
    at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-68 Selector1" prio=10 tid=0x00007fe01408b000 nid=0x4ebd runnable [0x00007fdfe3542000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)

  • locked <0x0000000781371638> (a sun.nio.ch.Util$2)
  • locked <0x0000000781371628> (a java.util.Collections$UnmodifiableSet)
  • locked <0x0000000781371500> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
    at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
    at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:745)

"qtp1384111178-67 Selector0" prio=10 tid=0x00007fe01408a800 nid=0x4ebc runnable [0x00007fdfe3643000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)

  • locked <0x0000000781370f78> (a sun.nio.ch.Util$2)
  • locked <0x0000000781370f68> (a java.util.Collections$UnmodifiableSet)
  • locked <0x0000000781370e30> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
    at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
    at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Thread.java:745)

"JMX OSGi Blueprint State Event Dispatcher" daemon prio=10 tid=0x00007fe00c007000 nid=0x4ebb waiting on condition [0x00007fdfe3744000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x000000078189f188> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"Bundle File Closer" daemon prio=10 tid=0x00007fe01c23d000 nid=0x4eb9 in Object.wait() [0x00007fdfe3af9000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x00000007818ad250> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
    at java.lang.Object.wait(Object.java:503)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:400)
  • locked <0x00000007818ad250> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:336)

"JMX OSGi Service State Event Dispatcher" daemon prio=10 tid=0x00000000015a9800 nid=0x4eb5 waiting on condition [0x00007fdfe3cfb000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007818a89b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"JMX OSGi Bundle State Event Dispatcher" daemon prio=10 tid=0x00007fdfe4009000 nid=0x4eb4 waiting on condition [0x00007fdff86c5000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007818a8f38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"RMI RenewClean-[127.0.1.1:44444]" daemon prio=10 tid=0x00007fe000005000 nid=0x4eb2 in Object.wait() [0x00007fdfe3efd000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x000000078190daf0> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  • locked <0x000000078190daf0> (a java.lang.ref.ReferenceQueue$Lock)
    at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:535)
    at java.lang.Thread.run(Thread.java:745)

"RMI Scheduler(0)" daemon prio=10 tid=0x00007fdfe802a000 nid=0x4eb1 waiting on condition [0x00007fdfe3ffe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007803693f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"JMX OSGi Agent" daemon prio=10 tid=0x00000000015a9000 nid=0x4eaf waiting on condition [0x00007fdff82c1000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000780f43ac8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"GC Daemon" daemon prio=10 tid=0x00007fdfe8015800 nid=0x4eae in Object.wait() [0x00007fdff83c2000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x0000000780c705a8> (a sun.misc.GC$LatencyLock)
    at sun.misc.GC$Daemon.run(GC.java:117)
  • locked <0x0000000780c705a8> (a sun.misc.GC$LatencyLock)

"RMI Reaper" prio=10 tid=0x00007fdfe8013800 nid=0x4ead in Object.wait() [0x00007fdff84c3000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x00000007804536d8> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
  • locked <0x00000007804536d8> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
    at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:351)
    at java.lang.Thread.run(Thread.java:745)

"RMI TCP Accept-44444" daemon prio=10 tid=0x00007fdfe8011800 nid=0x4eac runnable [0x00007fdff85c4000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
at java.net.ServerSocket.implAccept(ServerSocket.java:530)
at java.net.ServerSocket.accept(ServerSocket.java:498)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:388)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:360)
at java.lang.Thread.run(Thread.java:745)

"RMI TCP Accept-1099" daemon prio=10 tid=0x0000000001170800 nid=0x4eaa runnable [0x00007fdff87c6000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
at java.net.ServerSocket.implAccept(ServerSocket.java:530)
at java.net.ServerSocket.accept(ServerSocket.java:498)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:388)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:360)
at java.lang.Thread.run(Thread.java:745)

"Thread-12" daemon prio=10 tid=0x000000000155a000 nid=0x4ea8 runnable [0x00007fdff9b28000]
java.lang.Thread.State:...

Generated at Wed Feb 07 20:00:54 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.