[INFRAUTILS-24] JobCoordinator should use bounded Executors and bound its own job queue Created: 04/Nov/17  Updated: 13/Apr/21  Resolved: 13/Apr/21

Status: Resolved
Project: infrautils
Component/s: jobcoordinator
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: High
Reporter: Michael Vorburger Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Blocks
blocks GENIUS-96 DataStoreJobCoordinator OOM Verified
is blocked by INFRAUTILS-21 Automatic deadlock detection logging Resolved

 Description   

motivated by GENIUS-96

as per https://lists.opendaylight.org/pipermail/infrautils-dev/2017-November/000487.html

it seems to me that JobCoordinatorImpl unbounded datastructures are very risky, and that we should limit and reject new jobs at the time enqueueJob() is called if we're full, instead of OOM'ing.

NB there are at least 3 if not more datastructures here I see we need to protect:

  • ForkJoinPool fjPool
  • ScheduledExecutorService scheduledExecutorService
  • Map<String, JobQueue> jobQueueMap keys
  • Map<String, JobQueue> jobQueueMap internal JobQueue (separate from the keys)

And when JC is "full", it would probably be useful in future analysis to have a dump of all jobs that are stuck in the queue at that point? Or force a thread dump via INFRAUTILS-21 ?

 Comments   
Comment by Robert Varga [ 13/Apr/21 ]

Superseded by INFRAUTILS-75.

Generated at Wed Feb 07 20:02:04 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.