https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/INFRAUTILS-57 infrautils <p>The merge job of <a href="https://git.opendaylight.org/gerrit/#/c/76810/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/76810/</a> in <a href="https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/infrautils-merge-neon/84" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/infrautils-merge-neon/84</a> just failed with:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[jobcoordinator-main-task-1] ERROR org.opendaylight.infrautils.jobcoordinator.internal.JobCoordinatorImpl - Job still failed on final retry: JobEntry{key='org.opendaylight.infrautils.jobcoordinator.tests.JobCoordinatorTest', mainWorker=org.opendaylight.infrautils.jobcoordinator.tests.JobCoordinatorTest$TestCallable@21affc49, rollbackWorker=null, retryCount=3/3, futures=[com.google.common.util.concurrent.ImmediateFuture$ImmediateFailedFuture@671256e1[status=FAILURE, cause=[org.opendaylight.infrautils.jobcoordinator.tests.JobCoordinatorTest$JobException: Job is failed intentionally]]]} org.opendaylight.infrautils.jobcoordinator.tests.JobCoordinatorTest$JobException: Job is failed intentionally at org.opendaylight.infrautils.jobcoordinator.tests.JobCoordinatorTest.&lt;clinit&gt;(JobCoordinatorTest.java:49)</pre> </div></div> <p>This is the first time I've seen this failure. It passes locally, and I expect it will also pass on the "remerge".</p> <p>Still, someone should look into if this can be made more reliable. </p> <p>Must be concurrency issue.</p> INFRAUTILS-57

JobCoordinatorTest is flaky and (very rarely) fails on the build

Bug Medium Resolved Done Michael Vorburger Michael Vorburger Thu, 11 Oct 2018 20:51:25 +0000 Fri, 11 Jan 2019 02:17:15 +0000 Fri, 11 Jan 2019 02:17:15 +0000 Neon 0 3 <p>Looks like the failure is actually:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Not true that executeAttempts (&lt;0&gt;) is equal to &lt;1&gt; at org.opendaylight.infrautils.jobcoordinator.tests.JobCoordinatorTest.assertExecuteAttempts(JobCoordinatorTest.java:362) at org.opendaylight.infrautils.jobcoordinator.tests.JobCoordinatorTest.testJobNoExceptionReturnNull(JobCoordinatorTest.java:173) </pre> </div></div> <p>This is a timing issue where jobExecuteAttempts may not get marked/incremented. The JobQueueHandler does this:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> JobQueue jobQueue = entry.getValue(); if (jobQueue.getExecutingEntry() != null) {     jobExecuteAttempts.mark();     continue; } </pre> </div></div> <p>So jobExecuteAttempts is only incremented if the JobQueue already has an executing entry. I'm unclear as to the reason for this logic and what jobExecuteAttempts is intended to signify. The test will fail if the JobQueueHandler happens to start and initially runs the above code (when getExecutingEntry() is null) and also the job task is executed and completes <b>before</b> the isJobAvailable flag is set. This can be reproduced every time by inserting a sleep before setting isJobAvailable in signalForNextJob.</p> <p>As mentioned I don't really see the purpose of jobExecuteAttempts wrt how it's implemented nor the purpose of asserting that it's 1 in the test(s). I think we should remove the call to assertExecuteAttempts(1) since it's non-deterministic whether or not it will be 1. I'll push a patch.</p> <p><a href="https://git.opendaylight.org/gerrit/#/c/76915/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/76915/</a></p> <p>I've merged <a href="https://git.opendaylight.org/gerrit/#/c/76915/" class="external-link" target="_blank" rel="nofollow noopener">c/76915</a> to prevent this from failing &amp; blocking builds, but have a follow-up question:</p> <p>If you find that jobExecuteAttempts is basically pointless (becaues unreliable), how about just removing that counter all together? I mean in the Impl, not just in the test.</p> <p>PS: Added <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=k.faseela" class="user-hover" rel="k.faseela">k.faseela</a> as Watcher to this issue, Faseela jump in when you are back, or point this out to your internal friends with an intersted in the JobCoordinator.</p> <p>agreed but first I just wanted to prevent the sporadic failures. I don't know the history or intent of that metric or if it's used - let's see what Faseela et al say.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=k.faseela" class="user-hover" rel="k.faseela">k.faseela</a> given above do you (or any of your friends at E//) have any objections to the complete removal of the ExecuteAttempts metrics as just proposed via <a href="https://git.opendaylight.org/gerrit/#/c/77378/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/77378/</a> ? <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=tpantelis" class="user-hover" rel="tpantelis">tpantelis</a> FYI - I'm just trying to wrap up this loose end.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=vorburger" class="user-hover" rel="vorburger">vorburger</a> : I have just circulated an internal email to check this, but won't it be good to have this metrics, in case some job is stuck?</p> <p>My understanding, based just on reading the comments from <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=tpantelis" class="user-hover" rel="tpantelis">tpantelis</a> above, is that this particular metric is unreliable / broken, so best removed to avoid confusion.</p> Development Rank 0|i03jk7: