[CONTROLLER-1592] DistributedEntityOwnershipIntegrationTest.testFunctionalityWithThreeNodes is unstable (flaky) test

Tue, 25 Jul 2023 08:24:21 +0000

~~CONTROLLER-1578~~'s https://git.opendaylight.org/gerrit/#/c/51486/'s https://jenkins.opendaylight.org/releng/job/controller-verify-carbon-mvn33-openjdk8/1791/console failed with:

00:36:54 DistributedEntityOwnershipIntegrationTest.testFunctionalityWithThreeNodes:201
00:36:54 follower2MockListener.ownershipChanged(
00:36:54 <DOMEntityOwnershipChange [entity=DOMEntity [type=entityType1, id=/(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)entity/entity[

{(urn:opendaylight:params:xml:ns:yang:mdsal:core:general-entity?revision=2015-09-30)name=entity2}

]], state=REMOTE_OWNERSHIP_CHANGED [wasOwner=false, isOwner=false, hasOwner=true], inJeopardy=false]>

00:34:05 Wanted 1 time:
00:34:05 -> at org.opendaylight.controller.cluster.datastore.entityownership.DistributedEntityOwnershipIntegrationTest.testFunctionalityWithThreeNodes(DistributedEntityOwnershipIntegrationTest.java:201)
00:34:05 But was 2 times. Undesired invocation:
00:34:05 -> at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerActor.onEntityOwnershipChanged(EntityOwnershipListenerActor.java:44)

I highly doubt that the change made for ~~CONTROLLER-1578~~ in c/51486 can cause this, as e.g. the DataBrokerTestCustomizer and the AbstractDataBrokerTest do not appear to be used at all by this DistributedEntityOwnershipIntegrationTest.

I tried to have a look locally, and saw it fail once on the branch of c/51486, and then on relaunch locally pass again... and couldn't get it to fail again. Using my RunUntilFailureRule from infrautils, I ran it about 27 times on both master and the branch of c/51486, and could never get it to fail again locally. So it appears to be a rare race condition or something ("heisenbug" ? and we should consider DistributedEntityOwnershipIntegrationTest.testFunctionalityWithThreeNodes to be an occasionally unstable (flaky) test...

No action for the moment, but if we see it again, I can @Ignore it, or someone else could see if they can fix it?

_____
Here's how to use my RunUntilFailureRule from infrautils:

<dependency>
<groupId>org.opendaylight.infrautils</groupId>
<artifactId>infrautils-testutils</artifactId>
<version>1.1.0-SNAPSHOT</version>
<scope>test</scope>
</dependency>

public class DistributedEntityOwnershipIntegrationTest {

public static @ClassRule RunUntilFailureClassRule classRepeater = new RunUntilFailureClassRule();
public @Rule RunUntilFailureRule repeater = new RunUntilFailureRule(classRepeater);

....

OpenDaylight JIRA

[CONTROLLER-1592] DistributedEntityOwnershipIntegrationTest.testFunctionalityWithThreeNodes is unstable (flaky) test