[ODLPARENT-27] PowerMock tests aren't accounted for in JaCoCo coverage Created: 12/Jan/16  Updated: 17/Oct/21  Resolved: 17/Oct/21

Status: Resolved
Project: odlparent
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Stephen Kitt Assignee: Robert Varga
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Issue Links:
Relates
relates to ODLPARENT-275 Remove powermock declaration Resolved
External issue ID: 4941

 Description   

Despite the version hold on JaCoCo, PowerMock tests don't appear in the coverage results; for example https://sonar.opendaylight.org/drilldown/measures/12541?metric=coverage&rids[]=15639&rids[]=45971 currently shows 0.1% coverage in NeutronL3Adapter, yet the test code covers over 60% of that class (as measured independently using IntelliJ and JaCoCo).

The latest version of PowerMock (1.6.4) is supposed to fix this, but apparently doesn't. https://git.opendaylight.org/gerrit/#/c/32427/ shows how to instrument the code so it gets measured correctly. That unfortunately causes a number of "class already instrumented" errors to be logged, so it feels sub-optimal.

I'm investigating whether the latest version of JaCoCo (which would involve a Sonar plugin upgrade) fixes things.



 Comments   
Comment by Konstantin Blagov [ 22/Sep/16 ]

Still confirmed:

https://sonar.opendaylight.org/component/index?id=org.opendaylight.groupbasedpolicy%3Aios-xe-renderer%3Asrc%2Fmain%2Fjava%2Forg%2Fopendaylight%2Fgroupbasedpolicy%2Frenderer%2Fios_xe_provider%2Fimpl%2Fwriter%2FPolicyWriterUtil.java

shows line coverage of 19.9%, while in IntelliJ IDEA it is 93%. Total Groupbasedpolicycoverage is 70.2% (Sonar) vs 78% (IntelliJ)

On Groupbasedpolicy, we are striving (and being required too) to make 80% covered, so it is very unfortunate to have PowerMock tests skipped by Sonar. As PowerMock team states that the JaCoCo incompatibiliy was fixed in 1.6.4 (Dec 2015), we hope it is just a matter of ODL's Sonar setup.

Comment by Stephen Kitt [ 22/Sep/16 ]

The incompatibility is supposed to be fixed, but in my tests that's not the case (I haven't looked into it enough to re-open the bug upstream).

Long term I'd like us to move away from using PowerMock altogether, it tends to encourage sub-optimal design and test choices...

Comment by Robert Varga [ 22/Aug/19 ]

skitt does this still reproduce with pm-2.0.2?

Comment by Stephen Kitt [ 22/Aug/19 ]

Apparently yes, Sonar reports 0% coverage for SouthboundUtil.getManagingNode in ovsdb even though it’s tested by SouthboundUtilTest (which uses PowerMock).

Comment by Robert Varga [ 17/Oct/21 ]

So we are getting an execution profile for southbound-impl, but we also get the following:

[INFO] --- jacoco-maven-plugin:0.8.7:report (report) @ southbound-impl ---
[INFO] Loading execution data file /home/nite/odl/ovsdb/southbound/southbound-impl/target/code-coverage/jacoco.exec
[INFO] Analyzed bundle 'ODL :: ovsdb :: southbound-impl' with 79 classes
[WARNING] Classes in bundle 'ODL :: ovsdb :: southbound-impl' do not match with execution data. For report generation the same class files must be used as at runtime.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TerminationPointUpdateCommand does not match.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TerminationPointCreateCommand does not match.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/OvsdbConnectionManager$ConnectionReconciliationTriggers does not match.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TerminationPointDeleteCommand does not match.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TransactUtils does not match.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/OvsdbNodeUpdateCommand does not match.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/OvsdbConnectionManager$1 does not match.
[WARNING] Execution data for class org/opendaylight/ovsdb/southbound/OvsdbConnectionManager$2 does not match.

Which is indicative of these classes being manipulated by powermock's JUnitRunner. I do not see how we can work this around

There is quite a bit of background: here is one powermock issue and corresponding writeup. PowerMocks migration to ByteBuddy is dead in the water. The other side of the coin is JaCoCo, which has a long-standing issue around this.

All in all, I do not believe we can fix this globally – the offline instrumentation workaround might work, but is inappropriate for pretty much the entire world. Since this is quite limited in terms of affected projects (see ODLPARENT-275), I think the proper fix is to deploy one of the solutions outlined in project-local manner.

Comment by Robert Varga [ 17/Oct/21 ]

Downstreams affected by this issue are advised to deploy one of the referenced workarounds, or, preferably, migrate to Mockito, which can now mock static methods as well as spy final classes.

Comment by Robert Varga [ 17/Oct/21 ]

So specifically here, the reason for coverage not being shown is that SouthboundUtil is subject to @PrepareForTest, which loads the class from disk and hence blows away JaCoCo instrumentation. That is the only way Javassist can work with classes and the reason we have migrated away from it in mdsal-binding-dom-codec – ByteBuddy is way better for this sort of task.

Generated at Wed Feb 07 20:27:25 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.