[ODLPARENT-27] PowerMock tests aren't accounted for in JaCoCo coverage Created: 12/Jan/16 Updated: 17/Oct/21 Resolved: 17/Oct/21 |
|
| Status: | Resolved |
| Project: | odlparent |
| Component/s: | General |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | ||
| Reporter: | Stephen Kitt | Assignee: | Robert Varga |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Operating System: All |
||
| Issue Links: |
|
||||||||
| External issue ID: | 4941 | ||||||||
| Description |
|
Despite the version hold on JaCoCo, PowerMock tests don't appear in the coverage results; for example https://sonar.opendaylight.org/drilldown/measures/12541?metric=coverage&rids[]=15639&rids[]=45971 currently shows 0.1% coverage in NeutronL3Adapter, yet the test code covers over 60% of that class (as measured independently using IntelliJ and JaCoCo). The latest version of PowerMock (1.6.4) is supposed to fix this, but apparently doesn't. https://git.opendaylight.org/gerrit/#/c/32427/ shows how to instrument the code so it gets measured correctly. That unfortunately causes a number of "class already instrumented" errors to be logged, so it feels sub-optimal. I'm investigating whether the latest version of JaCoCo (which would involve a Sonar plugin upgrade) fixes things. |
| Comments |
| Comment by Konstantin Blagov [ 22/Sep/16 ] |
|
Still confirmed: shows line coverage of 19.9%, while in IntelliJ IDEA it is 93%. Total Groupbasedpolicycoverage is 70.2% (Sonar) vs 78% (IntelliJ) On Groupbasedpolicy, we are striving (and being required too) to make 80% covered, so it is very unfortunate to have PowerMock tests skipped by Sonar. As PowerMock team states that the JaCoCo incompatibiliy was fixed in 1.6.4 (Dec 2015), we hope it is just a matter of ODL's Sonar setup. |
| Comment by Stephen Kitt [ 22/Sep/16 ] |
|
The incompatibility is supposed to be fixed, but in my tests that's not the case (I haven't looked into it enough to re-open the bug upstream). Long term I'd like us to move away from using PowerMock altogether, it tends to encourage sub-optimal design and test choices... |
| Comment by Robert Varga [ 22/Aug/19 ] |
|
skitt does this still reproduce with pm-2.0.2? |
| Comment by Stephen Kitt [ 22/Aug/19 ] |
|
Apparently yes, Sonar reports 0% coverage for SouthboundUtil.getManagingNode in ovsdb even though it’s tested by SouthboundUtilTest (which uses PowerMock). |
| Comment by Robert Varga [ 17/Oct/21 ] |
|
So we are getting an execution profile for southbound-impl, but we also get the following: [INFO] --- jacoco-maven-plugin:0.8.7:report (report) @ southbound-impl --- [INFO] Loading execution data file /home/nite/odl/ovsdb/southbound/southbound-impl/target/code-coverage/jacoco.exec [INFO] Analyzed bundle 'ODL :: ovsdb :: southbound-impl' with 79 classes [WARNING] Classes in bundle 'ODL :: ovsdb :: southbound-impl' do not match with execution data. For report generation the same class files must be used as at runtime. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TerminationPointUpdateCommand does not match. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TerminationPointCreateCommand does not match. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/OvsdbConnectionManager$ConnectionReconciliationTriggers does not match. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TerminationPointDeleteCommand does not match. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/TransactUtils does not match. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/ovsdb/transact/OvsdbNodeUpdateCommand does not match. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/OvsdbConnectionManager$1 does not match. [WARNING] Execution data for class org/opendaylight/ovsdb/southbound/OvsdbConnectionManager$2 does not match. Which is indicative of these classes being manipulated by powermock's JUnitRunner. I do not see how we can work this around There is quite a bit of background: here is one powermock issue and corresponding writeup. PowerMocks migration to ByteBuddy is dead in the water. The other side of the coin is JaCoCo, which has a long-standing issue around this. All in all, I do not believe we can fix this globally – the offline instrumentation workaround might work, but is inappropriate for pretty much the entire world. Since this is quite limited in terms of affected projects (see |
| Comment by Robert Varga [ 17/Oct/21 ] |
|
Downstreams affected by this issue are advised to deploy one of the referenced workarounds, or, preferably, migrate to Mockito, which can now mock static methods as well as spy final classes. |
| Comment by Robert Varga [ 17/Oct/21 ] |
|
So specifically here, the reason for coverage not being shown is that SouthboundUtil is subject to @PrepareForTest, which loads the class from disk and hence blows away JaCoCo instrumentation. That is the only way Javassist can work with classes and the reason we have migrated away from it in mdsal-binding-dom-codec – ByteBuddy is way better for this sort of task. |