[LISPMAP-154] Integration tests fail intermittently with unexpected SMR (a.k.a. the Heisenbug) Created: 12/May/17 Updated: 19/Oct/17 Resolved: 18/May/17 |
|
| Status: | Verified |
| Project: | lispflowmapping |
| Component/s: | General |
| Affects Version/s: | Carbon |
| Fix Version/s: | None |
| Type: | Bug | ||
| Reporter: | Lori Jakab | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Operating System: All |
||
| External issue ID: | 8429 |
| Priority: | High |
| Description |
|
Integration tests fail relatively rarely when running on Karaf 3 and more often when running on Karaf 4 with message similar to this: MappingServiceIntegrationTest.testMultiSite:418->testMultiSiteScenarioA:762 SMR contained Eid [_address=Ipv4PrefixBinary [_ipv4AddressBinary=Ipv4AddressBinary [_value=[-64, 0, 1, 1]], _ipv4MaskLength=32, augmentation=[]], _addressType=class org.opendaylight.yang.gen.v1.urn.opendaylight.lfm.lisp.binary.address.types.rev160504.Ipv4PrefixBinaryAfi, _virtualNetworkId=InstanceIdType [_value=2], augmentation=[]] which wasn't expected. This points to a concurrency issue in the SMR handling code, which may lead to incorrect SMRs being sent and xTRs maintaining state that is out of date, potentially blackholing traffic. See https://logs.opendaylight.org/releng/jenkins092/lispflowmapping-merge-carbon/119/console.log.gz for an example. |
| Comments |
| Comment by Lori Jakab [ 12/May/17 ] |
|
Fix proposed: https://git.opendaylight.org/gerrit/#/c/56842/ |
| Comment by Vina Ermagan [ 16/May/17 ] |
|
Fix is now verified, and cherry picked to stable/carbon. https://git.opendaylight.org/gerrit/#/c/57200/ The IT related to this bug was disabled in stable/carbon to unlock autorelease. Considering the fix, now the IT disabling patch should also be reverted. The revert patch is here: |
| Comment by A H [ 17/May/17 ] |
|
We are looking to build Carbon RC2 tomorrow 5/18 at 23:59 UTC time assuming there are no blocker bugs. Is there an ETA for when a fix can be merged and this bug resolved for stable/carbon branch? |
| Comment by Vina Ermagan [ 17/May/17 ] |
|
(In reply to A H from comment #3) Both the fix and the revert of IT disabling patches linked above have been +2-ed. An would appreciate if you can merge them prior to RC2 build start. Thanks, |