[NETVIRT-359] VPNInstance bailing out of wait loop Created: 14/Dec/16  Updated: 24/Jan/17  Resolved: 24/Jan/17

Status: Resolved
Project: netvirt
Component/s: General
Affects Version/s: Boron
Fix Version/s: None

Type: Bug
Reporter: Sam Hague Assignee: Kiran N Upadhyaya
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: File odl1_karaf.log.gz.rar     File odl1_karaf.tar.xz    
Issue Links:
Duplicate
duplicates NETVIRT-371 NullPointerException at org.opendayl... Resolved
External issue ID: 7371

 Description   

Found in upstream and downstream csit: https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-mitaka-upstream-transparent-boron/69/archives/

2016-12-14 14:03:43,638 | INFO | CountersThread | CountersDumperThread | 276 - org.opendaylight.infrautils.counters-impl - 1.0.2.SNAPSHOT | dsjcc.jobs_cleared: +36 , dsjcc.jobs_pending: 0 , nsc.subnetmap_update: +2
2016-12-14 14:03:43,783 | INFO | nPool-1-worker-0 | VpnInstanceListener | 317 - org.opendaylight.netvirt.vpnmanager-impl - 0.3.2.SNAPSHOT | VPNInstance bailing out of wait loop as current interface count is 1 and max retries exceeded for for vpnName 9a837a52-561f-43d2-81fa-756b5fb650d6, rd 9a837a52-561f-43d2-81fa-756b5fb650d6
2016-12-14 14:03:43,783 | INFO | nPool-1-worker-0 | VpnInstanceListener | 317 - org.opendaylight.netvirt.vpnmanager-impl - 0.3.2.SNAPSHOT | Returned out of waiting for Op Data removal for rd 9a837a52-561f-43d2-81fa-756b5fb650d6, vpnname 9a837a52-561f-43d2-81fa-756b5fb650d6
2016-12-14 14:03:43,784 | WARN | nPool-1-worker-0 | DataStoreJobCoordinator | 281 - org.opendaylight.genius.mdsalutil-api - 0.1.2.SNAPSHOT | Job VPN-9a837a52-561f-43d2-81fa-756b5fb650d6 took 40004ms to complete
2016-12-14 14:03:43,786 | ERROR | ool-47-thread-40 | VrfEntryListener | 319 - org.opendaylight.netvirt.fibmanager-impl - 0.3.2.SNAPSHOT | VPN Instance for rd 9a837a52-561f-43d2-81fa-756b5fb650d6 is not available from VPN Op Instance Datastore
2016-12-14 14:03:43,786 | INFO | ool-47-thread-40 | VrfEntryListener | 319 - org.opendaylight.netvirt.fibmanager-impl - 0.3.2.SNAPSHOT | REMOVE: Removed Fib Entry rd 9a837a52-561f-43d2-81fa-756b5fb650d6 prefix 2003::3/128 nexthop [10.29.13.72] label 70034

 Comments   
Comment by Sam Hague [ 14/Dec/16 ]

Attachment odl1_karaf.tar.xz has been added with description: karaf.log

Comment by Olga Schukin [ 22/Jan/17 ]

Sometimes we got into endless loop because a current interface counter value inside the function waitForOpRemoval() is less that original counter but it still not zero, then we are back to beginning of the cycle without any action and sleep again .
The infinite loop doesn’t allow to DataStoreJobCoordinator to run any other job within same key.
Please see an example in the attached log, the vpnname ccc2cbb7-efaa-4243-91fb-bc89f3f549db.

Comment by Olga Schukin [ 22/Jan/17 ]

Attachment odl1_karaf.log.gz.rar has been added with description: log - look for vpnname ccc2cbb7-efaa-4243-91fb-bc89f3f549db

Comment by Kiran N Upadhyaya [ 24/Jan/17 ]

The infinite loop problem is being handled by https://git.opendaylight.org/gerrit/50826.

Thanks for raising this review Olga!!

Comment by Kiran N Upadhyaya [ 24/Jan/17 ]

The Vpn Instance bailing out of wait loop issue is because deletion of one of the vpn interfaces is failing. Hence, after the fixed number of retries, we are bailing out and proceeding with deleting the operational Vpn Instance data.
The issue with Vpn Interface deletion is being tracked through NETVIRT-371 .

Generated at Wed Feb 07 20:21:22 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.