[BGPCEP-442] BGP Route injection fails after controller restart Created: 18/Apr/16  Updated: 03/Mar/19  Resolved: 15/Aug/16

Status: Resolved
Project: bgpcep
Component/s: BGP
Affects Version/s: Bugzilla Migration
Fix Version/s: Bugzilla Migration

Type: Bug
Reporter: Ajay Chhabria Assignee: Claudio David Gasparini
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: Text File Delete_from_datastore_and_re-inject.txt     Text File First_Route_injection_after_restart.txt     Text File karaf_controller_1.log     Text File karaf_controller_2.log     Text File karaf_controller_3.log    
External issue ID: 5742

 Description   

SYMPTOM: BGP route injection fails through REST after all the 3 nodes clustered controllers are restarted. Please refer the attached text files for error messages.

CONDITIONS: 3-node clustering environment.

WORK-AROUND: Re-install the controllers and inject routes.

 Comments   
Comment by Ajay Chhabria [ 18/Apr/16 ]

Attachment Delete_from_datastore_and_re-inject.txt has been added with description: Delete from Datastore and re-inject

Comment by Ajay Chhabria [ 18/Apr/16 ]

Attachment First_Route_injection_after_restart.txt has been added with description: First route injection after restart

Comment by Milos Fabian [ 19/Apr/16 ]

Ajay, are there any exceptions/errors while closing the controllers? Could you please share full log file from all three nodes?

Comment by Ajay Chhabria [ 21/Apr/16 ]

Attachment karaf_controller_1.log has been added with description: Karaf Logs on the first node

Comment by Ajay Chhabria [ 21/Apr/16 ]

Attachment karaf_controller_2.log has been added with description: Karaf Logs on the second node

Comment by Ajay Chhabria [ 21/Apr/16 ]

Attachment karaf_controller_3.log has been added with description: Karaf Logs on the third node

Comment by Milos Fabian [ 21/Apr/16 ]

Those shutdown errors looks familiar - https://bugs.opendaylight.org/show_bug.cgi?id=4366

Comment by Milos Fabian [ 25/Apr/16 ]

I was able to reproduce the bug with one node ODL setup.

Comment by Claudio David Gasparini [ 14/Jul/16 ]

No able to reproduce with Be SR2 or master branch.

Steps done :
-Configure Rib
-Configure App Peer
-Configure Peer
-Send Route
-Logout
-Send new Route

I followed the next step trying to replicate the BUG, without success.

Ajay, Could you try it with latest version and confirm if you still able to replicate. And attach the steps you did.

Thank you

Comment by Ajay Chhabria [ 14/Jul/16 ]

Hi Claudio,

I have seen this issue in a 3-node clustering environment with BGP_RIB shard replicated across all the controller nodes. Following are the steps I did to reproduce it:

1. Got 3-node cluster UP.

2. Inject a BGP route.

3. Restart all the 3 controller nodes.

4. Inject the route again.

You could try the same steps with 1 node controller also.

Let me know if you need any further information regarding the same.

Thanks,
Ajay Chhabria

Comment by Claudio David Gasparini [ 20/Jul/16 ]

The issue is not reproducible in 1 node controller.
Confirmed on Be SR2 3 node clustering.

Issue is seen when application peers goes up after restart.

2016-07-20 11:47:38,654 | WARN | ult-dispatcher-6 | ConcurrentDOMDataBroker | 174 - org.opendaylight.controller.sal-distributed-datastore - 1.3.3.SNAPSHOT | Tx: DOM-CHAIN-35-0 Error during phase CAN_COMMIT, starting Abort
OptimisticLockFailedException

{message=Optimistic lock failed., errorList=[RpcError [message=Optimistic lock failed., severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was created by other transaction.]]}
Comment by Milos Fabian [ 25/Jul/16 ]

stable/beryllium: https://git.opendaylight.org/gerrit/#/c/42295/
TODO cherry-pick to master

Comment by Claudio David Gasparini [ 25/Jul/16 ]

Master: https://git.opendaylight.org/gerrit/#/c/42386/

Comment by Milos Fabian [ 29/Jul/16 ]

Re-opening as the bug fix patch was reverted.

Comment by Claudio David Gasparini [ 15/Aug/16 ]

Fixed by https://bugs.opendaylight.org/show_bug.cgi?id=6330

Generated at Wed Feb 07 19:13:04 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.