[CONTROLLER-1721] binding-broker-impl occasionally takes too long to finish Created: 13/Jun/17  Updated: 25/Jul/23  Resolved: 05/Oct/18

Status: Resolved
Project: controller
Component/s: blueprint
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Vratko Polak Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

External issue ID: 8678

 Description   

Seen once so far, in Netconf all 3node job. Karaf.log lines are from [0].

This is very similar to AAA-132 but different config subsystem capability is taking too long this time:
2017-06-11 12:47:31,609 | WARN | saction-28-30'}} | DeadlockMonitor | 103 - org.opendaylight.controller.config-manager - 0.6.1.Carbon | ModuleIdentifier

{factoryName='binding-broker-impl', instanceName='binding-broker-impl'}

did not finish after 74969 ms

Leading to clustering failure:
2017-06-11 12:47:35,960 | ERROR | Event Dispatcher | AbstractDataStore | 180 - org.opendaylight.controller.sal-distributed-datastore - 1.5.1.Carbon | Shard leaders failed to settle in 90 seconds, giving up

[0] https://logs.opendaylight.org/releng/jenkins092/netconf-csit-3node-clustering-all-carbon/308/odl1_karaf.log.gz

 Comments   
Comment by Robert Varga [ 24/Aug/17 ]

This is probably caused by blueprint service start ordering.

Comment by Tom Pantelis [ 15/Sep/17 ]

The DeadlockMonitor warn does not lead to the clustering error. In fact it's the other way around. CDS waits up to 90s for all shard to elect leaders which blocks all other dependent blueprint containers from starting, including the binding broker which in turn blocks the CSS "binding-broker-impl" module from completing startup, hence the DeadlockMonitor warning. As we've seen in other cases, it seems test environment load can delay startup and node connectivity.

Generated at Wed Feb 07 19:56:17 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.