[BGPCEP-145] "executor's queue is full" breaks PCEP pipeline Created: 09/Oct/14  Updated: 03/Mar/19  Resolved: 28/Oct/14

Status: Resolved
Project: bgpcep
Component/s: PCEP
Affects Version/s: Bugzilla Migration
Fix Version/s: Bugzilla Migration

Type: Bug
Reporter: Vratko Polak Assignee: Dana Kutenicsova
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Linux
Platform: PC


Attachments: Text File cut.log    
External issue ID: 2169

 Description   

Using pcc-mock to try and push 2x50000 tunnels, connections went down.
Even after closing pcc-mock and waiting (low CPU utilization on ODL), 31185 tunnels were left present in pcep-topology.

Expected result for fixed ODL: When queue gets full, pcep-topology should retry after some time, or do some kind of blocking insert. Input from PCEP session would be slowed down or blocked, and this state would be recognized to suspend usual deadtimer event consequences.

Segment of karaf.log attached, most of exceptions happened two times, for each PCEP session once. Highlights:

2014-10-09 16:17:53,140 | ERROR | oupCloseable-4-2 | DOMDataCommitCoordinatorImpl | 157 - org.opendaylight.controller.sal-broker-impl - 1.1.0.Helium | The commit executor's queue is full - submit task was rejected.
(2 times)

2014-10-09 16:17:53,160 | ERROR | oupCloseable-4-2 | TopologyNodeState | 261 - org.opendaylight.bgpcep.pcep-topology-provider - 0.3.1.Helium | Unexpected transaction failure in node
(2 times)

2014-10-09 16:17:53,167 | ERROR | oupCloseable-4-2 | AbstractTopologySessionListener | 261 - org.opendaylight.bgpcep.pcep-topology-provider - 0.3.1.Helium | Failed to update internal state for session
(2 times)

2014-10-09 16:17:53,175 | INFO | oupCloseable-4-2 | PCEPSessionImpl | 251 - org.opendaylight.bgpcep.pcep-impl - 0.3.1.Helium | Closing PCEP session:
(2 times)

2014-10-09 16:17:53,191 | WARN | oupCloseable-4-3 | DefaultChannelPipeline | 126 - io.netty.common - 4.0.23.Final | An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
(roughly thousand times in full log)



 Comments   
Comment by Vratko Polak [ 09/Oct/14 ]

Attachment cut.log has been added with description: Segment of karaf.log showing exceptions.

Comment by Dana Kutenicsova [ 27/Oct/14 ]

Wont fix in PCEP.

https://bugs.opendaylight.org/show_bug.cgi?id=2255

Comment by Robert Varga [ 28/Oct/14 ]

Clarification: BUG-2255 is a follow-up bug, with the request to provide a generic solution to the problem, as we do not want to replicate point solution in each MD-SAL user. This issue will be fixed once that solution is available.

Generated at Wed Feb 07 19:12:10 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.