[OPNFLWJAVA-79] Infinite reschedule of flush Created: 06/Sep/16  Updated: 14/Nov/17  Resolved: 08/Sep/16

Status: Resolved
Project: openflowjava
Component/s: General
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Andrej Leitner Assignee: Michal Polkorab
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Issue Links:
Blocks
blocks OPNFLWPLUG-727 cbench throughput test does not work ... Resolved
External issue ID: 6646

 Description   

We observed in openflowplugin cbench test (before logging was set to ERROR)
https://jenkins.opendaylight.org/releng/job/openflowplugin-csit-1node-cbench-performance-only-boron many logs from OutboundQueueProviderImpl like "Replacing queue null with null". This is caused by infinite loop of flushing and closing AbstractOutboundQueueManager in ofjava - sometimes (on disconnect) there can be still some unflushed segments but thery are not able to be flushed if channel is not writable anymore. In local testing we got stuck with cpu on 97%

 Comments   
Comment by Andrej Leitner [ 06/Sep/16 ]

carbon: https://git.opendaylight.org/gerrit/#/c/45230/
boron: https://git.opendaylight.org/gerrit/#/c/45231/

Comment by A H [ 06/Sep/16 ]

To better assess the impact of this bug and fix, could someone from your team please help us identify the following:
Severity: Could you elaborate on the severity of this bug? Is this a BLOCKER such that we cannot release Boron without it? Is there a workaround such that we can write a release note and fix in future Boron SR1?
Testing: Could you also elaborate on the testing of this patch? How extensively has this patch been tested? Is it covered by any unit tests or system tests?
Impact: Does this fix impact any dependent projects?

Comment by Andrej Leitner [ 06/Sep/16 ]

Severity
This bug is quite serious thing since we stays with cpu usage nearly 100% almost everytime after openflowplugin cbench performance test is finished. This could be a security vulnerability as mentioned here:
https://bugs.opendaylight.org/show_bug.cgi?id=6176#c6
The issue was found during analysis of OPNFLWPLUG-727 which is already blocker and should help to resolve it. We don't see there a possibility of workaround.

Testing
There aren't any unit tests for modified part of ofjava. Actual internal integration tests passed successfully. We also successfully reproduce csit job locally https://jenkins.opendaylight.org/releng/job/openflowplugin-csit-1node-cbench-performance-only-boron. There aren't CSIT jobs for openflowjava because it should be covered by openflowplugin CSIT tests.

Impact
The fix impacts openflowplugin project, positively, we belive.

Comment by Andrej Leitner [ 06/Sep/16 ]

merged in boron

Comment by Luis Gomez [ 06/Sep/16 ]

This is fixed now according to:

https://jenkins.opendaylight.org/releng/job/openflowplugin-csit-1node-cbench-performance-only-boron/

BR/Luis

Comment by Andrej Leitner [ 07/Sep/16 ]

Thank you Luis for tracking. We need to merge it also in master branch.

Comment by Andrej Leitner [ 07/Sep/16 ]

merged in carbon

Comment by A H [ 08/Sep/16 ]

(In reply to Luis Gomez from comment #5)
> This is fixed now according to:
>
> https://jenkins.opendaylight.org/releng/job/openflowplugin-csit-1node-cbench-
> performance-only-boron/
>
> BR/Luis

Based on Luis's comment, can I safely assume that this bug been verified as fixed in the latest Boron RC 3.1 Build?

Comment by Luis Gomez [ 08/Sep/16 ]

Yes, this issue is fixed and working in RC3.1.

Generated at Wed Feb 07 20:28:53 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.