Uploaded image for project: 'controller'
  1. controller
  2. CONTROLLER-2043

Circuit breaker timeout with BGP and tell-based protocol

    XMLWordPrintable

Details

    • Bug
    • Status: Confirmed
    • Highest
    • Resolution: Unresolved
    • 5.0.0, 5.0.2, 5.0.3, 5.0.5
    • 9.0.0
    • clustering

    Description

      We are hitting this problem:

      2022-04-24T06:12:59,752 | ERROR | opendaylight-cluster-data-akka.actor.default-dispatcher-17 | Shard                            | 284 - org.opendaylight.controller.repackaged-akka - 5.0.2 | Failed to persist event type [org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry] with sequence number [262218] for persistenceId [member-1-shard-default-config].
akka.pattern.CircuitBreaker$$anon$13: Circuit Breaker Timed out.
2022-04-24T06:12:59,767 | INFO  | opendaylight-cluster-data-shard-dispatcher-31 | Shard                            | 289 - org.opendaylight.controller.sal-clustering-commons - 5.0.2 | Stopping Shard member-1-shard-default-config

      in BGP test suite:
      https://s3-logs.opendaylight.org/logs/releng/vex-yul-odl-jenkins-1/bgpcep-csit-1node-bgp-ingest-all-sulfur/216/odl_1/odl1_karaf.log.gz

      Previous versions work out fine, except they end up saving snapshot only at the 160K mark due to using ask-based protocol, which uses fewer persistence entries.

      Attachments

        1. data-1.log.xz
          21 kB
        2. segmented-journal.tar.xz
          64 kB
        3. vim-data-1-log.png
          vim-data-1-log.png
          527 kB

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. CONTROLLER-2056 Minimize serialization proxy names in controller.datastore.persisted

          • Highest - This problem will block progress.
          • Resolved

          Task - A task that needs to be done. CONTROLLER-2064 Switch to using RaftVersion.ARGON

          • Medium - Has the potential to affect progress.
          • Resolved

          Task - A task that needs to be done. CONTROLLER-2071 Switch to our fork of atomix-storage

          • Medium - Has the potential to affect progress.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. CONTROLLER-2037 Fail to serialize oversized message

          • Medium - Has the potential to affect progress.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. CONTROLLER-2026 Circuit breaker timeout

          • High - Serious problem that could block progress.
          • Confirmed

          Improvement - An improvement or enhancement to an existing feature or task. CONTROLLER-2044 Improve sal-akka-raft serialization protocol

          • High - Serious problem that could block progress.
          • Confirmed
          # Subject Branch Project Status CR V
          101859,6 Add Test asserting duration of writes master controller Status: NEW 0 +1
          100774,2 Add more tracing to DataJournalV0 master controller Status: MERGED +2 +1
          100775,3 Improve IdentifiablePayload.toString() implementations master controller Status: MERGED +2 +1
          100777,1 Add more tracing to DataJournalV0 4.0.x controller Status: MERGED +2 +1
          100778,1 Improve IdentifiablePayload.toString() implementations 4.0.x controller Status: MERGED +2 +1
          100824,2 Correct tracing arguments master controller Status: MERGED +2 +1
          100825,2 Improve SegmentedJournalActor logging master controller Status: MERGED +2 +1

          Activity

            People

              Unassigned Unassigned
              rovarga Robert Varga
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: