Uploaded image for project: 'controller'
  1. controller
  2. CONTROLLER-528

NotificationBrokerImpl is not scalable

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Resolution: Done
    • Helium
    • None
    • mdsal
    • None
    • Operating System: All
      Platform: All

    • 1120

    Description

      Profiling has shown that NotificationBrokerImpl does not allow for proper scaling. The problem is that each notification incurs a full lock on the listeners map, as it may have been modified by listener registration/unregistration. This leads to severe contention in the notification threads, as they keep bouncing the monitor for no good reason.

      We need to bias this structure heavily towards lookups and to do that we need to do the following:

      • make the map a constant in lookup path, accessible via an AtomicReference
      • rebuild it completely when a registration/unregistration occurs, and update the AtomicReference

      There is also a race condition present: notification dispatch is scheduled on an executor, so we can end up delivering notifications after the registration has been closed. Given the change in lookup/update balance, this race window will be enlarged, so fix it by having the task dispatch the event through the registration object, so that unregistration occurs atomically.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            rovarga Robert Varga
            rovarga Robert Varga
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: