Details
-
Bug
-
Status: Resolved
-
Resolution: Won't Do
-
0.4.0
-
None
-
None
-
Operating System: All
Platform: All
-
7314
Description
We'd met a problem of using up JVM heap while running OpenDaylight controller. The reason was that the consuming rate of data tree change listeners was much slower than the producing rate of shard data trees. At the same time, the mailbox of actor for data tree change listener uses unlimited-size queue. So the data tree change messages had cumulated infinitely until using up the JVM heap.
After we limited the size of the mailbox queue, the JVM won't run into the mire of fully garbage-collecting of a used-up heap and kept stable no matter how slowly the data tree change listeners were consuming their data tree change messages.
Additionally, we added a new metric for the data tree change listener actor to track the number of dropped messages due to the limited-size queue. Attachment is the screenshot of an example of such metric.