[CONTROLLER-1986] Dead letters: datastore.messages.DataTreeChangedReply Created: 13/Jul/21 Updated: 12/Aug/21 Resolved: 13/Jul/21 |
|
| Status: | Resolved |
| Project: | controller |
| Component/s: | clustering |
| Affects Version/s: | Sodium SR4, Magnesium SR3, 2.0.8, 4.0.0, 3.0.8 |
| Fix Version/s: | 4.0.1, 3.0.10, 2.0.9 |
| Type: | Bug | Priority: | High |
| Reporter: | Robert Varga | Assignee: | Robert Varga |
| Resolution: | Done | Votes: | 0 |
| Labels: | pt | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
Debugging some (probably unrelated) issues, I have come across https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/bgpcep-csit-1node-bgp-ingest-all-phosphorus/87/odl_1/odl1_karaf.log.gz which seems to log a ton of dead letters: 2021-07-04T14:00:32,185 | INFO | opendaylight-cluster-data-akka.actor.default-dispatcher-26 | LocalActorRef | 292 - org.opendaylight.controller.repackaged-akka - 3.0.7 | Message [org.opendaylight.controller.cluster.datastore.messages.DataTreeChangedReply] from Actor[akka://opendaylight-cluster-data/user/$Z#1237726688] to Actor[akka://opendaylight-cluster-data/user/shardmanager-operational/member-1-shard-default-operational#-13984916] was unhandled. [82804] dead letters encountered, of which 82793 were not logged. The counter will be reset now. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. The reason for these needs to be investigated. |
| Comments |
| Comment by Robert Varga [ 13/Jul/21 ] |
|
This seems to be caused by https://git.opendaylight.org/gerrit/c/controller/+/16690 as part of initial support in https://jira.opendaylight.org/browse/CONTROLLER-1135 , i.e. this is a day-zero bug. I think we can safely not generate these messages, but let's take a two step approach, where we first handle them correctly. |