[CONTROLLER-1798] ShardManager will miss sending a PeerAddressResolved message to local shards when UpdateSchemaContext comes after MemberUp Created: 27/Nov/17 Updated: 05/Dec/17 Resolved: 05/Dec/17 |
|
| Status: | Resolved |
| Project: | controller |
| Component/s: | clustering |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Medium |
| Reporter: | Kangqian Yin | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
ShardManager will miss sending a PeerAddressResolved message to local shards when UpdateSchemaContext comes after MemberUp. Currently PeerAddressResolved is only sent when receiving MemberUp/MemberReachable messages. At that time, the local shards actors may still have not been created for the UpdateSchemaContext message has not come. So the PeerAddressResolved message won't be sent to the local shards actors for they're null. When UpdateSchemaContext arrives later, local shards actors will be created with no PeerAddress MBean information. To avoid such missing, PeerAddressResolved should also be sent after receiving UpdateSchemaContext if there're peer members. |
| Comments |
| Comment by Tom Pantelis [ 30/Nov/17 ] |
|
This really shouldn't matter since RaftActorContextImpl#getPeerAddress will obtain the peer's address from the PeerAddressResolver if the address isn't set in the peerInfoMap |
| Comment by Kangqian Yin [ 04/Dec/17 ] |
|
Hi, Tom, just as you said, RaftActorContextImpl#getPeerAddress will obtain the peer's address and set it to the peerInfoMap if it's not set. But RaftActorContextImpl#getPeerAddress won't put a new PeerInfo object into the peerInfoMap if the peerInfoMap doesn't contain the peer. And RaftActorContextImpl#getPeerAddress is not called by RaftActorContextImpl#getPeers which is called by RaftActor#onGetOnDemandRaftState when a shard responses to a GetOnDemandRaftState message issued by the shard mbean ShardStats. So the JMX client cannot retrieve the address information of the peer which doesn't exist in peerInfoMap. So we have either to modify RaftActorContextImpl#getPeers to get all existing peers in the peer address resolver, or to modify the ShardManager#updateSchemaContext to send PeerAddressResolved messages for all existing peers to the shard. Because PeerAddressResolver interface only has a resolve(String peerId) API. I think it's better to modify the ShardManager#updateSchemaContext. I'll upload a patch to fix this later. |
| Comment by Kangqian Yin [ 04/Dec/17 ] |