[CONTROLLER-1854] Metrics/counters for shard leadership changes Created: 23/Jul/18 Updated: 05/Oct/18 Resolved: 05/Oct/18 |
|
| Status: | Resolved |
| Project: | controller |
| Component/s: | clustering |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Medium |
| Reporter: | Vishal Thapar | Assignee: | Tom Pantelis |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Epic Link: | Clustering Stability |
| Description |
|
Today shard leadership changes have impact on services and too many shard leadership changes in a short duration are sign of unhealthy cluster. We need some means to track and detect frequency of shard leadership changes. For this we want some metrics/counters exposed by Controller. |
| Comments |
| Comment by Michael Vorburger [ 24/Jul/18 ] |
|
tpantelis on https://lists.opendaylight.org/pipermail/controller-dev/2018-July/014543.html said: "ShardStatsMXBean already has getLeadershipChangeCount and getLastLeadershipChangeTime." thapar so we can just read that (via Jolokia's URL) ... close this (controller) issue? |
| Comment by Vishal Thapar [ 24/Jul/18 ] |
|
Yep. Will get started on changes to use this in ODLTools. |
| Comment by Vishal Thapar [ 24/Jul/18 ] |
|
Yep. Will get started on changes to use this in ODLTools. |
| Comment by Muthukumaran Kothandaraman [ 25/Jul/18 ] |
|
The curl command which we use for this is curl --silent -u admin:admin http://localhost:8181/jolokia/read/org.opendaylight.controller:type=DistributedConfigDatastore,Category=Shards,name=member-1-shard-inventory-config/LeadershipChangeCount wherein we can replace the shard-name with all interested shards. I assume that the monitoring tools sit external to ODL and runs in a cron. Actually, that's how we use it |