[CONTROLLER-1137] Clustering : A 3-Node cluster is not stable when connecting mininet to the inventory Leader node Created: 04/Feb/15 Updated: 24/Feb/15 Resolved: 24/Feb/15 |
|
| Status: | Resolved |
| Project: | controller |
| Component/s: | mdsal |
| Affects Version/s: | Post-Helium |
| Fix Version/s: | None |
| Type: | Bug | ||
| Reporter: | Moiz Raja | Assignee: | Moiz Raja |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Operating System: All |
||
| External issue ID: | 2676 |
| Priority: | Highest |
| Description |
|
When a mininet with any number of switches is connected to a 3-Node cluster, especially when connected to the node with the inventory Leader, the cluster appears to lose connections and eventually crashes. One of the reasons for this is that the StatisticsManager puts statistics into inventory at a very rapid rate (refreshed every 3 seconds). We need to add some back pressure mechanism on transaction creation to limit the work that the datastore takes on, thus giving it time to handle the work. |
| Comments |
| Comment by Tom Pantelis [ 12/Feb/15 ] |
|
Patch https://git.opendaylight.org/gerrit/#/c/14870/. This was merged - do we still need this open? |
| Comment by Moiz Raja [ 17/Feb/15 ] |
|
Till now we have tested the fix with 1 real controller and 2 simulated followers and it's been working fine. But I really would like to run this with 3 real controllers before I close this. Let's keep it open for now. |
| Comment by Moiz Raja [ 24/Feb/15 ] |
|
Tested with real cluster - seems to be stable. Will open more specific bugs as needed. Here are all the related commits, |