[CONTROLLER-911] Clustering : Make persistence configurable at the datastore or shard level Created: 02/Oct/14  Updated: 25/May/20  Resolved: 31/Oct/14

Status: Resolved
Project: controller
Component/s: mdsal
Affects Version/s: Helium
Fix Version/s: None

Type: Bug
Reporter: Moiz Raja Assignee: Moiz Raja
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Issue Links:
Duplicate
is duplicated by CONTROLLER-871 Clustering: Make shard persistence co... Resolved
is duplicated by CONTROLLER-1916 Allow persistence to be enabled/disab... Resolved
External issue ID: 2134

 Description   

Operational data does not need to be persisted. We should therefore be able to disable persistence for the whole operational datastore or a maybe even specific shards so that we can use this as a means to optimize performance.

Currently we persist by default to ensure that when a node is restarted it can restore it's state from disk and subsequently get synced up with the Leader. If we do not restore the state from disk the complete state will be replicated from the leader. We need to ensure that replicating the whole state of a shard is fast enough so that using replication vs persistence for recovery does not have an adverse effect on performance.

 Comments   
Comment by Moiz Raja [ 14/Oct/14 ]

1. When starting up check if persistence is enabled. If it is not do not recover from disk.
2. On recovery when the journal is empty the Leader should send complete state to the follower instead of checking for log entries.
3. Replication should work even when persistence if turned off.
4. Configurable using config subsystem. For Helium-1 this will only be at the datastore level.

Comment by Moiz Raja [ 16/Oct/14 ]

The original idea was to use a an in-memory journal/snapshot store when persistence was not enabled. That may not work however because currently you can have only one journal/snapshot store per actor system. See https://groups.google.com/forum/#!topic/akka-user/8Vlf6UonURo.

Comment by Moiz Raja [ 16/Oct/14 ]

Since persistence plugins are configurable only at the actor system level it looks like we will need to create a separate actor system for the config datastore. This actor system will use an inmmemory-journal/snapshot store for pllugins. The rest of the design stays as is.

Comment by Moiz Raja [ 31/Oct/14 ]

Finally went with conditionally turning persistence on and off on the Shard/RaftActor

Generated at Wed Feb 07 19:54:11 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.