[CONTROLLER-1988] Migrate from akka persistence to a custom solution

Wed, 21 Jul 2021 12:30:46 +0000

We need to move to akka typed due to message chunking. During this it would be helpfull
if we could also migrate away from akka persistence as it does not match our usecase completely.

RaftActor.persistData() has synchronous callback from akka persistence
seems like we should be able to get rid of this by transforming the
callback from persistence into a message, notifying the RaftActor
that persistence has completed for a given entry.
We could potentially switch mailbox implementation to
UnboundedStablePriorityMailbox in RaftActor to give the
replies faster turn around time after an entry has been persisted
so commit of an entry is not blocked by other messages.

There is another bunch of direct calls to handleApplyState().
On the leader they are triggered from the persistence callback above.
We should be able to move the execution of these to the new message
from persistence outlined above, while still keeping the direct call
in (with CheckConsensusReached or Replicate messages) in the execution
of the new message from persistence.

On follower we end up calling handleApplyState() as part of handleAppendEntries()
(from applyLogToStateMachine()) execution, this should stay as is as we
want to apply an entry as soon as we can and not introduce a potential delay.

The only time handleApplyState() is handled asynchronously, is in the PreLeader
state where its used to check whether all entries from previous term were commited
and switches behavior to Leader.

Delete of entries from the journal and snapshot is needed as well.
Both are needed during snapshotiting for old journal entries or old snapshots.
ie: DataPersistenceProvider.deleteMessages() is used in RaftActorRecoverySupport
and SnapshotManager to delete journal after snapshot has been saved.
DataPersistenceProvider.deleteSnapshots() is used in SnapshotManager to delete
old snapshots after a snapshot has been saved.

Apart from the replacement of synchronous callback, it looks like the switch
should be relatively painless and the majority of time spent needs to
happen on akka-persistence replacement(atomix.io?) and rewriting to akka typed.

OpenDaylight JIRA

[CONTROLLER-1988] Migrate from akka persistence to a custom solution