[CONTROLLER-1169] Clustering : Snapshotting logic throws an exception when creating snapshot with replicatedLogIndex=-1

Tue, 24 Feb 2015 02:53:34 +0000

When creating a real snapshot the replicatedToAllLogIndex in CaptureSnapshot may be -1 in the following cases,

1. When the replicatedToAllLogIndex has not moved forward at all and is still initialized to -1
2. When there replicatedToAllLogIndex was not found in the replicated log.

This can result in the snapshotting logic throwing an exception as follows,

{lastTerm:1, lastIndex:19999, LastAppliedIndex:19998, LastAppliedTerm:1, UnAppliedEntries size:4}

2015-02-21 10:18:47,996 | INFO | lt-dispatcher-15 | LocalActorRef | 164 - com.typesafe.akka.slf4j - 2.3.4 | Message [akka.dispatch.sysmsg.Suspend] from Actorakka://opendaylight-cluster-data/user/shardmanager-operational/member-1-shard-inventory-operational/shard-createSnapshot1#-225575473 to Actorakka://opendaylight-cluster-data/user/shardmanager-operational/member-1-shard-inventory-operational/shard-createSnapshot1#-225575473 was not delivered. [9] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2015-02-21 10:18:47,997 | WARN | ult-dispatcher-8 | ShardManager | 178 - org.opendaylight.controller.sal-distributed-datastore - 1.2.0.SNAPSHOT | Supervisor Strategy caught unexpected exception - resuming
java.lang.IllegalArgumentException: fromIndex(0) > toIndex(-19998)
at java.util.ArrayList.subListRangeCheck(ArrayList.java:964)[:1.7.0_67]
at java.util.ArrayList.subList(ArrayList.java:954)[:1.7.0_67]
at org.opendaylight.controller.cluster.raft.AbstractReplicatedLogImpl.snapshotPreCommit(AbstractReplicatedLogImpl.java:192)[171:org.opendaylight.controller.sal-akka-raft:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCaptureSnapshotReply(RaftActor.java:682)[171:org.opendaylight.controller.sal-akka-raft:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:365)[171:org.opendaylight.controller.sal-akka-raft:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:39)[170:org.opendaylight.controller.sal-clustering-commons:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.onReceiveCommand(Shard.java:282)[178:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(Eventsourced.scala:430)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)[163:com.typesafe.akka.actor:2.3.4]
at akka.persistence.Recovery$State$class.process(Recovery.scala:30)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.ProcessorImpl$$anon$2.process(Processor.scala:103)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.ProcessorImpl$$anon$2.aroundReceive(Processor.scala:114)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.Recovery$class.aroundReceive(Recovery.scala:256)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(Eventsourced.scala:428)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.Eventsourced$$anon$2.doAroundReceive(Eventsourced.scala:82)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.Eventsourced$$anon$2.aroundReceive(Eventsourced.scala:78)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:369)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.persistence.UntypedPersistentActor.aroundReceive(Eventsourced.scala:428)[168:com.typesafe.akka.persistence.experimental:2.3.4]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)[163:com.typesafe.akka.actor:2.3.4]
at akka.actor.ActorCell.invoke(ActorCell.scala:487)[163:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)[163:com.typesafe.akka.actor:2.3.4]
at akka.dispatch.Mailbox.run(Mailbox.scala:220)[163:com.typesafe.akka.actor:2.3.4]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_67]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_67]
2015-02-21 10:18:47,999 | WARN | ult-dispatcher-9 | OneForOneStrategy | 164 - com.typesafe.akka.slf4j - 2.3.4 | fromIndex(0) > toIndex(-19998)

OpenDaylight JIRA

[CONTROLLER-1169] Clustering : Snapshotting logic throws an exception when creating snapshot with replicatedLogIndex=-1