[#CONTROLLER-1610] InvalidActorNameException

[CONTROLLER-1610] InvalidActorNameException Created: 17/Feb/17 Updated: 25/Jul/23 Resolved: 24/Mar/17
Status:	Resolved
Project:	controller
Component/s:	clustering
Affects Version/s:	None
Fix Version/s:	None

Type:

Bug

Reporter:

Claudio David Gasparini

Assignee:

Unassigned

Resolution:

Done

Votes:

Labels:

None

Remaining Estimate:

Not Specified

Time Spent:

Not Specified

Original Estimate:

Not Specified

Environment:

Operating System: All
Platform: All

Attachments:

odl1_karaf.log.gz

Issue Links:

Blocks
blocks	BGPCEP-392	BGP scaling target not met in 3-node ...	Confirmed

External issue ID:

7814

Description

Issue observed during bgpcep-csit-3node-periodic-bgpclustering-only-carbon

2017-02-17 08:37:34,007 | WARN | lt-dispatcher-23 | BrokerFacade | 285 - org.opendaylight.netconf.sal-rest-connector - 1.5.0.SNAPSHOT | Exception by reading OPERATIONAL via Restconf: /(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)network-topology/topology/topology[

{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)topology-id=example-ipv4-topology}

]
ReadFailedException{message=Error executeRead ReadData for path /(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)network-topology/topology/topology[

{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)topology-id=example-ipv4-topology}

], errorList=[RpcError [message=Error executeRead ReadData for path /(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)network-topology/topology/topology[

{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)topology-id=example-ipv4-topology}

], severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=java.lang.Exception: Error creating READ_ONLY transaction on shard topology]]}
at org.opendaylight.controller.cluster.datastore.NoOpTransactionContext.executeRead(NoOpTransactionContext.java:67)
at org.opendaylight.controller.cluster.datastore.TransactionProxy$1.invoke(TransactionProxy.java:94)
at org.opendaylight.controller.cluster.datastore.TransactionContextWrapper.executePriorTransactionOperations(TransactionContextWrapper.java:133)
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:232)
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.onCreateTransactionComplete(RemoteTransactionContextSupport.java:195)
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.access$000(RemoteTransactionContextSupport.java:39)
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport$1.onComplete(RemoteTransactionContextSupport.java:135)
at akka.dispatch.OnComplete.internal(Future.scala:258)
at akka.dispatch.OnComplete.internal(Future.scala:256)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:186)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:183)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.Exception: Error creating READ_ONLY transaction on shard topology
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:217)
... 20 more
Caused by: akka.actor.InvalidActorNameException: actor name [shard-member-1:datastore-operational@0:19] is not unique!
at akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.reserve(ChildrenContainer.scala:129)
at akka.actor.dungeon.Children$class.reserveChild(Children.scala:130)
at akka.actor.ActorCell.reserveChild(ActorCell.scala:374)
at akka.actor.dungeon.Children$class.makeChild(Children.scala:268)
at akka.actor.dungeon.Children$class.actorOf(Children.scala:42)
at akka.actor.ActorCell.actorOf(ActorCell.scala:374)
at org.opendaylight.controller.cluster.datastore.ShardTransactionActorFactory.newShardTransaction(ShardTransactionActorFactory.java:78)
at org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.java:656)
at org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.java:644)
at org.opendaylight.controller.cluster.datastore.Shard.handleCreateTransaction(Shard.java:623)
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:289)
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:268)
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:31)
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)
at akka.actor.Actor$class.aroundReceive(Actor.scala:496)
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:664)
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
... 4 more

Comments

Comment by Claudio David Gasparini [ 17/Feb/17 ]

Attachment odl1_karaf.log.gz has been added with description: karaf.log

Comment by Tom Pantelis [ 09/Mar/17 ]

Submitted https://git.opendaylight.org/gerrit/#/c/53100/

Comment by Claudio David Gasparini [ 20/Mar/17 ]

Issue still present

https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/210/archives/odl1_karaf.log.gz

Comment by Tom Pantelis [ 20/Mar/17 ]

A chain ID wasn't unique this time.

Caused by: akka.actor.InvalidActorNameException: actor name [shard-default-member-1:datastore-operational@0:3-121] is not unique!
at akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.reserve(ChildrenContainer.scala:129)
at akka.actor.dungeon.Children$class.reserveChild(Children.scala:130)
at akka.actor.ActorCell.reserveChild(ActorCell.scala:374)
at akka.actor.dungeon.Children$class.makeChild(Children.scala:268)
at akka.actor.dungeon.Children$class.actorOf(Children.scala:42)
at akka.actor.ActorCell.actorOf(ActorCell.scala:374)
at org.opendaylight.controller.cluster.datastore.ShardTransactionActorFactory.newShardTransaction(ShardTransactionActorFactory.java:82)
at org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.java:661)
at org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.java:649)
at org.opendaylight.controller.cluster.datastore.Shard.handleCreateTransaction(Shard.java:626)
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:292)

Comment by Claudio David Gasparini [ 20/Mar/17 ]

(In reply to Tom Pantelis from comment #3)
> A chain ID wasn't unique this time.
>
> Caused by: akka.actor.InvalidActorNameException: actor name
> [shard-default-member-1:datastore-operational@0:3-121] is not unique!
> at
> akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.
> reserve(ChildrenContainer.scala:129)
> at akka.actor.dungeon.Children$class.reserveChild(Children.scala:130)
> at akka.actor.ActorCell.reserveChild(ActorCell.scala:374)
> at akka.actor.dungeon.Children$class.makeChild(Children.scala:268)
> at akka.actor.dungeon.Children$class.actorOf(Children.scala:42)
> at akka.actor.ActorCell.actorOf(ActorCell.scala:374)
> at
> org.opendaylight.controller.cluster.datastore.ShardTransactionActorFactory.
> newShardTransaction(ShardTransactionActorFactory.java:82)
> at
> org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.
> java:661)
> at
> org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.
> java:649)
> at
> org.opendaylight.controller.cluster.datastore.Shard.
> handleCreateTransaction(Shard.java:626)
> at
> org.opendaylight.controller.cluster.datastore.Shard.
> handleNonRaftCommand(Shard.java:292)

yeah, sorry. Do you want me to open a different bug and close this one again?

Comment by Tom Pantelis [ 20/Mar/17 ]

No - same underlying issue. We thought adding the shard name would resolve the uniqueness but apparently not completely.

(In reply to Claudio D. Gasparini from comment #4)
> (In reply to Tom Pantelis from comment #3)
> > A chain ID wasn't unique this time.
> >
> > Caused by: akka.actor.InvalidActorNameException: actor name
> > [shard-default-member-1:datastore-operational@0:3-121] is not unique!
> > at
> > akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.
> > reserve(ChildrenContainer.scala:129)
> > at akka.actor.dungeon.Children$class.reserveChild(Children.scala:130)
> > at akka.actor.ActorCell.reserveChild(ActorCell.scala:374)
> > at akka.actor.dungeon.Children$class.makeChild(Children.scala:268)
> > at akka.actor.dungeon.Children$class.actorOf(Children.scala:42)
> > at akka.actor.ActorCell.actorOf(ActorCell.scala:374)
> > at
> > org.opendaylight.controller.cluster.datastore.ShardTransactionActorFactory.
> > newShardTransaction(ShardTransactionActorFactory.java:82)
> > at
> > org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.
> > java:661)
> > at
> > org.opendaylight.controller.cluster.datastore.Shard.createTransaction(Shard.
> > java:649)
> > at
> > org.opendaylight.controller.cluster.datastore.Shard.
> > handleCreateTransaction(Shard.java:626)
> > at
> > org.opendaylight.controller.cluster.datastore.Shard.
> > handleNonRaftCommand(Shard.java:292)
>
> yeah, sorry. Do you want me to open a different bug and close this one again?

Comment by Tom Pantelis [ 20/Mar/17 ]

What exactly does the test do? I see from the log on member-1 that it restarted once, 7 minutes apart:

The InvalidActorNameException occurred at 2017-03-20 07:29:04 after member-2 became the leader again. However the front-end ID in the generated name "shard-default-member-1:datastore-operational@0:3-121" was still 0. This number is persisted in the snapshots so it should have incremented to 1 after the restart at 07:18:16. I'm wondering if the test cleaned out the journal/snapshot directories prior to the restart - this would explain the front-end ID being 0.

Comment by Robert Varga [ 20/Mar/17 ]

Is the test wiping the entire persisted state?

Comment by Vratko Polak [ 20/Mar/17 ]

> Is the test wiping the entire persisted state?

Not only persisted state, also whole data/ and similar [0].
Basically, the job runs suites on default configuration (each shard replicated on each member), then clears everything up, starts ODL with non-replicated configuration, and repeats the suites.

Possibly the list [0] is not complete, and some state from the first part survives the cleanup.

Note that there are failures already in the first part [1], possibly ~~CONTROLLER-1615~~.

[0] https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/210/archives/log.html.gz#s1-s5-t2-k2-k2-k1-k2-k1-k3-k1-k3-k1-k1
[1] https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/210/archives/log.html.gz#s1-s2-t6-k3-k1-k3-k7-k1

Comment by Tom Pantelis [ 22/Mar/17 ]

Submitted 2nd patch which will fix it once and for all: https://git.opendaylight.org/gerrit/#/c/53646/

Comment by Claudio David Gasparini [ 24/Mar/17 ]

(In reply to Tom Pantelis from comment #9)
> Submitted 2nd patch which will fix it once and for all:
> https://git.opendaylight.org/gerrit/#/c/53646/

seems that it doesn't want to go. I saw this error, not sure if still the same issue.

https://logs.opendaylight.org/sandbox/jenkins091/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/17/archives/odl1_karaf.log.gz

{test-device}

: Slave mount point registered.
2017-03-24 08:18:41,744 | ERROR | on-dispatcher-45 | DataTreeChangeListenerActor | 226 - org.opendaylight.controller.sal-clustering-commons - 1.5.0.SNAPSHOT | Error notifying listener org.opendaylight.controller.md.sal.binding.impl.BindingClusteredDOMDataTreeChangeListenerAdapter@322294da
akka.actor.InvalidActorNameException: actor name [test-device] is not unique!
at akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.reserve(ChildrenContainer.scala:129)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.dungeon.Children$class.reserveChild(Children.scala:130)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.ActorCell.reserveChild(ActorCell.scala:374)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.dungeon.Children$class.makeChild(Children.scala:268)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.dungeon.Children$class.attachChild(Children.scala:46)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.ActorCell.attachChild(ActorCell.scala:374)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.ActorSystemImpl.actorOf(ActorSystem.scala:732)[200:com.typesafe.akka.actor:2.4.17]
at org.opendaylight.netconf.topology.singleton.impl.NetconfNodeManager.createActorRef(NetconfNodeManager.java:137)[345:org.opendaylight.netconf.topology-singleton:1.2.0.SNAPSHOT]
at org.opendaylight.netconf.topology.singleton.impl.NetconfNodeManager.handleSlaveMountPoint(NetconfNodeManager.java:124)[345:org.opendaylight.netconf.topology-singleton:1.2.0.SNAPSHOT]
at org.opendaylight.netconf.topology.singleton.impl.NetconfNodeManager.onDataTreeChanged(NetconfNodeManager.java:82)[345:org.opendaylight.netconf.topology-singleton:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.binding.impl.BindingDOMDataTreeChangeListenerAdapter.onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41)[188:org.opendaylight.controller.sal-binding-broker-impl:1.5.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataChanged(DataTreeChangeListenerActor.java:56)[233:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:37)[233:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:28)[226:org.opendaylight.controller.sal-clustering-commons:1.5.0.SNAPSHOT]
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.Actor$class.aroundReceive(Actor.scala:497)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[200:com.typesafe.akka.actor:2.4.17]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[200:com.typesafe.akka.actor:2.4.17]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[200:com.typesafe.akka.actor:2.4.17]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[200:com.typesafe.akka.actor:2.4.17]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]

Comment by Claudio David Gasparini [ 24/Mar/17 ]

(In reply to Claudio D. Gasparini from comment #10)
> (In reply to Tom Pantelis from comment #9)
> > Submitted 2nd patch which will fix it once and for all:
> > https://git.opendaylight.org/gerrit/#/c/53646/
>
> seems that it doesn't want to go. I saw this error, not sure if still the
> same issue.
>
> https://logs.opendaylight.org/sandbox/jenkins091/bgpcep-csit-3node-periodic-
> bgpclustering-only-carbon/17/archives/odl1_karaf.log.gz
>
>
> 2017-03-24 08:18:38,698 | INFO | lt-dispatcher-53 | SlaveSalFacade
> | 345 - org.opendaylight.netconf.topology-singleton - 1.2.0.SNAPSHOT |
> RemoteDevice

{test-device}

: Slave mount point registered.
> 2017-03-24 08:18:41,744 | ERROR | on-dispatcher-45 |
> DataTreeChangeListenerActor | 226 -
> org.opendaylight.controller.sal-clustering-commons - 1.5.0.SNAPSHOT | Error
> notifying listener
> org.opendaylight.controller.md.sal.binding.impl.
> BindingClusteredDOMDataTreeChangeListenerAdapter@322294da
> akka.actor.InvalidActorNameException: actor name [test-device] is not unique!
> at
> akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.
> reserve(ChildrenContainer.scala:129)[200:com.typesafe.akka.actor:2.4.17]
> at
> akka.actor.dungeon.Children$class.reserveChild(Children.scala:130)[200:com.
> typesafe.akka.actor:2.4.17]
> at
> akka.actor.ActorCell.reserveChild(ActorCell.scala:374)[200:com.typesafe.akka.
> actor:2.4.17]
> at
> akka.actor.dungeon.Children$class.makeChild(Children.scala:268)[200:com.
> typesafe.akka.actor:2.4.17]
> at
> akka.actor.dungeon.Children$class.attachChild(Children.scala:46)[200:com.
> typesafe.akka.actor:2.4.17]
> at
> akka.actor.ActorCell.attachChild(ActorCell.scala:374)[200:com.typesafe.akka.
> actor:2.4.17]
> at
> akka.actor.ActorSystemImpl.actorOf(ActorSystem.scala:732)[200:com.typesafe.
> akka.actor:2.4.17]
> at
> org.opendaylight.netconf.topology.singleton.impl.NetconfNodeManager.
> createActorRef(NetconfNodeManager.java:137)[345:org.opendaylight.netconf.
> topology-singleton:1.2.0.SNAPSHOT]
> at
> org.opendaylight.netconf.topology.singleton.impl.NetconfNodeManager.
> handleSlaveMountPoint(NetconfNodeManager.java:124)[345:org.opendaylight.
> netconf.topology-singleton:1.2.0.SNAPSHOT]
> at
> org.opendaylight.netconf.topology.singleton.impl.NetconfNodeManager.
> onDataTreeChanged(NetconfNodeManager.java:82)[345:org.opendaylight.netconf.
> topology-singleton:1.2.0.SNAPSHOT]
> at
> org.opendaylight.controller.md.sal.binding.impl.
> BindingDOMDataTreeChangeListenerAdapter.
> onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41)[188:org.
> opendaylight.controller.sal-binding-broker-impl:1.5.0.SNAPSHOT]
> at
> org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.
> dataChanged(DataTreeChangeListenerActor.java:56)[233:org.opendaylight.
> controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
> at
> org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.
> handleReceive(DataTreeChangeListenerActor.java:37)[233:org.opendaylight.
> controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
> at
> org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.
> onReceive(AbstractUntypedActor.java:28)[226:org.opendaylight.controller.sal-
> clustering-commons:1.5.0.SNAPSHOT]
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:
> 165)[200:com.typesafe.akka.actor:2.4.17]
> at
> akka.actor.Actor$class.aroundReceive(Actor.scala:497)[200:com.typesafe.akka.
> actor:2.4.17]
> at
> akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)[200:com.
> typesafe.akka.actor:2.4.17]
> at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[200:com.typesafe.
> akka.actor:2.4.17]
> at
> akka.actor.ActorCell.invoke(ActorCell.scala:495)[200:com.typesafe.akka.actor:
> 2.4.17]
> at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[200:com.typesafe.
> akka.actor:2.4.17]
> at
> akka.dispatch.Mailbox.run(Mailbox.scala:224)[200:com.typesafe.akka.actor:2.4.
> 17]
> at
> akka.dispatch.Mailbox.exec(Mailbox.scala:234)[200:com.typesafe.akka.actor:2.
> 4.17]
> at
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[196:org.
> scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:
> 1339)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[196:
> org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:
> 107)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]

Before this happens I see

2017-03-24 08:18:25,334 | WARN | lt-dispatcher-19 | FrontendClientMetadataBuilder | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Unknown history for commited transaction member-1-entity-ownership-internal-fe-0-txn-9-0, ignoring
2017-03-24 08:18:25,451 | ERROR | on-dispatcher-46 | DataTreeChangeListenerActor | 226 - org.opendaylight.controller.sal-clustering-commons - 1.5.0.SNAPSHOT | Error notifying listener org.opendaylight.controller.md.sal.binding.impl.BindingClusteredDOMDataTreeChangeListenerAdapter@4321baf2
java.lang.RuntimeException: com.google.common.base.VerifyException
at org.opendaylight.mdsal.singleton.dom.impl.ClusterSingletonServiceGroupImpl.registerService(ClusterSingletonServiceGroupImpl.java:180)[182:org.opendaylight.mdsal.singleton-dom-impl:2.2.0.SNAPSHOT]
at org.opendaylight.mdsal.singleton.dom.impl.AbstractClusterSingletonServiceProviderImpl.registerClusterSingletonService(AbstractClusterSingletonServiceProviderImpl.java:107)[182:org.opendaylight.mdsal.singleton-dom-impl:2.2.0.SNAPSHOT]
at Proxy5a0b2628_9688_43ca_90af_1be2b0d547a8.registerClusterSingletonService(Unknown Source)[:]
at Proxy189d85ea_98c4_4191_b85a_3e3d52631df1.registerClusterSingletonService(Unknown Source)[:]
at org.opendaylight.netconf.topology.singleton.impl.NetconfTopologyManager.startNetconfDeviceContext(NetconfTopologyManager.java:156)[345:org.opendaylight.netconf.topology-singleton:1.2.0.SNAPSHOT]
at org.opendaylight.netconf.topology.singleton.impl.NetconfTopologyManager.onDataTreeChanged(NetconfTopologyManager.java:121)[345:org.opendaylight.netconf.topology-singleton:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.binding.impl.BindingDOMDataTreeChangeListenerAdapter.onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41)[188:org.opendaylight.controller.sal-binding-broker-impl:1.5.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.dataChanged(DataTreeChangeListenerActor.java:56)[233:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.handleReceive(DataTreeChangeListenerActor.java:37)[233:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:28)[226:org.opendaylight.controller.sal-clustering-commons:1.5.0.SNAPSHOT]
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.Actor$class.aroundReceive(Actor.scala:497)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[200:com.typesafe.akka.actor:2.4.17]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[200:com.typesafe.akka.actor:2.4.17]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[200:com.typesafe.akka.actor:2.4.17]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[200:com.typesafe.akka.actor:2.4.17]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[200:com.typesafe.akka.actor:2.4.17]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
Caused by: com.google.common.base.VerifyException
at com.google.common.base.Verify.verify(Verify.java:98)[84:com.google.guava:18.0.0]
at org.opendaylight.mdsal.singleton.dom.impl.ClusterSingletonServiceGroupImpl.registerService(ClusterSingletonServiceGroupImpl.java:171)[182:org.opendaylight.mdsal.singleton-dom-impl:2.2.0.SNAPSHOT]
... 21 more

Comment by Tom Pantelis [ 24/Mar/17 ]

That exception is emanating from the ClusterSingletonServiceGroupImpl not related to this bug. That is in the mdsal project so any bug should be created there.

(In reply to Claudio D. Gasparini from comment #11)
> Before this happens I see
>
> 2017-03-24 08:18:25,334 | WARN | lt-dispatcher-19 |
> FrontendClientMetadataBuilder | 233 -
> org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT |
> Unknown history for commited transaction
> member-1-entity-ownership-internal-fe-0-txn-9-0, ignoring
> 2017-03-24 08:18:25,451 | ERROR | on-dispatcher-46 |
> DataTreeChangeListenerActor | 226 -
> org.opendaylight.controller.sal-clustering-commons - 1.5.0.SNAPSHOT | Error
> notifying listener
> org.opendaylight.controller.md.sal.binding.impl.
> BindingClusteredDOMDataTreeChangeListenerAdapter@4321baf2
> java.lang.RuntimeException: com.google.common.base.VerifyException
> at
> org.opendaylight.mdsal.singleton.dom.impl.ClusterSingletonServiceGroupImpl.
> registerService(ClusterSingletonServiceGroupImpl.java:180)[182:org.
> opendaylight.mdsal.singleton-dom-impl:2.2.0.SNAPSHOT]
> at
> org.opendaylight.mdsal.singleton.dom.impl.
> AbstractClusterSingletonServiceProviderImpl.
> registerClusterSingletonService(AbstractClusterSingletonServiceProviderImpl.
> java:107)[182:org.opendaylight.mdsal.singleton-dom-impl:2.2.0.SNAPSHOT]
> at
> Proxy5a0b2628_9688_43ca_90af_1be2b0d547a8.
> registerClusterSingletonService(Unknown Source)[:]
> at
> Proxy189d85ea_98c4_4191_b85a_3e3d52631df1.
> registerClusterSingletonService(Unknown Source)[:]
> at
> org.opendaylight.netconf.topology.singleton.impl.NetconfTopologyManager.
> startNetconfDeviceContext(NetconfTopologyManager.java:156)[345:org.
> opendaylight.netconf.topology-singleton:1.2.0.SNAPSHOT]
> at
> org.opendaylight.netconf.topology.singleton.impl.NetconfTopologyManager.
> onDataTreeChanged(NetconfTopologyManager.java:121)[345:org.opendaylight.
> netconf.topology-singleton:1.2.0.SNAPSHOT]
> at
> org.opendaylight.controller.md.sal.binding.impl.
> BindingDOMDataTreeChangeListenerAdapter.
> onDataTreeChanged(BindingDOMDataTreeChangeListenerAdapter.java:41)[188:org.
> opendaylight.controller.sal-binding-broker-impl:1.5.0.SNAPSHOT]
> at
> org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.
> dataChanged(DataTreeChangeListenerActor.java:56)[233:org.opendaylight.
> controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
> at
> org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor.
> handleReceive(DataTreeChangeListenerActor.java:37)[233:org.opendaylight.
> controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
> at
> org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.
> onReceive(AbstractUntypedActor.java:28)[226:org.opendaylight.controller.sal-
> clustering-commons:1.5.0.SNAPSHOT]
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:
> 165)[200:com.typesafe.akka.actor:2.4.17]
> at
> akka.actor.Actor$class.aroundReceive(Actor.scala:497)[200:com.typesafe.akka.
> actor:2.4.17]
> at
> akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)[200:com.
> typesafe.akka.actor:2.4.17]
> at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[200:com.typesafe.
> akka.actor:2.4.17]
> at
> akka.actor.ActorCell.invoke(ActorCell.scala:495)[200:com.typesafe.akka.actor:
> 2.4.17]
> at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[200:com.typesafe.
> akka.actor:2.4.17]
> at
> akka.dispatch.Mailbox.run(Mailbox.scala:224)[200:com.typesafe.akka.actor:2.4.
> 17]
> at
> akka.dispatch.Mailbox.exec(Mailbox.scala:234)[200:com.typesafe.akka.actor:2.
> 4.17]
> at
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[196:org.
> scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:
> 1339)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[196:
> org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:
> 107)[196:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
> Caused by: com.google.common.base.VerifyException
> at
> com.google.common.base.Verify.verify(Verify.java:98)[84:com.google.guava:18.
> 0.0]
> at
> org.opendaylight.mdsal.singleton.dom.impl.ClusterSingletonServiceGroupImpl.
> registerService(ClusterSingletonServiceGroupImpl.java:171)[182:org.
> opendaylight.mdsal.singleton-dom-impl:2.2.0.SNAPSHOT]
> ... 21 more

Generated at Wed Feb 07 19:55:59 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.

[CONTROLLER-1610] InvalidActorNameException Created: 17/Feb/17 Updated: 25/Jul/23 Resolved: 24/Mar/17