|
This looks like some trouble was encountered by odl2 just around the time that transaction was allocated:
ODL1:
2017-03-28 11:19:14,044 | DEBUG | lt-dispatcher-23 | ShardDataTreeTransactionChain | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Allocated read-write transaction member-2-datastore-operational-fe-0-chn-5-txn-585-0 snapshot org.opendaylight.yangtools.yang.data.api.schema.tree.spi.Version@274968e9
2017-03-28 11:19:14,051 | WARN | ult-dispatcher-4 | ClusterCoreDaemon | 201 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@10.29.13.68:2550] - Marking node(s) as UNREACHABLE [Member(address = akka.tcp://opendaylight-cluster-data@10.29.12.39:2550, status = Up), Member(address = akka.tcp://opendaylight-cluster-data@10.29.13.13:2550, status = Up)]. Node roles [member-1]
2017-03-28 11:19:14,051 | INFO | ult-dispatcher-4 | ShardManager | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Received UnreachableMember: memberName MemberName
{name=member-2}, address: akka.tcp://opendaylight-cluster-data@10.29.12.39:2550
2017-03-28 11:19:14,051 | INFO | lt-dispatcher-46 | ShardManager | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Received UnreachableMember: memberName MemberName{name=member-2}
, address: akka.tcp://opendaylight-cluster-data@10.29.12.39:2550
ODL2:
2017-03-28 11:19:10,166 | WARN | on-dispatcher-54 | OperationLimiter | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Failed to acquire operation permit for transaction member-2-datastore-operational-fe-0-chn-5-txn-585-0
2017-03-28 11:19:12,335 | WARN | lt-dispatcher-85 | ClusterCoreDaemon | 201 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@10.29.12.39:2550] - Marking node(s) as UNREACHABLE [Member(address = akka.tcp://opendaylight-cluster-data@10.29.13.68:2550, status = Up)]
. Node roles [member-2]
2017-03-28 11:19:12,335 | INFO | lt-dispatcher-71 | ShardManager | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Received UnreachableMember: memberName MemberName
{name=member-1}, address: akka.tcp://opendaylight-cluster-data@10.29.13.68:2550
2017-03-28 11:19:12,335 | INFO | lt-dispatcher-18 | ShardManager | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Received UnreachableMember: memberName MemberName{name=member-1}
, address: akka.tcp://opendaylight-cluster-data@10.29.13.68:2550
2017-03-28 11:19:12,336 | INFO | lt-dispatcher-71 | EntityOwnershipShard | 226 - org.opendaylight.controller.sal-clustering-commons - 1.5.0.SNAPSHOT | member-2-shard-entity-ownership-operational: onPeerDown: PeerDown [memberName=member-1, peerId=member-1-shard-entity-ownership-operational]
2017-03-28 11:19:14,241 | INFO | lt-dispatcher-22 | EmptyLocalActorRef | 201 - com.typesafe.akka.slf4j - 2.4.17 | Message [org.opendaylight.controller.cluster.datastore.messages.CreateTransactionReply] from Actor[akka.tcp://opendaylight-cluster-data@10.29.13.68:2550/user/shardmanager-operational/membe
r-1-shard-topology-operational#-1491023140] to Actor[akka://opendaylight-cluster-data/temp/$k2b] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
2017-03-28 11:19:14,242 | INFO | lt-dispatcher-22 | kka://opendaylight-cluster-data) | 201 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@10.29.12.39:2550] - Ignoring received gossip status from unreachable [UniqueAddress(akka.tcp://opendaylight-cluster-data@10.29.13.68:2550
,1066496970)]
2017-03-28 11:19:14,264 | WARN | lt-dispatcher-37 | ConcurrentDOMDataBroker | 233 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Tx: DOM-CHAIN-4-585 Error during phase CAN_COMMIT, starting Abort
java.lang.Exception: Error creating READ_WRITE transaction on shard topology
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:217)
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.onCreateTransactionComplete(RemoteTransactionContextSupport.java:195)
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.access$000(RemoteTransactionContextSupport.java:39)
at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport$1.onComplete(RemoteTransactionContextSupport.java:135)
at akka.dispatch.OnComplete.internal(Future.scala:258)
at akka.dispatch.OnComplete.internal(Future.scala:256)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:186)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:183)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.IllegalStateException: Transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-operational-fe-0-chn-5-txn-585-0, closed=false, snapshot=MutableDataTree [modification=NodeModification [identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, modificationType=NONE, childModification=
{
}]]}
is open
|