https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/CONTROLLER-1814 controller <p>An issue has been reported from the field, which manifested itself as an OpenFlow transaction which does not close and hits OperationLimiter continuously.</p> CONTROLLER-1814

Datastore transactions fail to converge during partitioning

Bug Highest Resolved Done Robert Varga Robert Varga Mon, 26 Feb 2018 08:41:23 +0000 Thu, 12 Apr 2018 18:31:48 +0000 Wed, 4 Apr 2018 15:15:53 +0000 Boron Carbon Nitrogen Oxygen Carbon Nitrogen Oxygen Fluorine clustering 0 1 <p>Relevant log snippets on leader:</p> <p>2018-01-26 18:21:05,339 | DEBUG | lt-dispatcher-17 | ShardReadWriteTransaction        | 185 - org.opendaylight.controller.sal-clustering-commons - 1.4.3.Boron-SR3 | Got ReceiveTimeout for inactivity - closing transaction member-3-datastore-operational-fe-1-chn-2373-txn-0<br/> 2018-01-26 18:21:05,339 | DEBUG | lt-dispatcher-17 | ShardDataTreeTransactionChain    | 191 - org.opendaylight.controller.sal-distributed-datastore - 1.4.3.Boron-SR3 | Aborted transaction ReadWriteShardDataTreeTransaction{id=member-3-datastore-operational-fe-1-chn-2373-txn-0, closed=true</p> <p>Log messages on frontend:</p> <p>2018-01-26 18:11:41,191 | DEBUG | ntLoopGroup-11-1 | RemoteTransactionContext         | 191 - org.opendaylight.controller.sal-distributed-datastore - 1.4.3.Boron-SR3 | Tx member-3-datastore-operational-fe-1-chn-2373-txn-0 sending 1000 batched modifications, ready: false<br/> 2018-01-26 18:11:46,199 | WARN  | ntLoopGroup-11-1 | RemoteTransactionContext         | 191 - org.opendaylight.controller.sal-distributed-datastore - 1.4.3.Boron-SR3 | Failed to acquire execute operation permit for transaction member-3-datastore-operational-fe-1-chn-2373-txn-0 on actor ActorSelection<span class="error">&#91;Anchor(akka.tcp://opendaylight-cluster-data@10.18.130.52:2550/), Path(/user/shardmanager-operational/member-1-shard-inventory-operational/shard-inventory-member-3:datastore-operational@1:2373-0#-587302156)&#93;</span></p> <p>2018-01-26 18:32:21,289 | WARN  | ntLoopGroup-11-1 | RemoteTransactionContext         | 191 - org.opendaylight.controller.sal-distributed-datastore - 1.4.3.Boron-SR3 | Failed to acquire execute operation permit for transaction member-3-d<br/> atastore-operational-fe-1-chn-2373-txn-0 on actor ActorSelection[Anchor(akka.tcp://opendaylight-cluster-data@10.18.130.52:2550/), Path(/user/shardmanager-operational/member-1-shard-inventory-operational/shard-inventory-member-3:datastore<br/> -operational@1:2373-0#-587302156)]</p> <p> </p> <p>As it turns out OperationLimiter accounting is wrong when faced with a combination of:</p> <ul> <li>Transaction spanning multiple full BatchedModifications</li> <li>AskTimeout of a BatchedModifications request</li> </ul> <p>It relies on the BatchedModificationsReply's content to release permits. Since the request times out, only a single permit is released instead of 1000, leading to the application being throttled and not being able to proceed to submit the transaction.</p> <p>Fluorine: <a href="https://git.opendaylight.org/gerrit/68757" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/68757</a></p> Relates CONTROLLER-1825 Development Rank 0|i03arj: