[CONTROLLER-711] Flow Statistics manager is updating store with incorrect key Created: 20/Aug/14  Updated: 01/Dec/14  Resolved: 01/Dec/14

Status: Resolved
Project: controller
Component/s: mdsal
Affects Version/s: Helium
Fix Version/s: None

Type: Bug
Reporter: Amit Mandke Assignee: Anil Vishnoi
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Mac OS
Platform: PC

External issue ID: 1593

 Description   

Following exception keeps flooding the logs once flows are installed on mininet. It looks like flows are being updated in a data store using incorrect key. Further analysis is required.

2014-08-20 11:07:40.558 PDT [CommitFutures-1] ERROR o.o.c.m.s.b.i.ForwardedBackwardsCompatibleDataBroker - Transaction DOM-7890 failed to complete
org.opendaylight.controller.md.sal.common.api.data.TransactionCommitFailedException: Data did not pass validation.
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:371) ~[na:na]
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:357) ~[na:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ModifiedNodeDoesNotExistException: Node (urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08
-19)node/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:1}

]/AugmentationIdentifier

{childNames=[(urn:opendaylight:flow:inventory?revisio n=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08 -19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)meter, (urn:opendaylight:flow:inventory?revision=2013-08-19)tabl e, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported- instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features]}

/(urn:opendaylight:flow:inventory?revision=201
3-08-19)table/(urn:opendaylight:flow:inventory?revision=2013-08-19)table[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=0}

]/(urn:opendaylight:flow:inventory?revision=2013-08-19)flow/(urn:opend
aylight:flow:inventory?revision=2013-08-19)flow[

{(urn:opendaylight:flow:inventory?revision=2013-08-19)id=#UF$TABLE*0-1}

] does not exist. Cannot apply modification to its children.
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:170) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na
:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:184) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:174) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:131) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTree.validate(InMemoryDataTree.java:89) ~[na:na]
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:361) ~[na:na]
... 5 common frames omitted

 Comments   
Comment by Anil Vishnoi [ 21/Aug/14 ]

Hi Amit,

I pushed the fix through following gerrit

https://git.opendaylight.org/gerrit/10135

Can you please pull this patch and test.

Thanks
Anil

Comment by Amit Mandke [ 21/Aug/14 ]

I tested with this patch I don't get the following exception anymore. However now it is actually writing to store it is generating lot of warnings. May not be issue but just an observation:

2014-08-21 11:58:49.295 PDT [pool-10-thread-1] WARN o.o.c.m.s.d.s.i.InMemoryDOMDataStore - Store Tx: DOM-OPER-172 Conflicting modification for (urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:inventory?revision=2013-08-19)node/(urn:opendaylight:inventory?revision=2013-08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:2}

]/(urn:opendaylight:inventory?revision=2013-08-19)node-connector/(urn:opendaylight:inventory?revision=2013-08-19)node-connector[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:2:2}

].
2014-08-21 11:58:49.343 PDT [md-sal-binding-notification-8] INFO o.o.l.flow.FlowWriterServiceImpl - In addMacToMacFlowsUsingShortestPath: No flows added. Source and Destination ports are same.
2014-08-21 11:58:49.344 PDT [WriteTxCommit-0] WARN o.o.c.m.s.d.b.i.DOMDataCommitCoordinatorImpl - Tx: DOM-170 Error during phase CAN_COMMIT, starting Abort
org.opendaylight.controller.md.sal.common.api.data.OptimisticLockFailedException: Optimistic lock failed.
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:366) ~[na:na]
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:356) ~[na:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: org.opendaylight.yangtools.yang.data.api.schema.tree.ConflictingModificationAppliedException: Node was replaced by other transaction.
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:79) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkNotConflicting(SchemaAwareApplyOperation.java:105) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:164) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:133) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:171) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkMergeApplicable(NormalizedNodeContainerModificationStrategy.java:179) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:136) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:171) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:161) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:130) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:171) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:161) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:130) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:171) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:161) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:130) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:171) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.NormalizedNodeContainerModificationStrategy.checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrategy.java:161) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:130) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.checkApplicable(RootModificationApplyOperation.java:72) ~[na:na]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTree.validate(InMemoryDataTree.java:89) ~[na:na]
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:360) ~[na:na]
... 5 common frames omitted

Comment by Amit Mandke [ 21/Aug/14 ]

I meant I don't get the previous exception any more...

(In reply to Amit Mandke from comment #2)
> I tested with this patch I don't get the following exception anymore.
> However now it is actually writing to store it is generating lot of
> warnings. May not be issue but just an observation:
>
> 2014-08-21 11:58:49.295 PDT [pool-10-thread-1] WARN
> o.o.c.m.s.d.s.i.InMemoryDOMDataStore - Store Tx: DOM-OPER-172 Conflicting
> modification for
> (urn:opendaylight:inventory?revision=2013-08-19)nodes/(urn:opendaylight:
> inventory?revision=2013-08-19)node/(urn:opendaylight:inventory?revision=2013-
> 08-19)node[

{(urn:opendaylight:inventory?revision=2013-08-19)id=openflow:2}

]/
> (urn:opendaylight:inventory?revision=2013-08-19)node-connector/(urn:
> opendaylight:inventory?revision=2013-08-19)node-connector[

{(urn:opendaylight: > inventory?revision=2013-08-19)id=openflow:2:2}

].
> 2014-08-21 11:58:49.343 PDT [md-sal-binding-notification-8] INFO
> o.o.l.flow.FlowWriterServiceImpl - In addMacToMacFlowsUsingShortestPath: No
> flows added. Source and Destination ports are same.
> 2014-08-21 11:58:49.344 PDT [WriteTxCommit-0] WARN
> o.o.c.m.s.d.b.i.DOMDataCommitCoordinatorImpl - Tx: DOM-170 Error during
> phase CAN_COMMIT, starting Abort
> org.opendaylight.controller.md.sal.common.api.data.
> OptimisticLockFailedException: Optimistic lock failed.
> at
> org.opendaylight.controller.md.sal.dom.store.impl.
> InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:
> 366) ~[na:na]
> at
> org.opendaylight.controller.md.sal.dom.store.impl.
> InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:
> 356) ~[na:na]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_67]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
> 1145) [na:1.7.0_67]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:
> 615) [na:1.7.0_67]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
> Caused by:
> org.opendaylight.yangtools.yang.data.api.schema.tree.
> ConflictingModificationAppliedException: Node was replaced by other
> transaction.
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:
> 79) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkNotConflicting(SchemaAwareApplyOperation.java:
> 105) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.
> java:164) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:
> 133) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:
> 171) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkMergeApplicable(NormalizedNodeContainerModificationStrategy.java:179)
> ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:
> 136) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:
> 171) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrateg
> y.java:161) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:
> 130) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:
> 171) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrateg
> y.java:161) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:
> 130) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:
> 171) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrateg
> y.java:161) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:
> 130) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkChildPreconditions(NormalizedNodeContainerModificationStrategy.java:
> 171) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> NormalizedNodeContainerModificationStrategy.
> checkSubtreeModificationApplicable(NormalizedNodeContainerModificationStrateg
> y.java:161) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:
> 130) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.
> RootModificationApplyOperation.
> checkApplicable(RootModificationApplyOperation.java:72) ~[na:na]
> at
> org.opendaylight.yangtools.yang.data.impl.schema.tree.InMemoryDataTree.
> validate(InMemoryDataTree.java:89) ~[na:na]
> at
> org.opendaylight.controller.md.sal.dom.store.impl.
> InMemoryDOMDataStore$ThreePhaseCommitImpl$1.call(InMemoryDOMDataStore.java:
> 360) ~[na:na]
> ... 5 common frames omitted

Comment by Anil Vishnoi [ 22/Aug/14 ]

Hi Amit,

I tried to recreate this issue locally with latest controller and openflowplugin master. I connect mininet switches to the controller and wait for few statistics update cycle to see if exceptions occur, but i don't see any exception in that duration. Then I exit mininet and all switch get disconnected, at this moment i see the new exception you mentioned. My guess is this is happening because by the time data store process the transaction submitted for the node, that node get removed from the data store. So this new exception is not related to the issue reported in this bug or the fix. I think there is a bug opened for this exception as well, I will dig and update if i find the bug.

Thanks
Anil

Generated at Wed Feb 07 19:53:41 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.