[YANGTOOLS-1404] Deviation of augmented node causes NPE Created: 03/Feb/22 Updated: 02/Mar/22 Resolved: 02/Mar/22 |
|
| Status: | Resolved |
| Project: | yangtools |
| Component/s: | model-util |
| Affects Version/s: | 6.0.0, 7.0.0, 7.0.9, 6.0.12, 7.0.14 |
| Fix Version/s: | 8.0.0, 7.0.15 |
| Type: | Bug | Priority: | Medium |
| Reporter: | Sangwook Ha | Assignee: | Robert Varga |
| Resolution: | Done | Votes: | 0 |
| Labels: | pt | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
An augmented data node changed by deviation (not-supported) can cause a null pointer exception or fail to verify non-null requirement while writing NormalizedNode. For example, the following 3 models together have two leaves (bar & qux) under foo - baz is removed by deviate.yang: orig.yang module orig {
namespace "urn:orig";
prefix orig;
container foo {
leaf bar {
type string;
}
}
}
aug.yang module aug {
namespace "urn:aug";
prefix aug;
import orig {
prefix orig;
}
augment /orig:foo {
leaf baz {
type string;
}
leaf qux {
type string;
}
}
}
deviate.yang module deviate {
namespace "urn:deviate";
prefix dev;
import orig {
prefix orig;
}
import aug {
prefix aug;
}
deviation /orig:foo/aug:baz {
deviate not-supported;
}
}
When PUT request is submitted like below: PUT /rests/data/network-topology:network-topology/topology=topology-netconf/node=node1/yang-ext:mount/orig:foo
{
"orig:foo": {
"bar": "apple",
"aug:qux": "orange"
}
}
This causes the following exception although status code of 201 is returned by the controller: yangtools 7.0.12 2022-02-03T23:12:08,009 | ERROR | opendaylight-cluster-data-akka.actor.default-dispatcher-4 | OneForOneStrategy | 213 - org.opendaylight.controller.repackaged-akka - 4.0.8 | No child matching (urn:aug)baz found com.google.common.base.VerifyException: No child matching (urn:aug)baz found at com.google.common.base.Verify.verify(Verify.java:124) ~[bundleFile:?] at com.google.common.base.Verify.verifyNotNull(Verify.java:500) ~[bundleFile:?] at org.opendaylight.yangtools.yang.model.api.DataNodeContainer.getDataChildByName(DataNodeContainer.java:84) ~[bundleFile:?] at org.opendaylight.yangtools.yang.model.util.EffectiveAugmentationSchema.create(EffectiveAugmentationSchema.java:67) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.util.NormalizedNodeStreamWriterStack.startAugmentationNode(NormalizedNodeStreamWriterStack.java:301) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.codec.xml.SchemaAwareXMLStreamNormalizedNodeStreamWriter.startAugmentationNode(SchemaAwareXMLStreamNormalizedNodeStreamWriter.java:136) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.ForwardingNormalizedNodeStreamWriter.startAugmentationNode(ForwardingNormalizedNodeStreamWriter.java:87) ~[bundleFile:?] at org.opendaylight.yangtools.rfc7952.data.util.NormalizedNodeStreamWriterMetadataDecorator.startAugmentationNode(NormalizedNodeStreamWriterMetadataDecorator.java:121) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:222) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:102) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.writeChildren(NormalizedNodeWriter.java:189) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:205) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:102) ~[bundleFile:?] at org.opendaylight.yangtools.rfc7952.data.util.NormalizedMetadataWriter.write(NormalizedMetadataWriter.java:114) ~[bundleFile:?] at org.opendaylight.netconf.util.NetconfUtil.writeNormalizedNode(NetconfUtil.java:182) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfMessageTransformUtil.createEditConfigAnyxml(NetconfMessageTransformUtil.java:377) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfRpcStructureTransformer.createEditConfigStructure(NetconfRpcStructureTransformer.java:64) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfBaseOps.createEditConfigStructure(NetconfBaseOps.java:431) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.sal.AbstractNetconfDataTreeService.replace(AbstractNetconfDataTreeService.java:297) ~[bundleFile:?] at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfDataTreeServiceActor.onReceive(NetconfDataTreeServiceActor.java:104) ~[?:?] at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:332) ~[bundleFile:?] at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?] at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[bundleFile:?] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [bundleFile:?] at akka.actor.ActorCell.invoke(ActorCell.scala:548) [bundleFile:?] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [bundleFile:?] at akka.dispatch.Mailbox.run(Mailbox.scala:231) [bundleFile:?] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [bundleFile:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) [?:?] at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) [?:?] yangtools 6.0.12: 2022-02-03T23:47:18,614 | ERROR | opendaylight-cluster-data-akka.actor.default-dispatcher-6 | OneForOneStrategy | 212 - org.opendaylight.controller.repackaged-akka - 3.0.16 | null java.lang.NullPointerException: null at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:878) ~[bundleFile:?] at com.google.common.collect.ImmutableSet.construct(ImmutableSet.java:199) ~[bundleFile:?] at com.google.common.collect.ImmutableSet.copyOf(ImmutableSet.java:236) ~[bundleFile:?] at org.opendaylight.yangtools.yang.model.util.EffectiveAugmentationSchema.<init>(EffectiveAugmentationSchema.java:46) ~[bundleFile:?] at org.opendaylight.yangtools.yang.model.util.EffectiveAugmentationSchema.create(EffectiveAugmentationSchema.java:69) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.impl.codec.SchemaTracker.startAugmentationNode(SchemaTracker.java:262) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.codec.xml.SchemaAwareXMLStreamNormalizedNodeStreamWriter.startAugmentationNode(SchemaAwareXMLStreamNormalizedNodeStreamWriter.java:133) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.ForwardingNormalizedNodeStreamWriter.startAugmentationNode(ForwardingNormalizedNodeStreamWriter.java:87) ~[bundleFile:?] at org.opendaylight.yangtools.rfc7952.data.util.NormalizedNodeStreamWriterMetadataDecorator.startAugmentationNode(NormalizedNodeStreamWriterMetadataDecorator.java:121) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:228) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:103) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.writeChildren(NormalizedNodeWriter.java:190) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.wasProcessedAsCompositeNode(NormalizedNodeWriter.java:206) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.api.schema.stream.NormalizedNodeWriter.write(NormalizedNodeWriter.java:103) ~[bundleFile:?] at org.opendaylight.yangtools.rfc7952.data.util.NormalizedMetadataWriter.write(NormalizedMetadataWriter.java:114) ~[bundleFile:?] at org.opendaylight.netconf.util.NetconfUtil.writeNormalizedNode(NetconfUtil.java:180) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfMessageTransformUtil.createEditConfigAnyxml(NetconfMessageTransformUtil.java:414) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfRpcStructureTransformer.createEditConfigStructure(NetconfRpcStructureTransformer.java:64) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.util.NetconfBaseOps.createEditConfigStrcture(NetconfBaseOps.java:431) ~[bundleFile:?] at org.opendaylight.netconf.sal.connect.netconf.sal.AbstractNetconfDataTreeService.replace(AbstractNetconfDataTreeService.java:298) ~[bundleFile:?] at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfDataTreeServiceActor.onReceive(NetconfDataTreeServiceActor.java:104) ~[?:?] at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:332) ~[bundleFile:?] at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?] at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) ~[bundleFile:?] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [bundleFile:?] at akka.actor.ActorCell.invoke(ActorCell.scala:548) [bundleFile:?] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [bundleFile:?] at akka.dispatch.Mailbox.run(Mailbox.scala:231) [bundleFile:?] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [bundleFile:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) [?:?] at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) [?:?] Note that the problem goes away If augmentation is done separately for each node, i.e. if the following is used for aug.yang instead: module aug {
namespace "urn:aug";
prefix aug;
import orig {
prefix orig;
}
augment /orig:foo {
leaf baz {
type string;
}
}
augment /orig:foo {
leaf qux {
type string;
}
}
}
|
| Comments |
| Comment by Robert Varga [ 04/Feb/22 ] |
|
Can you provide the corresponding XML payloads, please? |
| Comment by Sangwook Ha [ 04/Feb/22 ] |
|
You mean the payload of the PUT call above? It's like this: <foo xmlns="urn:orig">
<bar>apple</bar>
<qux xmlns="urn:aug">orange</qux>
</foo>
|
| Comment by Robert Varga [ 04/Feb/22 ] |
|
Sorry, I mistook the stack trace for an inbound message. ENOCOFFEE. |
| Comment by Robert Varga [ 04/Feb/22 ] |
|
Hmm, is it happening in a single-node or is the request made to one node and the exception is on the other? Because it looks as though the NormalizedNode structure was created without the deviate and is being interpreted with the deviate, leading to a mismatch. |
| Comment by Sangwook Ha [ 04/Feb/22 ] |
|
In the NormalizedNode, the YangInstanceIdentifier for the AugmentationNode has both augmented leaf nodes, baz, removed by deviate.yang, as well as qux. So when trying to set aug, the process of creating EffectiveAugmentationSchema fails because AugmentationSchemaNode has both baz & qux but its parent DataNodeContainer does not have baz. |
| Comment by Robert Varga [ 08/Feb/22 ] |
|
I strongly believe this has to do with how NETCONF topology operates, hence we need to investigate it there. |
| Comment by Sangwook Ha [ 10/Feb/22 ] |
|
By the time effective statements are generated from the models, augmented effective statements include the 'not-supported' node but schema tree does not. And when creating EffectiveAugmentationSchema it enforces that all the children in the AugemntationSchema exist in its parent container. Both of these are done by yangtools, so I'm not sure how this can be addressed with NETCONF topology. |
| Comment by Robert Varga [ 10/Feb/22 ] |
|
Well, the thing is we parsed the payload without the deviation, hence the AugmentationIdentifier containing 'baz', but then we arrive at the mount point primary and try to send it out – and that primary does not see 'baz'. This related to yang-data treating augmentations specially, which is also an upgrade concern, tracked in some yangtools issue (sorry, takes to long to loot it up), but in a consistent cluster this should not be an issue and we need to understand what netconf-topology-singleton is doing before assigning blame. |
| Comment by Ivan Hrasko [ 24/Feb/22 ] |
|
The problem is reproducible in single node deployment. We are afraid that the problem can be that #fromInstanceId method returns data with identifier including also deviated leaf. Thus we need to first get rid of #fromInstanceId to be sure where the problem is. Lets wait for YANGTOOLS-1392. |
| Comment by Robert Varga [ 01/Mar/22 ] |
|
That is interesting. I strongly suspect this is related to the area All codec paths should be working on top of the latter, really. I suggest looking at code paths to see whether there is a disconnect involving one side using EffectiveAgumentationSchema and the other not. Note this may mean that AugmentationIdentifiers can get out of whack as well – the EffectiveAugmentationSchema view can have fewer children, so cross-referencing them back to the model may be challenging. What is suspect is this bit: final class AugmentationContextNode extends DataContainerContextNode<AugmentationIdentifier> { AugmentationContextNode(final AugmentationSchemaNode augmentation, final DataNodeContainer schema) { super(augmentationIdentifierFrom(augmentation), EffectiveAugmentationSchema.create(augmentation, schema), null); } so we are deriving the identifier from the declared view, but pass down the effective view – and therefore emit AugmentationIdentifier with leaf which was deviated away. |
| Comment by Robert Varga [ 02/Mar/22 ] |
|
sangwookha can you give https://git.opendaylight.org/gerrit/c/yangtools/+/99930 a try, please? |
| Comment by Sangwook Ha [ 02/Mar/22 ] |
|
Yes, the patch works. The following request generated config data as expected: PUT /rests/data/network-topology:network-topology/topology=topology-netconf/node=ncserver/yang-ext:mount/orig:foo
{
"orig:foo": {
"bar": "apple",
"aug:qux": "orange"
}
}
09:02:10.705 TRACE [globalWorkerGroup-3-2] Finished sending request <rpc message-id="m-18" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<edit-config>
<target>
<candidate/>
</target>
<error-option>rollback-on-error</error-option>
<config>
<foo xmlns="urn:orig" xmlns:op="urn:ietf:params:xml:ns:netconf:base:1.0" op:operation="replace">
<bar>apple</bar>
<qux xmlns="urn:aug">orange</qux>
</foo>
</config>
</edit-config>
</rpc>
|