Operating System: All
Platform: All

This is actually an MD-SAL Lithium issue, when binding aware transaction tries to create parent structure. The BindingDOMWriteTransactionAdapter.ensureParentsByMerge() is also trying to create parent data structure for the data itself (which should not be considered parents).

So the method should probably perform following action:

normalizedPath = normalizedPath.getParent();

Note: This has been fixed in Beryllium, where ensure parents is invoked at AbstractWriteTransaction.ensureParentsByMerge() and the method already contains the getParent() invocation.

One addition:

The "normalizedPath = normalizedPath.getParent();" action should be preceeded by a "if(normalizedPath.getParent() == null) return;" check. Because I think getParent may return a null.

It does not appear that Lithium code has the getParent() method. Suggestions?

Well, getParent() just creates new InstanceIdentifier instance with the same path without the last element:

YangInstanceIdentifier.create(this.path.subList(0, this.path.size() - 1));

So doing the the same thing in BindingDOMWriteTransactionAdapter.ensureParentsByMerge() should do the trick.

Its also possible to stop the loop before the last element is ensured as parent:

while (iterator.hasNext()) {
PathArgument currentArg = iterator.next();
// Skipping last element
if(!iterator.hasNext())

...

Create missingParent=true will never work with mandatory nodes,
since MD-SAL does not have any information how to fill out missing parents,
originally we objected for having this flag and planned to require community to create this parents manually, but community decided it is useful to have this flags, even with limitations, that it may fail on mandatory nodes.

It doesn't fail when writing against the same model in local MD-SAL storage. It only fails when writing to the netconf mounted data store (a ConfD server).
Note that it could be a ConfD Netconf protocol bug (which I asked your comment on in the associated mail). The Netconf server returns a "mandatory nodes missing" after the first edit config, even before the commit/validate.

Are we sure that puting of Interface [_description=parent-interface, _key=InterfaceKey [_name=GigabitEthernet0/8/0], _name=GigabitEthernet0/8/0, _type=class org.opendaylight.yang.gen.v1.urn.ietf.params.xml.ns.yang.iana._if.type.rev140508.EthernetCsmacd, augmentation=[Interface1 [_ipv4=Ipv4 [_enabled=true, _forwarding=true, augmentation=[]], _ipv6=Ipv6 [_enabled=true, _forwarding=true, augmentation=[]]]]]

failed?

Or was there another nested write or put which was trying to first put something inside such interface and/or put interface with only key specified and later
write actual content?

—

Lithium MD-SAL datastore does not enforce mandatory statements (implementation behaviour, patch exists for Be)

As I mentioned createMissingParents = true is bit flawed, that it can safely works only for parents which does not contain any mandatory nodes.

So given that you are using concrete model - you have more knowledge about how that data should be filled out rather then MD-SAL (e.g. we can not infer values for mandatory nodes).

> The Netconf server returns a "mandatory nodes missing" after the first edit
> config, even before the commit/validate.

https://tools.ietf.org/html/rfc6020#section-8.2 section specifies which checks happens at which point - unfortunately it does not specify when mandatory checks are run. I would interpret this they should be done during validate or commit phase if target is candidate, but that is my interpretation and others may interpret it differently.

Yes, the putting of that data failed, in a single transaction. If you look at the netconf trace, the first edit-config carries only the list-entry name, which is rejected by the netconf server (likely because of that config validation ambiguity in the spec), which meant that the second part of the config carrying the mandatory data never got transmitted
Using the same data and code, but against the local store, the sequence of operations is the same, however the first operation is accepted, and then the second delivers the mandatory data.

Also note that in this case the mandatory leafs are not part of any of the node's parent nodes, i.e. they are only mandatory for the node being created.

Hi Wojciech,

Just a quick recap, there are 2 issues here:

1. Creating parent structure with mandatory nodes is not possible because ensure parents does not provide them and should be avoided. (This would work against MD-SAL because it does't enforce mandatory yet) - But this is not your problem because you are creating only a simple parent structure and mandatory nodes are in the actual data you are submitting.

2. Creating parent structure using BA (at least) in Lithium: BA tries to create the parent structure also for the data itself. - This is unnecessary since the data is already present in the request and in your case this causes issues due to mandatory nodes being present in that data.

In your case the fix should be modifying ensure parents in the BA layer to not create parent structure for the data itself and strip the last item from InstanceIdentifier. I think you've tried to use this workaround (suggested earlier), but I don't remember if you were successful. Could you provide the details here ?

Thanks

The issue also appears when attempting to add configuration via RESTCONF. It actually renders the API unusable for config changes.

API :
PUT /config/opendaylight-inventory:nodes/node/172.20.118.63/yang-ext:mount/ietf-interfaces:interfaces/interface/

Input :
{
"interface": [

]
}

Netconf data doesn't carry the mandatory parameters and fails:

2016-02-13 04:03:24.861 GMT+00:00 [nettyThreadgroupModule$NioEventLoopGroupCloseable-7-3] TRACE o.o.c.s.c.n.u.NetconfRpcFutureCallback - RemoteDevice

: Sending message <rpc message-id="m-17" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<edit-config>
<target>
<candidate/>
</target>
<error-option>rollback-on-error</error-option>
<config>
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"/>
</config>
</edit-config>
</rpc>

2016-02-13 04:03:24.862 GMT+00:00 [nettyThreadgroupModule$NioEventLoopGroupCloseable-7-3] TRACE o.o.c.s.c.n.l.NetconfDeviceCommunicator - Finished sending request <rpc message-id="m-17" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<edit-config>
<target>
<candidate/>
</target>
<error-option>rollback-on-error</error-option>
<config>
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"/>
</config>
</edit-config>
</rpc>

2016-02-13 04:03:24.972 GMT+00:00 [nettyThreadgroupModule$NioEventLoopGroupCloseable-7-3] DEBUG o.o.c.s.c.n.l.NetconfDeviceCommunicator - RemoteDevice

: Message received <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="m-17">
<rpc-error>
<error-type>application</error-type>
<error-tag>operation-failed</error-tag>
<error-severity>error</error-severity>
<error-message>Datastore fails to validate (Element config failed to validate content)</error-message>
</rpc-error>
<default:rpc-error xmlns:default="urn:ietf:params:xml:ns:netconf:base:1.0">
<error-type>application</error-type>
<error-tag>operation-failed</error-tag>
<error-severity>error</error-severity>
<error-message>Datastore fails to validate (Extra element interfaces in interleave)</error-message>
</default:rpc-error>
</rpc-reply>

@Wojciech: This is separate issue, could you open bug for it?

(In reply to Wojciech Dec from comment #10)
> The issue also appears when attempting to add configuration via RESTCONF. It
> actually renders the API unusable for config changes.
>
> API :
> PUT
> /config/opendaylight-inventory:nodes/node/172.20.118.63/yang-ext:mount/ietf-
> interfaces:interfaces/interface/

>
> Input :
> {
> "interface": [
>

> ]
> }
>
> Netconf data doesn't carry the mandatory parameters and fails:
>
> 2016-02-13 04:03:24.861 GMT+00:00
> [nettyThreadgroupModule$NioEventLoopGroupCloseable-7-3] TRACE
> o.o.c.s.c.n.u.NetconfRpcFutureCallback - RemoteDevice

:
> Sending message <rpc message-id="m-17"
> xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
> <edit-config>
> <target>
> <candidate/>
> </target>
> <error-option>rollback-on-error</error-option>
> <config>
> <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"/>
> </config>
> </edit-config>
> </rpc>
>
> 2016-02-13 04:03:24.862 GMT+00:00
> [nettyThreadgroupModule$NioEventLoopGroupCloseable-7-3] TRACE
> o.o.c.s.c.n.l.NetconfDeviceCommunicator - Finished sending request <rpc
> message-id="m-17" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
> <edit-config>
> <target>
> <candidate/>
> </target>
> <error-option>rollback-on-error</error-option>
> <config>
> <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces"/>
> </config>
> </edit-config>
> </rpc>
>
> 2016-02-13 04:03:24.972 GMT+00:00
> [nettyThreadgroupModule$NioEventLoopGroupCloseable-7-3] DEBUG
> o.o.c.s.c.n.l.NetconfDeviceCommunicator - RemoteDevice

:
> Message received <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"
> message-id="m-17">
> <rpc-error>
> <error-type>application</error-type>
> <error-tag>operation-failed</error-tag>
> <error-severity>error</error-severity>
> <error-message>Datastore fails to validate (Element config failed to
> validate content)</error-message>
> </rpc-error>
> <default:rpc-error xmlns:default="urn:ietf:params:xml:ns:netconf:base:1.0">
> <error-type>application</error-type>
> <error-tag>operation-failed</error-tag>
> <error-severity>error</error-severity>
> <error-message>Datastore fails to validate (Extra element interfaces in
> interleave)</error-message>
> </default:rpc-error>
> </rpc-reply>