Operating System: All
Platform: All

Attachment netconf.txt has been added with description: netconf request

Attachment restconf.txt has been added with description: restconf request

Honeycomb log

https://jira.fd.io/secure/attachment/11007/full_honeycomb.zip

Netconf request is working fine in stable/boron (except that you are missing <type> attribute in the rpc with id m-67).

You are refering to SR2 or SR3 snapshot?
How should valid netconf request look like?
Btw, we are getting the request from ODL client - we have no control over it.

I added <type/> and result is:

<rpc message-id="m-67" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<edit-config>
<target>
<candidate/>
</target>
<config>
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
<interface>
<name>GigabitEthernet0/8/0</name>
<type/>
<sub-interfaces xmlns="urn:opendaylight:params:xml:ns:yang:vpp:vlan"/>
</interface>
</interfaces>
</config>
</edit-config>
</rpc>
]]>]]>

Working rest request:

PUT http://localhost:8181/restconf/config/ietf-interfaces:interfaces/interface/GigabitEthernet0%2F8%2F0

{
"interface" :

}

GigabitEthernet0/8/0 was defined in cfg before edit-cfg msg was received (perhaps this is why mandatory type attribute was not included in the series of messages).

What we are more interesting in
is the vlan-tag part (union type deserialzation).

Have you checked boron SR2?

Yes I have checked release/boron-sr2 release and it worked fine.

It is possible it works for honeycomb/vbd alone. But our setup is a bit different. ODL app (honeycomb/vbd) sends netconf requests to hc2vpp. Then hc2vpp fails to deserialize vlan-id union type.

Please look at attached logs (DEBUG). Let me know if you need additional logs or increasing log level for some components to reproduce issue.

Ultimate solution to reproduce bug is to install and run hc2vpp https://docs.fd.io/hc2vpp/1.17.01-SNAPSHOT/release-notes-aggregator/release_notes.html#_install_guide (will try to simplify the setup).

cherry picked https://git.opendaylight.org/gerrit/#/c/52203/

Steps to reproduce:

Add to software repositories:
https://nexus.fd.io/content/repositories/fd.io.stable.1701.ubuntu.xenial.main/

then install:

sudo apt-get install vpp vpp-lib

the start vpp:

sudo service vpp start

check:
sudo vppctl show version
sudo vppctl show int

Download from honeycomb (if you want to change odl dependencies) and hc2vpp.

Build hc2vpp and run it:
sudo sh vpp-integration/minimal-distribution/target/vpp-integration-distribution-1.17.01-hc/vpp-integration-distribution-1.17.01/honeycomb-debug

Debugging port is 5005.

You can connect to honeycomb:
ssh admin@localhost -p 2831 -s netconf

Send hello message and run rpcs from file local0.txt (this file uses interface local0)

Check if interface local0 exists and/or create it with postman.collection.json.

Attachment local0.txt has been added with description: local0.txt file with requests to reproduce bug

Attachment postman.collection.json has been added with description: postman.collection.json file

In honeycomb project an hcv2vpp I used branch (stable/1701.

honeycomb and hc2vpp sources can be downloaded from:
https://gerrit.fd.io/r/#/admin/projects/

In honeycomb master they use workaround:
https://jira.fd.io/browse/HC2VPP-62

I also noticed that union types handling in general fails in our scenario
(string type is choosen regardless type order or deserialization fails if
there is no string type).

Perhaps some simple model with union type would speed up issue identification.

We use BindingToNormalizedNodeCodec for BA<>BI translation:

StreamWriterGenerator serializerGenerator =
new StreamWriterGenerator(JavassistUtils.forClassPool(ClassPool.getDefault()));
BindingNormalizedNodeCodecRegistry codecRegistry = new BindingNormalizedNodeCodecRegistry(serializerGenerator);
BindingRuntimeContext ctx = BindingRuntimeContext.create(mibCtx, mibCtx.getSchemaContext());
codecRegistry.onBindingRuntimeContextUpdated(ctx);
BindingToNormalizedNodeCodec codec = new BindingToNormalizedNodeCodec(mibCtx, codecRegistry);
codec.onGlobalContextUpdated(mibCtx.getSchemaContext());

...

codec.fromNormalizedNode(id, data)

BI data we deserialize is present in the full log
(link attached in one of previous comments).
Please let us know if you need to increase log level of some component.

If it is not enough to reproduce issue with non HC environment,
I can try to at least remove VPP from the picture.

In Beryllium-2 it worked because there was always used string constructor to deserialize. Now there is type checking, so data must have correct type.

Problem is probably somewhere outside mdsal.

Hi,

I tried to write test for this (added you to review) - https://git.opendaylight.org/gerrit/#/c/53067/ - and it seems to be working for us (mdsal). Can you confirm this test?

Thanks

We use ModuleInfoBackedContext, except for that test looks good.
But after updating your test still no exception...

Codec and schema service is shared among various ODL componets,
so perhaps issue is in how components interact.

Will try to simplify the setup needed to reproduce issue...

Here is simplified version of our setup with instructions how to reproduce issue:

https://gerrit.fd.io/r/#/c/5699/2

I checked also SR3 snapshots:

https://gerrit.fd.io/r/#/c/5699/3

and the issue is also present

Codecs prepared in mdsal are used only in mdsal with specific input data. I added test to previous test[0] - negative test - where input data (NormalizedNode) contains other type of value like it is defined by yang and I got your case with IllegalArgumentException. This mean, that you have to have good type of value in input data according to type of value define by yang. This is why you can't deserialize input data with string if there has to be int. Our codecs are OK + generated code from yangs is OK.

[0] - https://git.opendaylight.org/gerrit/#/c/53264/

Looking at the log, I see the following:

2017-02-08 09:57:18.527 CET [netconf-netty-1] DEBUG o.o.y.b.d.c.i.UnionValueOptionContext - Codec org.opendaylight.yangtools.binding.data.codec.impl.EnumerationCodec@1b169815 failed to deserialize input 2420
java.lang.IllegalArgumentException: Invalid enumeration value 2420. Valid values are [any]

Based on the example case you have provided, this makes sense, your JSON has quotes around the number, which means it is a String and treated as such, hence the integer-based union members are immediately skipped and it hits EnumerationCodec directly.

At some point we will need to parse values more strictly in the JSON/XML world, but that is a change we cannot do this late in the release.

Looking at the codec infrastructure, though, I am failing to see how that string could have passed through.

Are you getting the same results via RESTCONF and NETCONF?

Looking at the various classes, I am not sure which codec we are hitting and they do have slight variances:

https://github.com/opendaylight/yangtools/blob/master/yang/yang-data-impl/src/main/java/org/opendaylight/yangtools/yang/data/impl/codec/UnionStringCodec.java

does the wrong thing by returning the original string representation and it seems to be hooked into NETCONF.

https://github.com/opendaylight/yangtools/blob/master/yang/yang-data-util/src/main/java/org/opendaylight/yangtools/yang/data/util/AbstractStringUnionCodec.java seems to be doing the right thing (in most cases, I am not sure about edge cases) and is hooked into RESTCONF (for both XML and JSON).

I am positive MD-SAL is off the hook and this needs to be fixed either in yangtools (by fixing the old codecs) or (better) netconf (by switching to yang-data-codec-xml).

Can you guys confirm the following:

1) this issue occurs when the data is introduced via RESTCONF (if HC has such a thing) (both XML and JSON formats)
2) which union codec is being used when the request is deserialized in EditConfig.parseIntoNormalizedNode()

After we have the results we can decide which strategy to take.

I stand corrected, netconf is still using yang-data-impl XML codecs across the board.

This needs to gate Boron-3, because it is a regression from a previous release.

master: https://git.opendaylight.org/gerrit/53297

(In reply to Robert Varga from comment #25)
> I stand corrected, netconf is still using yang-data-impl XML codecs across
> the board.

Yeah, you are correct. Netconf and also Restconf is using data.impl.schema.transform parser/serializer infrastructure.

boron: https://git.opendaylight.org/gerrit/53308

Verified with test patch https://gerrit.fd.io/r/#/c/5699/, no issue present

(In reply to Jan Srnicek from comment #30)
> Verified with test patch https://gerrit.fd.io/r/#/c/5699/, no issue present
+ https://git.opendaylight.org/gerrit/#/c/53308/