https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/MDSAL-309 mdsal <p>If enum value is not valid Java identifier, e.g.:</p> <p>      type enumeration {<br/>         enum '*' </p> {           description           "Any source address.";         } <p>      }</p> <p>V1 code generator produces invalid enum constant identifier:</p> <p> </p> <p>public class Ipv4MulticastSourceAddress<br/>  implements Serializable {<br/>     private static final long serialVersionUID = -1734346858985024066L;<br/>     public enum Enumeration {<br/>         /**<br/>          * Any source address.<br/>          *<br/>          */<br/>         <b>(0, "</b>")<br/>         ;<br/>    <span class="error">&#91;...&#93;</span></p> <p> </p> <p>hence compilation fails.</p> MDSAL-309

V1 codegen generates enum constants that are not valid Java identifiers

Bug Medium Resolved Done Robert Varga Marek Gradzki Thu, 8 Feb 2018 13:22:38 +0000 Thu, 5 Apr 2018 09:11:00 +0000 Fri, 23 Mar 2018 11:09:10 +0000 Nitrogen SR1 Carbon SR4 Oxygen SR1 Fluorine Nitrogen SR3 Binding codegen 0 4 <p>The issue is related to ietf-routing-types@2017-02-27 so simplest fix would be to</p> <p>transform characters that are not Java IdentifierChars to UNICODE names (Character.getName) + underscore separators.</p> <p>Solution is backward compatible (in YANG-&gt;Java direction, not sure if other direction is relevant),</p> <p>but such mapping might produce collisions (e.g. "*" and "<em>ASTERIX</em>").</p> <p> </p> <p>Solution 2:</p> <p>Replace all underscores in the enum name to pars of underscores (required to prevent collisions as in S1),</p> <p>then replace non-java IdentifierChars to UNICODE names + underscore separators.</p> <p>Such mapping is not backwards compatible, but it's injective (probably 1-1).</p> <p> </p> <p>RFC7950 specifies enum statement argument as: </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> The "enum" statement, which is a substatement to the "type" statement, MUST be present if the type is "enumeration". It is repeatedly used to specify each assigned name of an enumeration type. It takes as an argument a string that is the assigned name. The string MUST NOT be zero-length and MUST NOT have any leading or trailing whitespace characters (any Unicode character with the "White_Space" property). The use of Unicode control codes SHOULD be avoided.</pre> </div></div> <p>Which means we need to handle any old Unicode string. Generated name has to be a valid Java identifier, as per <a href="https://docs.oracle.com/javase/specs/jls/se9/html/jls-3.html#jls-3.8" class="external-link" target="_blank" rel="nofollow noopener">https://docs.oracle.com/javase/specs/jls/se9/html/jls-3.html#jls-3.8</a>, which specifies it as:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>A "Java letter" is a character for which the method Character.isJavaIdentifierStart(int) returns true. A "Java letter-or-digit" is a character for which the method Character.isJavaIdentifierPart(int) returns true. The "Java letters" include uppercase and lowercase ASCII Latin letters A-Z (\u0041-\u005a), and a-z (\u0061-\u007a), and, for historical reasons, the ASCII dollar sign ($, or \u0024) and underscore (_, or \u005f). The dollar sign should be used only in mechanically generated source code or, rarely, to access pre-existing names on legacy systems. The underscore may be used in identifiers formed of two or more characters, but it cannot be used as a one-character identifier due to being a keyword. The "Java digits" include the ASCII digits 0-9 (\u0030-\u0039). Letters and digits may be drawn from the entire Unicode character set, which supports most writing scripts in use in the world today, including the large sets for Chinese, Japanese, and Korean. This allows programmers to use identifiers in their programs that are written in their native languages.</pre> </div></div> <p>YANG/Java mapping in this case needs to be at least an injection, so codegen works in a non-conflicting manner. We can work out the reverse assignment based on value (and sanity-check with the same function).</p> <p>I have (inexactly) examined the currently-public corpus of YANG models from github, which shows 16432 unique strings being used, some of which are fun (single-letter, ?, *). The corpus is attached to this issue.</p> <p>Based on this data and the expectation we will be allowed to break the spec in Fluorine, I would propose actually having the mapping function discontinuous, with separate mapping rules based on the contents of the entire string. Extracted corpus shows that '$' is not used anywhere, which makes it a good escape character from JLS perspective (use in generated source code, can be start of an identifier) and observed corpus (it does not show a need to escape it).</p> <p>Hence the rules would be the following:</p> <ul> <li>if the string is a valid java identifier and does not contain '$', use it as-is</li> <li>otherwise prefix it with '$' and replace any invalid character with '$XXX$', where XXX is hex-encoded unicode codepoint (including plane, stripping leading zeroes)</li> </ul> <p>This would break the users thoroughly, I am afraid <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/sad.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>As a mitigation, we can perhaps consider enums within the enumeration to decide how to map members and reflect that in the enumeration class name... Perhaps use current mapping schema when the result is known to work and use a trailing '$' when we know the enums are a mess and we are using the above schema...</p> <p>Any thoughts on this?</p> <p>Using current mapping schema when result is Java identifier, otherwise the one you propose looks backward compatible to me.</p> <p> </p> <p>Such mapping is not invertible unless we know which schema was used.</p> <p>AFAIR Enumerations class is not generated if enumeration is defined using typedef, so</p> <p>perhaps we should consider using getter instead of class name. That would not break current users.</p> <p> </p> <p>But not sure why we can't use yang names of enums instead. They are already present in the generated code.</p> <p> </p> <p> </p> <p> </p> <p>As it turns out we do not need to mark the enumeration name at all. The only place that needs to reconcile Binding-&gt;Yang names (and vice-versa) is the binding codec, which has both the class and EnumTypeDefinition already available – hence pairing the two is a rather trivial matter.</p> <p>It would be possible to reconstruct this even without the help of schema, as the class itself captures and exposes YANG names, except it is not captured in a common contract, so we'd have to resort to reflection. I have filed <a href="https://jira.opendaylight.org/browse/MDSAL-317" title="Generated enumerations should be implementing a yang-binding Enumeration interface" class="issue-link" data-issue-key="MDSAL-317"><del>MDSAL-317</del></a> to track that improvement.</p> <p>Fluorine: <a href="https://git.opendaylight.org/gerrit/69317" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/69317</a></p> <p> </p> <p>Since the fix is backwards-compatible, it will be applied to all currently-supported streams.</p> <p>Oxygen: <a href="https://git.opendaylight.org/gerrit/69334" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/69334</a></p> <p>Nitrogen: <a href="https://git.opendaylight.org/gerrit/69336" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/69336</a></p> <p>Carbon: <a href="https://git.opendaylight.org/gerrit/69338" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/69338</a></p> <p> </p> Blocks MDSAL-317 Duplicate MDSAL-208 MDSAL-333 Relates MDSAL-269 Development Rank 0|i03a3r: