[NETCONF-713] Stackoverflow in sal-rest-docgen Created: 21/Jul/20  Updated: 28/Mar/22  Resolved: 25/Mar/22

Status: Resolved
Project: netconf
Component/s: restconf-nb
Affects Version/s: Magnesium SR1, Aluminium
Fix Version/s: 3.0.0, 2.0.15

Type: Bug Priority: Medium
Reporter: Richard Kosegi Assignee: Samuel Schneider
Resolution: Done Votes: 0
Labels: pt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Microsoft Word [NETCONF-713] Steps to reproduce.rtf     Text File [NETCONF-713]karaf.log.txt     File regexp-module.yang    

 Description   

When schema contains yang model that defines particular regexp, apidoc explorer is not able to list resource definition and throw java.lang.StackOverflowError

It was tracked down that this happen when we have list of containers with string leaf restricted by following regexp:

typedef module-name {
   reference "RFC 6020: YANG - A Data Modelling Language Section 5.2. File Layout";
   type string {
     pattern '[a-zA-Z_][a-zA-Z0-9\-_.]*(@\d{4}-\d{2}-\d{2})?';
   }
 }

 

Here is excerpt from stacktrace (shortened to relevant part)

Caused by: java.lang.StackOverflowError
 at java.util.HashMap$HashIterator.<init>(HashMap.java:1475) ~[?:?]
 at java.util.HashMap$KeyIterator.<init>(HashMap.java:1514) ~[?:?]
 at java.util.HashMap$KeySet.iterator(HashMap.java:912) ~[?:?]
 at java.util.HashSet.iterator(HashSet.java:173) ~[?:?]
 at java.util.AbstractCollection.toArray(AbstractCollection.java:184) ~[?:?]
 at dk.brics.automaton.State.getSortedTransitionArray(State.java:136) ~[?:?]
 at dk.brics.automaton.State.getSortedTransitions(State.java:147) ~[?:?]
 at com.mifmif.common.regex.Generex.prepareRandom(Generex.java:340) ~[?:?]
 at com.mifmif.common.regex.Generex.prepareRandom(Generex.java:366) ~[?:?]
 at com.mifmif.common.regex.Generex.prepareRandom(Generex.java:366) ~[?:?]
 at com.mifmif.common.regex.Generex.prepareRandom(Generex.java:366) ~[?:?]
 at com.mifmif.common.regex.Generex.prepareRandom(Generex.java:366) ~[?:?]

 



 Comments   
Comment by Richard Kosegi [ 21/Jul/20 ]

Maybe related https://github.com/mifmif/Generex/issues/21 https://github.com/mifmif/Generex/issues/26

Comment by Robert Varga [ 24/Nov/21 ]

Based on steps to reproduce and the Generex issues, some more exploration is needed.

Specifically we need to see if we can leverage https://github.com/mifmif/Generex/issues/22 to solve this.

Comment by Ivan Hrasko [ 03/Mar/22 ]

[generex|https://github.com/mifmif/Generex] looks to be not supported for over 5 years. We know that in the background it uses [link automaton| https://github.com/cs-au-dk/dk.brics.automaton]. Automaton looks to require @ to be escaped...

https://github.com/mifmif/Generex/issues/22 implementation looks to be not complete: https://github.com/mifmif/Generex/pull/48

Can we remove generex, escape @ and similar characters and use automaton directly?

Comment by Robert Varga [ 03/Mar/22 ]

Sure, whatever as long as the functionality works

Comment by Ivan Hrasko [ 17/Mar/22 ]

OK, I have changed my mind. Update/change of library is  probably work for another task - lets stay wit what we have for now.

Comment by Ivan Hrasko [ 17/Mar/22 ]

SamoSchneider can you please explain why we cannot use: https://github.com/mifmif/Generex/issues/22?

Comment by Samuel Schneider [ 17/Mar/22 ]

We can, but it does not make solution simpler.

https://github.com/mifmif/Generex/issues/22 just provide possibility to escape some parts of regex from generation without need to escape each special character one by one.

It escapes all special characters which is not desired. We  would still need to denote only characters which are problematic with \Q\E quotation.

I think escaping them directly is simpler.

Comment by Samuel Schneider [ 21/Mar/22 ]

This particular issue is caused by '@' being spacial character of [link automaton| https://github.com/cs-au-dk/dk.brics.automaton] used by [generex|https://github.com/mifmif/Generex]. My patch solve this by escaping this character with other character considered special by Automaton.

Another part of this issue is that Generex ends with StackOverflowError when generating string for some regexes (contains .* or some [negative set]* and maybe more).
For Automaton '@' is any string, so string (incorrect one) should be generated even if '@' is not escaped.
This issue can be also reproduced by replacing '@' with '.*'

pattern '[a-zA-Z_][a-zA-Z0-9\-_.]*(.*\d\{4}-\d\{2}-\d\{2})?';

I think this is caused by bug in Generex and my patch does not solve this.

Also when working on this issue I found cases of incorrect string being generated because Generex or I think used Automaton not recognizing some rules of regular expression.
Main problem being: non-capturing group '(?:ABC)' added by RegexUtils.getJavaRegexFromXSD. I found '\n','\t' and more also not working.
This is also conected to https://jira.opendaylight.org/browse/YANGTOOLS-887 because to generate correct string we would need something like XSD -> Java -> automaton regex translator.

 

Generated at Wed Feb 07 20:15:42 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.