https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/RELENG-79 releng <p>Presently the autorelease-validate-autorelease-{$STREAM}, only throws up a WARNING when a project is removed from AR but not removed from int/dist. </p> <p><span class="error">&#91;1.&#93;</span> <a href="https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-validate-autorelease-oxygen/42/console.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-validate-autorelease-oxygen/42/console.log.gz</a></p> <p> </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>04:54:11 [WARNING] The POM for org.opendaylight.bier:odl-bier-all:xml:features:0.3.0 is missing, no dependency information available    </pre> </div></div> <p>The job should be doing some kind of validation by default such that if we remove a project from AR and not from int/dist, by checking the dependencies between projects or should figure out int/dist project requires a dependency that is not there anymore.</p> <p>The better way may be for `autorelease-validate-autorelease-oxygen` does the dependency validation and makes the job fail instead of failing the AR job after several hours.</p> RELENG-79

Validate autorelease must check dependencies before project removal

Task Medium Resolved Done Anil Belur Anil Belur autorelease-validate Thu, 15 Feb 2018 02:14:58 +0000 Fri, 16 Mar 2018 16:33:47 +0000 Fri, 16 Mar 2018 05:42:49 +0000 unspecified 0 5 <p>I think the best way to handle this case to compare the list of int/dist projects with the list of autorelease projects and flag if they don't match.</p> <p>The tricky part here will be to handle projects that have subprojects like honeycomb/vbd and ofextensions/circuitsw kind of projects. So special care and attention should be paid to this kind of case.</p> <p>&gt; While we get some script to do the magic, is it possible to fail the build if we get these <br/> &gt; dependencies WARNING? if this is difficult to achieve then we just wait until someone has the <br/> &gt; cycles for writing the script.</p> <p>I was not able to find any flag which can magically do this with mvn, also confirmed this with <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=zxiiro" class="user-hover" rel="zxiiro">zxiiro</a>. I work on a script to get this sorted out. </p> <p>Something I do not understand is if a project (e.g. NIC) tries to add itself to int/dist but it is not in autorelease, the distribution project autorelease-validate fails like here:</p> <p><a href="https://jenkins.opendaylight.org/releng/job/integration-distribution-validate-autorelease-oxygen/193/console" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/releng/job/integration-distribution-validate-autorelease-oxygen/193/console</a></p> <p>However the opposite case, when a project is removed from autorelease but it is still in int/dist, autorelease project autorelease-validate does not fail.  </p> <p>If a project adds itself to int/dist validate-autorelease needs to know that it's being added so that it can include them in the validate-autorelease build.</p> <p>In the other case if it's removed from autorelease the validate-autorelease job should still fail it's probably a timing thing that's allowing it to pass like maybe validate-autorelease ran before the project was actually removed from autorelease.</p> <p>Edit: Validate-autorelease could be improved to detect that it is running against a project that is not in autorelease currently and adds them (only for the build, does not need to do it for real). This way validate-autorelease will always build the project that's being validated regardless of it's really in autorelease or not. This will make it easier to test projects as if they are in autorelease.</p> <p>To add ontop of this if we have a job that compares distribution with autorelease we can ensure that we are always in sync in an automated way.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> Please confirm if this (integration/distribution/features/repos/index/pom.xml) is the best file to read to get a list of all the projects in int/dist?</p> <p>Also I see that is AR has a few extra modules which are SR projects, can we exclude or skip them as a part of the validation?</p> <p>       integration/distribution <br/>        mdsal <br/>        odlparent <br/>        yangtools</p> <p> </p> <p>The validation should be simple: when removing a project from AR, make sure it does not have an active template in integration/distribution/features/repos/index/pom.xml</p> <p>For your comment: odlparent and yangtools should not be in AR from Oxygen onwards as they are not part of the SR. mdsal project can never be removed from AR unless we want to get rid of all projects as this is now at the root of the SR. int/dist is the distribution itself so no point to remove it unless we want to remove all projects again.</p> <p>Check project removal in validate autorelease<br/> <a href="https://git.opendaylight.org/gerrit/69105" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/69105</a></p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> When I run the validation with stable/oxygen or master branch. <br/> I get the following differences in projects:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>---&gt; validate-projects.sh ERROR: List of projects mismatch in releng/autorelease and integration/distribution: mdsal nic unimgr ocpplugin faas eman releng/autorelease: mdsal integration/distribution: nic unimgr ocpplugin faas eman </pre> </div></div> <p>I think `mdsal` is not in the list for `integration/distribution` or required directly? Do we need to remove `nic unimgr ocpplugin faas eman` projects from int/dist aswell?</p> <p>mdsal does not have a feature by itself and that is why you do not see it in int/dist. We should allow this mismatch.</p> <p>The other projects you mention are disabled in int/dist index pom.xml: <a href="https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob;f=features/repos/index/pom.xml;h=6982eff18abf0f2abe6600b807faa9e1e482884f;hb=refs/heads/stable/oxygen" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob;f=features/repos/index/pom.xml;h=6982eff18abf0f2abe6600b807faa9e1e482884f;hb=refs/heads/stable/oxygen</a></p> <p>So there must be some issue in the script.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> Any specific reason they have been disabled? Can we remove the projects from the int/dist index pom.xml to get them on par with AR, instead of them being disabled?</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> Another thing I noticed if that `integration/distribution/features/repos/index/pom.xml` pom file structure is completely different with every release and more consistent from stable/oxygen onwards. For instance, the activeByDefault tag is not available in the previous release. Maybe we should only enable this script for releases which supports them in the pom file and going forward?</p> <p>2 answers:</p> <ul class="alternate" type="square"> <li>The maven templates in int/dist were introduced in Oxygen which means it is enough if script works from Oxygen onwards.</li> <li>The way I see the templates will work going forward is: all projects part of the new Managed snapshot integrated group (what we call today SR) will have a template in int/dist. If a project miss-behave like break others or we have an upstream bump that requires removing projects from int/dist, we will do all this by just disabling/enabling templates. So this means 2 things:</li> </ul> <p>1) In general AR script has to be template enable/disable aware.<br/> 2) While the above should work, we can also go ahead and remove from int/dist those projects that are out of Oxygen: nic unimgr ocpplugin faas eman.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> thanks for the response. I am been trying with xmlstarlet if its possible to work around with the have that activation tag checked and set to 'true', but hitting some issues hopefully should get resolved.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=zxiiro" class="user-hover" rel="zxiiro">zxiiro</a> Would you know why the first and last element is concatenated in the xmlstarlet output? The pom file looks good to me. Please note, this is on stable/oxygen branch.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>$ xmlstarlet sel -N x=http://maven.apache.org/POM/4.0.0 -t -m '/x:project/x:profiles/x:profile/x:activation[x:activeByDefault="true"]' --if /x:project/x:dependencies/x:depende/x:groupId -v /x:project/x:dependencies/x:dependency/x:groupId --elif /x:project/x:profiles/x:profile/x:id -v /x:project/x:profiles/x:profile/x:id --else -o '' integration/distribution/features/repos/index/pom.xml | sort -u aaa alto bgpcep bier coe controller daexim distribution dlux dluxapps eman faas genius groupbasedpolicy honeycombvbd infrautils jsonrpc l2switch lispflowmapping nemo netconf netvirt neutron nic ocpplugin odlparent ofconfig openflowplugin ovsdb p4plugin packetcable sfc snmp snmp4sdn sxp tsdr unimgr usc vtn vtnaaa </pre> </div></div> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=zxiiro" class="user-hover" rel="zxiiro">zxiiro</a> The issue with `xmlstarlet` is now resolved. </p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> This is tested here on sandbox.</p> <p><a href="https://jenkins.opendaylight.org/sandbox/job/autorelease-validate-autorelease-oxygen/4/consoleFull" class="external-link" target="_blank" rel="nofollow noopener">https://jenkins.opendaylight.org/sandbox/job/autorelease-validate-autorelease-oxygen/4/consoleFull</a></p> <p>Please review/merge:<br/> <a href="https://git.opendaylight.org/gerrit/69105" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/69105</a></p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=ecelgp" class="user-hover" rel="ecelgp">ecelgp</a> I have closed this issue as resolved. Please feel free to reopen the issue if something related to this task is still pending.</p> <p>Thanks, this works as far as I can tell <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> Development Rank 0|i03afr: