[RELENG-69] Create autorelease job to verify non SR projects upstream changes Created: 30/Nov/17  Updated: 24/Aug/18  Resolved: 24/Aug/18

Status: Closed
Project: releng
Component/s: Autorelease
Affects Version/s: None
Fix Version/s: None

Type: Story Priority: High
Reporter: Luis Gomez Assignee: Anil Belur
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

We need an autorelease job to verify non-SR projects (odlparent, yangtools) changes before they get released and consumed by downstream:

  • A first attempt of the job could just take a version (SNAPSHOT or release) of odlparent and yangtools and bump all SR projects to use this version instead of existing release. The resulting distribution could be sent to some sanity CSIT for verification. The goal for this first version is to detect breakages before the upstream project releases. This job could run periodically.
  • A second stab on the job could take a patch number for odlparent or yangtools, build the patch and do similar than first. The goal in this case is to verify an upstream patch before it is actually merged. This job could run on-demand.


 Comments   
Comment by Luis Gomez [ 30/Nov/17 ]

As discussed during integration call, we need this job but we can wait until we get new cloud infra.

Comment by Luis Gomez [ 09/Jan/18 ]

I start to think this kind of job will be much more stable and usable once we streamline the Core distribution (hopefully in Flourine).

Comment by Thanh Ha (zxiiro) [ 08/Feb/18 ]

askb The Oxygen release is fast approaching. I think we should try to get this task going if we want to have it available for the upcoming Flourine release.

Comment by Anil Belur [ 09/Feb/18 ]

Here is a summary of points as discussed with ecelgp:

  • The job verifies that version bumping to the snapshot version for the list of dependent projects does not break the build. The job is required to change the SR project version to the snapshot version for each of the dependent project and then build the projects to ensure that version bumping the SR projects passes. 
    • Clone all the projects similar to AR.
    • Change the versions in pom files to point to the snapshots for the SR projects. We mostly require to change parent.groupId or dependencies to snapshot version.
    • Build all the projects and verify they are passing 
  • Initially start with minimum number of projects (like controller, mdsal) or with a small distribution patch as provided.
  • Creating a single job for all SR project would be ideal such that it take a parameter (project=<version>, where <version> is the version number to build against). Refer [1.] for example: genius=32/53632/9,netvirt:59/50259/47
  • Note: if projects are using version numbers from properties then we need to fix them in the projects 

[1.] https://jenkins.opendaylight.org/releng/view/integration/job/integration-multipatch-test-oxygen/72/parameters

 

 

Comment by Anil Belur [ 18/Mar/18 ]

ecelgp Here is an initial version (WIP) of the job being tested on the sandbox

https://jenkins.opendaylight.org/sandbox/job/autorelease-verify-sr-projects-oxygen/9

Here is the link to log server which is working

https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/autorelease-verify-sr-projects-oxygen/11/

Comment by Anil Belur [ 18/Mar/18 ]

Tested with the change [x.] which bumps yangtools to 2.0.3-SNAPSHOT

[x.] https://git.opendaylight.org/gerrit/#/c/69366/

Comment by Anil Belur [ 19/Mar/18 ]

[WIP] Verify non SR projects changes upstream
https://git.opendaylight.org/gerrit/69596

Comment by Anil Belur [ 19/Mar/18 ]

ecelgp Please provide a odlparent version bump change for testing the job.

Comment by Luis Gomez [ 19/Mar/18 ]

I think for oxygen you can try any odlparent patch in 3.0.x branch (3.0.3-SNAPSHOT):

For example: https://git.opendaylight.org/gerrit/#/c/69609/

For master (fluorine) you can try any odlparent patch in 3.1.x branch (3.1.0-SNAPSHOT) but I do not see any opened patch right now. Would it be possible to build the 3.1.x branch head instead?

Comment by Anil Belur [ 20/Mar/18 ]

ecelgp I tried with both the patches, but unfortunately the pom files are not standard between the SR projects which makes it difficult to read the SR SNAPSHOT VERSION. For ex odlparent has:

https://jenkins.opendaylight.org/sandbox/job/autorelease-verify-sr-projects-oxygen/14/consoleText

    <parent>
        <groupId>org.opendaylight.odlparent</groupId>
        <artifactId>odlparent-lite</artifactId>
        <version>4.0.0-SNAPSHOT</version>
        <relativePath>odlparent-lite</relativePath>
    </parent>

While yangtools has:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <parent>
      <groupId>org.opendaylight.odlparent</groupId>
      <artifactId>odlparent-lite</artifactId>
      <version>3.0.2</version>
      <relativePath/>
    </parent>

    <modelVersion>4.0.0</modelVersion>
    <groupId>org.opendaylight.yangtools</groupId>
    <artifactId>yangtools-aggregator</artifactId>
    <version>2.0.3-SNAPSHOT</version>
    <name>yangtools</name> <!-- Used by Sonar to set project name -->
    <packaging>pom</packaging>
Comment by Anil Belur [ 20/Mar/18 ]

Retrigger the job along with handling diff pom file structures.

https://jenkins.opendaylight.org/sandbox/job/autorelease-verify-sr-projects-oxygen/14

Comment by Luis Gomez [ 20/Mar/18 ]

I guess if it is difficult to extract odlparent/yangtools version from a patch, we could pass those directly to the job as the most important thing (at least in the beginning) is to do the version bump and build of all projects that are in AR.

Comment by Anil Belur [ 20/Mar/18 ]

ecelgp No worries, I have made some improvement to the scripts to handle that part too.

However, I am unable to figure some inconsistencies in the pom.xml in controller project [x.] file which is causing parsing errors in [y.]:

[x.]https://github.com/opendaylight/controller/blob/master/opendaylight/archetypes/opendaylight-startup/src/main/resources/archetype-resources/artifacts/pom.xml

[y.]https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/autorelease-verify-sr-projects-oxygen/16

`controller/opendaylight/archetypes/opendaylight-startup/src/main/resources/archetype-resources/artifacts/pom.xml`

#set( $symbol_pound = '#' )
#set( $symbol_dollar = '$' )
#set( $symbol_escape = '\' )
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright © ${copyrightYear} ${copyright} and others. All rights reserved.

This program and the accompanying materials are made available under the
terms of the Eclipse Public License v1.0 which accompanies this distribution,
and is available at http://www.eclipse.org/legal/epl-v10.htm

 

 

 

Comment by Luis Gomez [ 24/Apr/18 ]

So today in Kernel call Robert brought a totally different approach on how to do the verification of Release-Integrated project (odlparent, yangtools) changes before they get released and consumed by downstream:

Instead of straight build an odlparent/yangtools patch and then bump all Managed projects to use this version, we could simply:

1) download odlparent/yangtools patch.
2) downgrade odlparent/yangtools version to match desired stream (oxygen, fluorine, etc).
4) clone all Managed projects and build normally.

This way we do not need to change version in all AR projects but just the ones we want to test (odlparent/yangtools).

Comment by Anil Belur [ 24/Aug/18 ]

As discussed over the int-dist call, this is implemented in the multipatch job, therefore not required any longer.

Generated at Wed Feb 07 20:37:25 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.