https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/INTPAK-161 integration-packaging <p>We currently don't have a good way to create really "scratch" packages, as our devel repos get consumed in CD pipelines. We can build throw-away packages in the sandbox, but we have to add any tests to the same build job as we can't archive them from the sandbox (and wouldn't want to put them in CD stream anyway).</p> <p>Should create repos like:</p> <p><a href="https://nexus.opendaylight.org/content/repositories/opendaylight-fluorine-epel-7-x86_64-scratch" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight-fluorine-epel-7-x86_64-scratch</a></p> <p>TODO: Better name than "scratch"?</p> INTPAK-161

Create scratch pkg repos

Task High Closed Done Anil Belur Daniel Farrell Wed, 18 Apr 2018 02:57:52 +0000 Fri, 29 Jun 2018 19:10:23 +0000 Wed, 27 Jun 2018 13:49:12 +0000 0 3 <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=dfarrell07" class="user-hover" rel="dfarrell07">dfarrell07</a> I would be happy to assist with creating the repositories. I need to understand more about the need for scratch pkgs and repo requirements. Do we need a scratch repo per project and per-release? Maybe we should discuss this on a call?</p> <p>Please note, presently the rpms/debs are being pushed to devel repos <span class="error">&#91;1.&#93;</span> get purged at 30 days (since they are acting as staging repos). Ideally, someone should raise a build promotion requests for the released rpm builds to copy them to official release repos on Nexus and we should have jobs which promote them automatically.</p> <p><span class="error">&#91;1&#93;</span> <a href="https://nexus.opendaylight.org/content/repositories/opendaylight-" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight-</a>{*}-epel-7-x86_64-devel</p> <p>Thanks for the help <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=askb" class="user-hover" rel="askb">askb</a> <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>There are two kinda-distinct usecases for the scratch repos:</p> <ul> <li>Most importantly, want to be able to build propose code into packages and push them there. So a dev would put a keyword in a Gerrit comment, something like the multipatch job will build that into a distro and then something like the build-rpm job will build that into a package. The result should get pushed to these scratch repos for the dev to test.</li> <li>Also, it would be cool if we could push to these from the sandbox. So if there is a snapshot distro that was built by something like a merge job, or a multipatch job, then a dev could push the build-rpm job to the sandbox, point it at the snapshot distro in question, trigger a build and have the resulting package pushed to these scratch repos.</li> </ul> <p>Regarding promoting releases - we need to document some process for how that should be done (requested?). I have a WIP patch to document the full int/pack release process here: <a href="https://git.opendaylight.org/gerrit/#/c/71650/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/71650/</a></p> <p>Promoting packages on Nexus isn't super critical I think because hosting them on the CBS works fine. <a href="http://cbs.centos.org/repos/" class="external-link" target="_blank" rel="nofollow noopener">http://cbs.centos.org/repos/</a></p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=dfarrell07" class="user-hover" rel="dfarrell07">dfarrell07</a> thanks for the detailed explanation. I will work on setting update the repos next week and update here once its ready.</p> <p>&gt; Promoting packages on Nexus isn't super critical I think because hosting them on the CBS works fine. <a href="http://cbs.centos.org/repos/" class="external-link" target="_blank" rel="nofollow noopener">http://cbs.centos.org/repos/</a></p> <p>We are ok with hosting them on CBS, the only caveat here is the *-devel repos have schedule tasks which remove packages older than 30 days from the time of the build (and not actual release), which requires a job in place which push the rpms into CBS or *-release repos soon after the release. </p> <p>I'm just pushing to the CBS manually for now. I think it would be cool to make a job to do that, but it might be tricky to get credentials that can be used safely in jobs. </p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=dfarrell07" class="user-hover" rel="dfarrell07">dfarrell07</a> I belive these are going to be used as test repos for testing individual patches, I would recommend we use these as generic devel repos, instead of having a separate repo per-release (oxygen, fluorine .. etc)?</p> <p>We have two repos available on Nexus2, let me know if these can be used: </p> <p>ex:<br/> <a href="https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel</a><br/> <a href="https://nexus.opendaylight.org/content/repositories/opendaylight-ubuntu-1604-x86_64-devel" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight-ubuntu-1604-x86_64-devel</a></p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=abelur" class="user-hover" rel="abelur">abelur</a> Yeah, those would work perfectly. Better than maintaining per-stream repos. All we need is somewhere to dump packages.</p> <p>I'm messing around with some JJB to play with them now.</p> <p><a href="https://git.opendaylight.org/gerrit/73418" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/73418</a> adds some logic that lets me pass opendaylight-epel-7-x86_64-devel as REPO_ID, thereby allowing deploys to that repo.</p> <p>When running in the sandbox, it seems to work but then we run into an (expected) permission denied error:</p> <blockquote><p>Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy-file (default-cli) on project standalone-pom: Failed to deploy artifacts: Could not transfer artifact org.opendaylight.integration-packaging:opendaylight:rpm:8.2.0-1.el7.noarch from/to opendaylight-epel-7-x86_64-devel (<a href="https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel):" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel):</a> Failed to transfer file: <a href="https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel/org/opendaylight/integration-packaging/opendaylight/8.2.0-1.el7.noarch/opendaylight-8.2.0-1.el7.noarch.rpm" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel/org/opendaylight/integration-packaging/opendaylight/8.2.0-1.el7.noarch/opendaylight-8.2.0-1.el7.noarch.rpm</a>. Return code is: 401, ReasonPhrase: Unauthorized.</p></blockquote> <p><a href="https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/packaging-build-rpm-fluorine/3/console.log.gz" class="external-link" target="_blank" rel="nofollow noopener">https://logs.opendaylight.org/sandbox/vex-yul-odl-jenkins-2/packaging-build-rpm-fluorine/3/console.log.gz</a></p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=abelur" class="user-hover" rel="abelur">abelur</a> Can we setup the sandbox to allow deploys to these scratch repos?</p> <p>&gt; When running in the sandbox, it seems to work but then we run into an (expected) permission denied error:</p> <p>That's because we don't allow deploying artifacts to Nexus from sandbox. However, we can provide a temp hook for testing purposes, and remove it once we have tested the jobs <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>I had a brief discussion with <a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=agrimberg" class="user-hover" rel="agrimberg">agrimberg</a> regarding setting up CBD credentials on Jenkins. We are not sure what CLI are being used for uploading packages on the CBS system. However, can we create/manage these credentials files or add the creds ~/.netrc (confined only on Jenkins Prod env jobs) and have packaging jobs on releng manage upload the released packages to CBS repos in a timely manner. Also will have to have to figure out a way to setup a separate CBS account for this task. </p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=abelur" class="user-hover" rel="abelur">abelur</a> These are the CBS docs:</p> <p><a href="https://wiki.centos.org/HowTos/CommunityBuildSystem" class="external-link" target="_blank" rel="nofollow noopener">https://wiki.centos.org/HowTos/CommunityBuildSystem</a></p> <p><a href="https://wiki.centos.org/HowTos/CentosPackager" class="external-link" target="_blank" rel="nofollow noopener">https://wiki.centos.org/HowTos/CentosPackager</a></p> <p>Would be cool to have this more automated and, most importantly, have someone other than myself with permissions to push to CBS. We're currently at a very low hit-by-bus number there. <img class="emoticon" src="https://jira.opendaylight.org/images/icons/emoticons/wink.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>&gt; That's because we don't allow deploying artifacts to Nexus from sandbox. However, we can provide a temp hook for testing purposes, and remove it once we have tested the jobs</p> <p>Can we enable pushing to some destinations in Nexus? That may be too much to ask, but if it's easier than I anticipate than it would be nice for developers to be able to build in the sandbox and have the resulting package hosted in these scratch repos. See <a href="https://jira.opendaylight.org/browse/INTPAK-151" title="Support custom RPM builds by unprivileged users" class="issue-link" data-issue-key="INTPAK-151"><del>INTPAK-151</del></a>.</p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=dfarrell07" class="user-hover" rel="dfarrell07">dfarrell07</a> Please try again, you should be able to push to that scratch repository.</p> <p>The deploy to scratch repos is working now</p> <p><a href="https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel/org/opendaylight/integration-packaging/opendaylight/8.2.0-1.el7.src/" class="external-link" target="_blank" rel="nofollow noopener">https://nexus.opendaylight.org/content/repositories/opendaylight-epel-7-x86_64-devel/org/opendaylight/integration-packaging/opendaylight/8.2.0-1.el7.src/</a></p> <p><a href="https://jira.opendaylight.org/secure/ViewProfile.jspa?name=abelur" class="user-hover" rel="abelur">abelur</a> - Awesome, confirmed this is working as desired. Thanks!</p> Relates INTPAK-189 INTPAK-151 Development Rank 0|i03djj: