[INTDIST-59] The distribution is too large Created: 04/Aug/16  Updated: 20/Oct/17  Resolved: 25/Aug/16

Status: Resolved
Project: integration-distribution
Component/s: Features
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Stephen Kitt Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


External issue ID: 6341

 Description   

Whenever the distribution artifacts end up larger than 512MB, Nexus uploads are rejected so new artifacts can't be tested. There's currently no need for the distribution to be so large, we need to fix the various issues causing the size increase.

This is a tracking bug for the various size reduction patches.



 Comments   
Comment by Vratko Polak [ 04/Aug/16 ]

Confirmed, distribution jobs are failing [0] like this:

https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/distribution-karaf/0.5.0-SNAPSHOT/distribution-karaf-0.5.0-20160803.223021-4083.tar.gz. Return code is: 413, ReasonPhrase: Request Entity Too Large.

[0] https://logs.opendaylight.org/releng/jenkins092/controller-distribution-boron/339/_console-output.log.gz

Comment by Jamo Luhrsen [ 04/Aug/16 ]

please do not close this bug until the nexus file size limit is reverted to 500M. please see tsc meeting minutes from 8/4/2016 for more info.

Comment by Luis Gomez [ 04/Aug/16 ]

We should also monitor this job so we never lose the offline deploy:

https://jenkins.opendaylight.org/releng/view/integration/job/integration-distribution-offline-boron/

Comment by Jamo Luhrsen [ 04/Aug/16 ]

(In reply to Luis Gomez from comment #3)
> We should also monitor this job so we never lose the offline deploy:
>
> https://jenkins.opendaylight.org/releng/view/integration/job/integration-
> distribution-offline-boron/

can we just make the distribution-check that happens on the gerrits run
in offline mode? then no patches could come in to bring this offline bug back?

Comment by Vratko Polak [ 04/Aug/16 ]

> can we just make the distribution-check that happens on the gerrits run
in offline mode?

+1
But wait until Bugs 6187 and this one are verified to be fixed.

Comment by Luis Gomez [ 04/Aug/16 ]

I think it would be possible yes and we would save 1 job.

Comment by Stephen Kitt [ 05/Aug/16 ]

Upgrading to Karaf 3.0.7 shrinks the distribution by ~70MB (resulting in a 499MB tarball). Comparing the 3.0.6 and 3.0.7 distributions reveals that the latter drops the following duplicated JARs (amongst others):

  • Bouncy Castle 1.49 (we're using 1.54)
  • CXF (again; do we actually need it?)
  • Groovy
  • GSON 2.2.2
  • Guava 13.0, 13.0.1, 17.0 (these are ~2MB JARs)
  • Guice
  • Hazelcast 1.9.4
  • Jackson 2.4.1 (we're using 2.3.2)
  • JClouds
  • Netty 3.3.1, 3.9.1 and 4.0.19 (we're using 4.0.37)
  • WSS4J

A number of ServiceMix bundles disappear too.

Comment by Stephen Kitt [ 05/Aug/16 ]

https://git.opendaylight.org/gerrit/43138 drops one extraneous version of Jackson (from Atrium).

Comment by Luis Gomez [ 05/Aug/16 ]

Very good Stephen, what is the plan for karaf 3.0.7 update?

Comment by Stephen Kitt [ 05/Aug/16 ]

(In reply to Luis Gomez from comment #9)
> Very good Stephen, what is the plan for karaf 3.0.7 update?

I'm hoping https://git.opendaylight.org/gerrit/40818 will pass distribution-check now that https://git.opendaylight.org/gerrit/43139 has been merged (once the merge job has completed...). After that I'd like to get a CSIT run, and given the late stage in the release we might want a TSC waiver or something like that...

Comment by Luis Gomez [ 05/Aug/16 ]

OK, let me know if i can help with anything.

Comment by Stephen Kitt [ 08/Aug/16 ]

(In reply to Luis Gomez from comment #11)
> OK, let me know if i can help with anything.

We have a complete test-integration build, https://jenkins.opendaylight.org/releng/job/integration-patch-test-boron/24/ — does it look OK?

Comment by Vratko Polak [ 24/Aug/16 ]

Is the size still at blocking level? As in, should we postpone Boron release until it improves (and by how much)?

Comment by Stephen Kitt [ 24/Aug/16 ]

(In reply to Vratko Polák from comment #13)
> Is the size still at blocking level? As in, should we postpone Boron release
> until it improves (and by how much)?

No, the RC1 distribution is only 417.41MB in size (as a gzipped tarball).

Comment by Jamo Luhrsen [ 24/Aug/16 ]

(In reply to Stephen Kitt from comment #14)
> (In reply to Vratko Polák from comment #13)
> > Is the size still at blocking level? As in, should we postpone Boron release
> > until it improves (and by how much)?
>
> No, the RC1 distribution is only 417.41MB in size (as a gzipped tarball).

The recent size is aprox 412M which is below our previous blocking level
of 512M which was relaxed in order to unblock us.

I also need to comment here that we should not be closing this bug until
we confirm that our nexus upload size limit has been reduced back to its'
original 512M level.

Comment by Vratko Polak [ 24/Aug/16 ]

Current sizes:

distribution-karaf-0.5.0-Boron-RC1.zip : 440465781 bytes.
distribution-karaf-0.6.0-20160824.182041-339.zip : 431629240 bytes.

Comment by Andrew Grimberg [ 25/Aug/16 ]

As of 2016-08-25 09:48 (PDT) the upload limit for nexus has been reduced back to 512MB as stipulated. Since I've been asked to mark this as RESOLVED upon the completion of this task, I'm now doing so.

Generated at Wed Feb 07 20:02:40 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.