[AAA-37] javax.ws.rs.WebApplicationException: org.opendaylight.aaa.sts.TokenAuthFilter.<clinit>(TokenAuthFilter.java:40) Created: 11/Jun/15  Updated: 21/Mar/19  Resolved: 02/Jul/15

Status: Resolved
Project: aaa
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Flavio Fernandes Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: Text File karaf.log    
External issue ID: 3680
Priority: Highest

 Description   

Even after more than 5 mins after ODL is started, rest calls fail
due to aaa service not being ready

Ref: https://lists.opendaylight.org/pipermail/controller-dev/2015-June/009628.html

2015-06-11 10:28:08,791 | ERROR | qtp335420420-194 | ContainerResponse | 212 - com.sun.jersey.jersey-server - 1.17.0 | Mapped exception to response: 503 (Service Unavailable)
javax.ws.rs.WebApplicationException
at org.opendaylight.aaa.sts.TokenAuthFilter.<clinit>(TokenAuthFilter.java:40)
at java.lang.Class.forName0(Native Method)[:1.7.0_79]
at java.lang.Class.forName(Class.java:191)[:1.7.0_79]
at com.sun.jersey.core.reflection.ReflectionHelper.classForNameWithException(ReflectionHelper.java:242)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.core.reflection.ReflectionHelper.classForNameWithException(ReflectionHelper.java:222)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.core.spi.component.ProviderServices.getInstances(ProviderServices.java:216)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.server.impl.container.filter.FilterFactory.getFilters(FilterFactory.java:146)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.container.filter.FilterFactory.init(FilterFactory.java:75)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl._initiate(WebApplicationImpl.java:1283)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl.access$700(WebApplicationImpl.java:168)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:774)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:770)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:770)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:765)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer.initiate(ServletContainer.java:489)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer$InternalWebComponent.initiate(ServletContainer.java:319)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.WebComponent.load(WebComponent.java:605)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:210)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:374)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:557)[210:com.sun.jersey.servlet:1.17.0]
at javax.servlet.GenericServlet.init(GenericServlet.java:161)[178:org.apache.geronimo.specs.geronimo-servlet_3.0_spec:1.0.0]
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:532)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:344)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:791)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:265)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:717)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doStart(HttpServiceContext.java:222)[193:org.ops4j.pax.web.pax-web-jetty:3.1.4]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.ops4j.pax.web.service.jetty.internal.JettyServerImpl$1.start(JettyServerImpl.java:202)[193:org.ops4j.pax.web.pax-web-jetty:3.1.4]
at org.ops4j.pax.web.service.internal.HttpServiceStarted.end(HttpServiceStarted.java:1032)[192:org.ops4j.pax.web.pax-web-runtime:3.1.4]
at org.ops4j.pax.web.service.internal.HttpServiceProxy.end(HttpServiceProxy.java:422)[192:org.ops4j.pax.web.pax-web-runtime:3.1.4]
at org.ops4j.pax.web.extender.war.internal.RegisterWebAppVisitorWC.end(RegisterWebAppVisitorWC.java:341)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.model.WebApp.accept(WebApp.java:678)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher$WebAppDependencyListener.register(WebAppPublisher.java:237)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher$WebAppDependencyListener.addingService(WebAppPublisher.java:182)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher$WebAppDependencyListener.addingService(WebAppPublisher.java:135)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:932)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:864)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:317)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:261)[karaf-org.osgi.core.jar:]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher.publish(WebAppPublisher.java:101)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebObserver.deploy(WebObserver.java:213)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebObserver$1.doStart(WebObserver.java:175)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.extender.SimpleExtension.start(SimpleExtension.java:58)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.extender.AbstractExtender$1.run(AbstractExtender.java:266)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_79]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_79]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)[:1.7.0_79]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)[:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_79]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_79]
2015-06-11 10:28:08,877 | ERROR | qtp335420420-194 | ContainerResponse | 212 - com.sun.jersey.jersey-server - 1.17.0 | Mapped exception to response: 503 (Service Unavailable)
javax.ws.rs.WebApplicationException
at org.opendaylight.aaa.sts.TokenAuthFilter.<clinit>(TokenAuthFilter.java:40)
at java.lang.Class.forName0(Native Method)[:1.7.0_79]
at java.lang.Class.forName(Class.java:191)[:1.7.0_79]
at com.sun.jersey.core.reflection.ReflectionHelper.classForNameWithException(ReflectionHelper.java:242)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.core.reflection.ReflectionHelper.classForNameWithException(ReflectionHelper.java:222)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.core.spi.component.ProviderServices.getInstances(ProviderServices.java:216)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.server.impl.container.filter.FilterFactory.getFilters(FilterFactory.java:146)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.container.filter.FilterFactory.init(FilterFactory.java:75)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl._initiate(WebApplicationImpl.java:1283)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl.access$700(WebApplicationImpl.java:168)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:774)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:770)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)[211:com.sun.jersey.core:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:770)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:765)[212:com.sun.jersey.jersey-server:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer.initiate(ServletContainer.java:489)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer$InternalWebComponent.initiate(ServletContainer.java:319)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.WebComponent.load(WebComponent.java:605)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:210)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:374)[210:com.sun.jersey.servlet:1.17.0]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:557)[210:com.sun.jersey.servlet:1.17.0]
at javax.servlet.GenericServlet.init(GenericServlet.java:161)[178:org.apache.geronimo.specs.geronimo-servlet_3.0_spec:1.0.0]
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:532)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:344)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:791)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:265)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:717)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doStart(HttpServiceContext.java:222)[193:org.ops4j.pax.web.pax-web-jetty:3.1.4]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)[184:org.eclipse.jetty.aggregate.jetty-all-server:8.1.15.v20140411]
at org.ops4j.pax.web.service.jetty.internal.JettyServerImpl$1.start(JettyServerImpl.java:202)[193:org.ops4j.pax.web.pax-web-jetty:3.1.4]
at org.ops4j.pax.web.service.internal.HttpServiceStarted.end(HttpServiceStarted.java:1032)[192:org.ops4j.pax.web.pax-web-runtime:3.1.4]
at org.ops4j.pax.web.service.internal.HttpServiceProxy.end(HttpServiceProxy.java:422)[192:org.ops4j.pax.web.pax-web-runtime:3.1.4]
at org.ops4j.pax.web.extender.war.internal.RegisterWebAppVisitorWC.end(RegisterWebAppVisitorWC.java:341)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.model.WebApp.accept(WebApp.java:678)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher$WebAppDependencyListener.register(WebAppPublisher.java:237)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher$WebAppDependencyListener.addingService(WebAppPublisher.java:182)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher$WebAppDependencyListener.addingService(WebAppPublisher.java:135)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:932)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:864)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:256)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.AbstractTracked.trackInitial(AbstractTracked.java:183)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:317)[karaf-org.osgi.core.jar:]
at org.osgi.util.tracker.ServiceTracker.open(ServiceTracker.java:261)[karaf-org.osgi.core.jar:]
at org.ops4j.pax.web.extender.war.internal.WebAppPublisher.publish(WebAppPublisher.java:101)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebObserver.deploy(WebObserver.java:213)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.WebObserver$1.doStart(WebObserver.java:175)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.extender.SimpleExtension.start(SimpleExtension.java:58)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at org.ops4j.pax.web.extender.war.internal.extender.AbstractExtender$1.run(AbstractExtender.java:266)[198:org.ops4j.pax.web.pax-web-extender-war:3.1.4]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_79]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_79]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)[:1.7.0_79]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)[:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_79]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_79]

 Comments   
Comment by Flavio Fernandes [ 11/Jun/15 ]

Ref irc excerpt:

https://gist.github.com/b6a92d04494ca1a0dd9d

[16:47:13] <edwarnicke> Is this helium or lithium ?
[16:47:26] <flaviof> edwarnicke: you will like this: its Lithium!
[16:47:49] <edwarnicke> flaviof: I do like that
[16:47:52] <mestery> lothium
[16:47:54] <mestery> yeah
[16:47:56] <flaviof> lol
[16:47:57] <edwarnicke> flaviof: Needed to know so I could look at the right code
[16:48:07] <flaviof> edwarnicke: ack.
[16:48:24] <edwarnicke> flaviof: mestery So question: is it the RC0 build? (note: there are no wrong answers here, just trying to figure out where to look )
[16:49:01] <flaviof> edwarnicke: it is last night build. the logic to grab what build to use is here:
[16:49:05] flaviof digging
[16:49:24] <mestery> yes, it's a snapshot build
[16:49:25] <mestery> the very latest
[16:49:32] <mestery> as of something like 30 minutes ago
[16:49:40] <edwarnicke> mestery: Cool
[16:49:46] <edwarnicke> mestery: flaviof Now I know where to look
[16:49:47] <flaviof> edwarnicke: https://github.com/stackforge/networking-odl/blob/master/devstack/release.lithium-snapshot-0.3.0
[16:53:12] <edwarnicke> mestery: flaviof OK... I know a little more
[16:53:41] <edwarnicke> mestery: flaviof Basically, the Exception is because the aaa system has not authentication service at the time it was asked to authenticate something
[16:53:54] <edwarnicke> mestery: You said that you had REST calls that went through before the failure?
[16:53:59] mestery shakes his fist at the ^@#$!@# aaa system
[16:54:33] <mestery> edwarnicke: I thought that, but on closer look, it wasn't the case.
[16:54:46] <flaviof> edwarnicke: is this an issue in the realm of.... if you waited longer it would have worked; or this is borked and aaa is gone fishing for good?
[16:54:48] <edwarnicke> mestery: Good to know, because that would be less mysterious
[16:55:03] <edwarnicke> flaviof: So given what mestery just said, yes, if you waited longer it should work
[16:55:08] <mestery>
[16:55:11] <edwarnicke> flaviof: mestery Not to imply its not a bug
[16:55:23] <mestery> We're already waiting 6 minutes I guess I can increase the timeout more
[16:55:25] <edwarnicke> flaviof: mestery Because you shouldn't have to wait
[16:55:27] <flaviof> sigh
[16:55:40] <edwarnicke> mestery: I figured out something that should make a lot of this better in general
[16:55:51] <flaviof> Kyle it is already waithing 5 mins?
[16:55:59] <edwarnicke> mestery: Basically... ODL should be building in a way that it doesn't need to fetch bundles via mvn at startup
[16:56:03] <mestery> flaviof: 6 minutes
[16:56:03] <mestery>
[16:56:05] <edwarnicke> mestery: But that is broken

Comment by Ryan Goulding [ 15/Jun/15 ]

Ed, are you working on this issue? Let me know if you need a second set of eyes, as I have encountered this as well .

Comment by Wojciech Dec [ 24/Jun/15 ]

A solution may be in moving AAA to use the config sub system and it's ordered dependencies.

That being said, issues with the fetching of mvn artefacts will likely not be solved by that.

Comment by Ed Warnicke [ 26/Jun/15 ]

I believe this to be a classic 'random bundle load order' bug caused by aaa using Activators instead of the config subsystem... switching to using the config-subsystem should fix it.

Comment by Wojciech Dec [ 26/Jun/15 ]

Same symptoms could be given if MD-SAL fails to come up:

https://bugs.opendaylight.org/show_bug.cgi?id=3878

Comment by Ryan Moats [ 30/Jun/15 ]

see https://git.opendaylight.org/gerrit/#/c/23608/ for the revert patch that should address this...

Comment by Wojciech Dec [ 30/Jun/15 ]

@rmoats : Thanks, but that commit is likely not the cause of the breakage covered by this bug, as it was merged after this bug was filed.

Comment by Wojciech Dec [ 01/Jul/15 ]

General request: Could anyone experiencing this issue attach their karaf.log?

Comment by Flavio Fernandes [ 01/Jul/15 ]

(In reply to Wojciech Dec from comment #8)
> General request: Could anyone experiencing this issue attach their karaf.log?

as requested, I will attach the log.
For ref, this is where it is located:

http://logs.openstack.org/79/197179/3/check/check-tempest-dsvm-networking-odl/b322a4a/logs/screen-karaf.log.2015-06-30-205206

http://logs.openstack.org/79/197179/3/check/check-tempest-dsvm-networking-odl/b322a4a/logs/

http://logs.openstack.org/79/197179/3/check/check-tempest-dsvm-networking-odl/b322a4a/

Comment by Flavio Fernandes [ 01/Jul/15 ]

http://logs.openstack.org/79/197179/3/check/check-tempest-dsvm-networking-odl/b322a4a/logs/screen-karaf.log.2015-06-30-205206

renamed to karaf.log

Comment by Flavio Fernandes [ 01/Jul/15 ]

Attachment karaf.log has been added with description: karaf.log

Comment by Wojciech Dec [ 02/Jul/15 ]

The following is one immediate fix:

https://git.opendaylight.org/gerrit/#/c/23683/

It likely resolves the most pressing problem.

Comment by Flavio Fernandes [ 02/Jul/15 ]

fixed on master.

[11:22:48] <regXboi> flaviof: I think we can also mark 3680 as resolved for now
[11:22:54] <regXboi> because we aren't seeing it in master

Generated at Wed Feb 07 19:08:28 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.