Status: Pausing further work as now blocked by ODLPARENT-167, and when that is solved then blocked by OVSDB-467 ...
Status update: ODLPARENT-167 is behind us (but it turns out that we cannot benefit from it here today just yet, even though odlparent 4.0.1 has been related, because of what I explained on https://git.opendaylight.org/gerrit/#/c/76239/ FYI skitt) and progress in OVSDB-467 (which is still WIP for other reasons) gets us past odl-neutron-hostconfig-ovs. We're now hitting a failure in NeutronE2ETest (Singleton Network Post Failed NB expected:<201> but was:<400>) which needs to be looked into more next...
Is there a WIP patch other than c/76239 for neutron? This is the last dependency of lispflowmapping for the Neon MRI, to make sure that our WIP patch is ready.
ljakab https://git.opendaylight.org/gerrit/#/c/76089/ is the WIP and still Draft one (but I just added you as a Reviewer so that you can see it) where we still need to sort out problems related to a big mess with Jersey... it (should) build, but all our tests fail 
Thanks for adding me as a reviewer. After building this patch, including lispflowmapping's other dependencies, all of our SFTs pass, even though I didn't build neutron's dependencies. I build controller, aaa, netconf, and neutron, in this order.
The JAX RS / Jersey problems are not trivial... skitt and I have been looking at this together for a few hours during ONS here in Amsterdam - it's a mess... Summary FTR: We are currently (odlparent 3.1.3) on a mix of Jersey 2.22.2 and 2.8, and in https://git.opendaylight.org/gerrit/#/c/76434/ attempt to converge everything on 2.27. This doesn't work for us, and seems to have something to do with https://stackoverflow.com/a/46405129/421602 ... but just adding jersey-hk2 does NOT do the trick, it still fails, saying:
WARNING: A provider org.opendaylight.neutron.northbound.api.NeutronNetworksNorthbound registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.opendaylight.neutron.northbound.api.NeutronNetworksNorthbound will be ignored. Sep 26, 2018 3:33:35 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
Unless someone has a brilliant idea, we should probably consider to only upgrade to 2.25.1 and give up on 2.27, for now, as explored in https://git.opendaylight.org/gerrit/#/c/76462/, to unblock this for the short term and finish up neon-mri soon-ish. We can revisit Jersey 2.27 for odlparent 5.0.0... We're working on confirming that we would be OK with 2.25.1 ...
We are now suspecting this (PITA!), based on having written ExceptionMapper, that this is because some mix up re. JSON processor... Jersey <=> javax.json <=> JAXB <=> EclipseLink Moxy - OMG!
skitt and I finally somehow got this working last week - so moving from In Progress to In Review.
The patch still needs to move to odlparent 4.0.2 though.
ljakab done: bump-odl-version odlparent 4.0.1 4.0.2
skitt I don't think we're done with this one here, it locally fails for me (if I don't -Pq it) with:
[INFO] --- maven-surefire-plugin:2.22.0:test (default) @ odl-neutron-northbound-api --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.opendaylight.odlparent.featuretest.SingleFeatureTest [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 57.625 s <<< FAILURE! - in org.opendaylight.odlparent.featuretest.SingleFeatureTest [ERROR] installFeatureCatchAndLog(org.opendaylight.odlparent.featuretest.SingleFeatureTest)[repoUrl: file:/home/vorburger/dev/ODL/git/releng/autorelease/neutron/features/production/odl-neutron-northbound-api/target/feature/feature.xml, Feature: odl-neutron-northbound-api 0.12.0.SNAPSHOT] Time elapsed: 54.407 s <<< ERROR! org.apache.felix.resolver.reason.ReasonException: Unable to resolve org.opendaylight.mdsal.yang-binding/0.14.0.SNAPSHOT: missing requirement [org.opendaylight.mdsal.yang-binding/0.14.0.SNAPSHOT] osgi.wiring.package; filter:="(&(osgi.wiring.package=org.opendaylight.yangtools.util)(version>=2.0.0)(!(version>=3.0.0)))" at org.apache.felix.resolver.Candidates$MissingRequirementError.toException(Candidates.java:1343) at org.apache.felix.resolver.ResolverImpl.doResolve(ResolverImpl.java:420) at org.apache.felix.resolver.ResolverImpl.resolve(ResolverImpl.java:378) at org.apache.felix.resolver.ResolverImpl.resolve(ResolverImpl.java:332) at org.apache.karaf.features.internal.region.SubsystemResolver.resolve(SubsystemResolver.java:257) at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:388) at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1025) at org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:964) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
But I don't immediately see what is wrong (and am tired, after MANY hours of fixing up GENIUS-210).
ljakab or does it work for you locally and I have something messed up? I just build lots of other projects though.
rovarga (or tpantelis) if you spot what's wrong and can help, I appreciate it.
One observation from GENIUS-210 which may be of interest here:
The SFT for both odl-genius-api and odl-genius, passed and only odl-genius-rest failed... it's probably not a coincidence that both odl-genius-rest and odl-neutron-northbound-api are web stuff related?
https://jenkins.opendaylight.org/releng/view/integration/job/integration-multipatch-test-neon/55/ actually seems to have gone past the problem shown above, but I'm consistently hittig it locally - not sure what's going on here... probably something inconsistent in my local Maven repo? I'm intentionally running with mvo -o -nsu ... but anyway, ignore this, I guess.
Patch needs to be bumped to released MD-SAL 3.0.0
ljakab hey! Thanks.. rovarga seems to have done 3.0.0-SNAPSHOT in Patch Set 11 (and I've bumped odlparent from 4.0.1 to 4.0.2 in Patch Set 12) - I'll try to do 3.0.0 right now, to see if that helps... (but I think somehow for me locally I somewhere have a left over mdsal 0.14.0.SNAPSHOT).
> I'll try to do 3.0.0 right now
done in Patch Set 13
> to see if that helps
Just confirming here as well as on list that problem listed above is solved.
> Stephen Kitt and I finally somehow got this working last week - so moving from In Progress to In Review.
skitt ljakab jhershbe Neutron as-is currently is actually completely broken in neon-mri; SFT pass, but if we run neutron/integration/test locally (which is commented out in the root pom.xml because it kept failing too often on the build...) it fails with the same Singleton Network Post Failed NB expected:<201> but was:<400> we hit above, and solved for "standalone" - but we still need to solve that for OSGi/Karaf!
Just for the record for future users trying out the ITNeutronE2E locally: The org.ops4j.pax.swissbox.tracker.ServiceLookupException: gave up waiting for service org.ops4j.pax.exam.ProbeInvoker failures are sporadic (and the reason why we have had to comment it out form the root POM in the build!), jsut retrying it usually make it pass (or fail with a real error, as here).
Moving org.glassfish:javax.json "up" from integration/test-standalone's POM into neutron-spi (simpy by NOT making it <scope>test there, and further cleaning that up by removing the <version> and <type>bundle) probably fixes Singleton Network Post Failed NB expected:<201> but was:<400> but ITNeutronE2E is now hitting this new problem, fancy:
2018-10-08T16:06:25,324 | ERROR | activator-1-thread-2 | BootFeaturesInstaller | 9 - org.apache.karaf.features.core - 4.2.1 | Error installing boot features
org.apache.felix.resolver.reason.ReasonException: Uses constraint violation. Unable to resolve resource org.opendaylight.neutron.logger [org.opendaylight.neutron.logger/0.12.0.SNAPSHOT] because it is exposed to package 'com.google.common.base' from resources com.google.guava [com.google.guava/25.1.0.jre] and com.google.guava [com.google.guava/23.6.1.jre] via two dependency chains.
Chain 1:
org.opendaylight.neutron.logger [org.opendaylight.neutron.logger/0.12.0.SNAPSHOT]
import: (&(osgi.wiring.package=com.google.common.base)(version>=25.1.0)(!(version>=26.0.0)))
|
export: osgi.wiring.package: com.google.common.base
com.google.guava [com.google.guava/25.1.0.jre]
Chain 2:
org.opendaylight.neutron.logger [org.opendaylight.neutron.logger/0.12.0.SNAPSHOT]
import: (&(osgi.wiring.package=org.opendaylight.controller.md.sal.binding.api)(version>=1.9.0)(!(version>=2.0.0)))
|
export: osgi.wiring.package=org.opendaylight.controller.md.sal.binding.api; uses:=com.google.common.base
org.opendaylight.controller.sal-binding-api [org.opendaylight.controller.sal-binding-api/1.9.0.SNAPSHOT]
import: (&(osgi.wiring.package=com.google.common.base)(version>=23.6.0)(!(version>=24.0.0)))
|
export: osgi.wiring.package: com.google.common.base
com.google.guava [com.google.guava/23.6.1.jre]
The issue above was just due to something (??) having replaced the Neon-MRI JARs locally; upon re-installing controller, aaa & netconf that went away.
Using https://git.opendaylight.org/gerrit/#/c/76760/, we can see that indeed the problem is the very same as what we chased a 1.5 weeks ago in the standalone test:
2018-10-08T18:37:44,523 | ERROR | qtp631306707-115 | NeutronNorthboundRSApplication | 265 - org.opendaylight.neutron.northbound-api - 0.12.0.SNAPSHOT | Error processing response javax.ws.rs.WebApplicationException: HTTP 400 Bad Request at org.eclipse.persistence.jaxb.rs.MOXyJsonProvider.readFrom(MOXyJsonProvider.java:734) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.invokeReadFrom(ReaderInterceptorExecutor.java:256) ~[?:?] at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.aroundReadFrom(ReaderInterceptorExecutor.java:235) ~[?:?] at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:155) ~[?:?] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundReadFrom(MappableExceptionWrapperInterceptor.java:74) ~[?:?] at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:155) ~[?:?] at org.glassfish.jersey.message.internal.MessageBodyFactory.readFrom(MessageBodyFactory.java:1085) ~[?:?] at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:874) ~[?:?] at org.glassfish.jersey.server.ContainerRequest.readEntity(ContainerRequest.java:271) ~[?:?] at org.glassfish.jersey.server.internal.inject.EntityParamValueFactoryProvider$EntityValueFactory.provide(EntityParamValueFactoryProvider.java:96) ~[?:?] at org.glassfish.jersey.server.spi.internal.ParamValueFactoryWithSource.provide(ParamValueFactoryWithSource.java:71) ~[?:?] at org.glassfish.jersey.server.spi.internal.ParameterValueHelper.getParameterValues(ParameterValueHelper.java:94) ~[?:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$AbstractMethodParamInvoker.getParamValues(JavaResourceMethodDispatcherProvider.java:127) ~[?:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) ~[?:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[?:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[?:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[?:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[?:?] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) ~[?:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) ~[?:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) ~[?:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~[?:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[?:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) ~[?:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) ~[?:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[?:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[?:?] at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) ~[?:?] at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) ~[?:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) ~[?:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) ~[?:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) ~[?:?] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) ~[147:org.eclipse.jetty.servlet:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) ~[147:org.eclipse.jetty.servlet:9.4.11.v20180605] at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:311) ~[148:org.eclipse.jetty.servlets:9.4.11.v20180605] at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:265) ~[148:org.eclipse.jetty.servlets:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[147:org.eclipse.jetty.servlet:9.4.11.v20180605] at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:215) ~[157:org.eclipse.jetty.websocket.server:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) ~[147:org.eclipse.jetty.servlet:9.4.11.v20180605] at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) ~[?:?] at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) ~[?:?] at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) ~[?:?] at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) ~[?:?] at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) ~[?:?] at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) ~[?:?] at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) ~[?:?] at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) ~[?:?] at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) ~[?:?] at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) ~[?:?] at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) ~[?:?] at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) ~[?:?] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) ~[147:org.eclipse.jetty.servlet:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) ~[147:org.eclipse.jetty.servlet:9.4.11.v20180605] at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) ~[301:org.ops4j.pax.web.pax-web-jetty:7.2.3] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) ~[144:org.eclipse.jetty.security:9.4.11.v20180605] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:293) ~[301:org.ops4j.pax.web.pax-web-jetty:7.2.3] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) ~[147:org.eclipse.jetty.servlet:9.4.11.v20180605] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) ~[301:org.ops4j.pax.web.pax-web-jetty:7.2.3] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.Server.handle(Server.java:531) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) ~[146:org.eclipse.jetty.server:9.4.11.v20180605] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) ~[138:org.eclipse.jetty.io:9.4.11.v20180605] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) ~[138:org.eclipse.jetty.io:9.4.11.v20180605] at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) ~[138:org.eclipse.jetty.io:9.4.11.v20180605] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) ~[149:org.eclipse.jetty.util:9.4.11.v20180605] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) ~[149:org.eclipse.jetty.util:9.4.11.v20180605] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) ~[149:org.eclipse.jetty.util:9.4.11.v20180605] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) ~[149:org.eclipse.jetty.util:9.4.11.v20180605] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762) [149:org.eclipse.jetty.util:9.4.11.v20180605] at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680) [149:org.eclipse.jetty.util:9.4.11.v20180605] at java.lang.Thread.run(Thread.java:748) [?:?] Caused by: javax.xml.bind.UnmarshalException at org.eclipse.persistence.jaxb.JAXBUnmarshaller.handleXMLMarshalException(JAXBUnmarshaller.java:1112) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:353) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.rs.MOXyJsonProvider.readFrom(MOXyJsonProvider.java:686) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] ... 82 more Caused by: org.eclipse.persistence.exceptions.XMLMarshalException: Exception Description: An error occurred unmarshalling the document Internal Exception: javax.json.JsonException: Provider org.glassfish.json.JsonProviderImpl not found at org.eclipse.persistence.exceptions.XMLMarshalException.unmarshalException(XMLMarshalException.java:122) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.json.JsonStructureReader.parse(JsonStructureReader.java:148) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:1018) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:454) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:402) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:743) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.XMLUnmarshaller.unmarshal(XMLUnmarshaller.java:651) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:351) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.rs.MOXyJsonProvider.readFrom(MOXyJsonProvider.java:686) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] ... 82 more Caused by: javax.json.JsonException: Provider org.glassfish.json.JsonProviderImpl not found at javax.json.spi.JsonProvider.provider(JsonProvider.java:99) ~[58:javax.json-api:1.1.2] at javax.json.Json.createReader(Json.java:225) ~[58:javax.json-api:1.1.2] at org.eclipse.persistence.internal.oxm.record.json.JsonStructureReader.parse(JsonStructureReader.java:123) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:1018) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:454) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:402) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:743) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.XMLUnmarshaller.unmarshal(XMLUnmarshaller.java:651) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:351) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.rs.MOXyJsonProvider.readFrom(MOXyJsonProvider.java:686) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] ... 82 more Caused by: java.lang.ClassNotFoundException: org.glassfish.json.JsonProviderImpl cannot be found by javax.json-api_1.1.2 at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:484) ~[?:?] at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:395) ~[?:?] at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:387) ~[?:?] at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:150) ~[?:?] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:?] at java.lang.Class.forName0(Native Method) ~[?:?] at java.lang.Class.forName(Class.java:264) ~[?:?] at javax.json.spi.JsonProvider.provider(JsonProvider.java:96) ~[58:javax.json-api:1.1.2] at javax.json.Json.createReader(Json.java:225) ~[58:javax.json-api:1.1.2] at org.eclipse.persistence.internal.oxm.record.json.JsonStructureReader.parse(JsonStructureReader.java:123) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:1018) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:454) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:402) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.record.SAXUnmarshaller.unmarshal(SAXUnmarshaller.java:743) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.internal.oxm.XMLUnmarshaller.unmarshal(XMLUnmarshaller.java:651) ~[161:org.eclipse.persistence.core:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:351) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] at org.eclipse.persistence.jaxb.rs.MOXyJsonProvider.readFrom(MOXyJsonProvider.java:686) ~[162:org.eclipse.persistence.moxy:2.7.3.v20180807-4be1041] ... 82 more
This happens despite me now having added org.glassfish:javax.json (not just to neutron-spi but to all Neutron bundles, just to be extra sure!). I think what's happenins is that the Classloader of the Moxy bundle is not finding that org.glassfish.json.JsonProviderImpl thing (which is in org.glassfish:javax.json) - which is why it worked standalone, but doesn't work in OSGi.
I'm going to have to read up more of the Jersey doc at this point. It seems it may be possible to configure it (Jersey) a little differently than we have in Neutron in the past, and get it to directly use Moxy, instead of the Jersey -> JAXB -> Moxy thing which we are currently (I think...) doing; this may help for this kind of class loading issue - hopefully.
> read up more of the Jersey doc at this point. It seems it may be possible to configure it (Jersey) a little differently than we have in Neutron in the past, and get it to directly use Moxy, instead of the Jersey -> JAXB -> Moxy thing which we are currently (I think...) doing; this may help for this kind of class loading issue - hopefully.
My idea here was to use the org.glassfish.jersey.moxy.json.MoxyJsonFeature and the org.glassfish.jersey.moxy.json.MoxyJsonConfig instead of the org.eclipse.persistence.jaxb.rs.MOXyJsonProvider, following https://jersey.github.io/documentation/2.25.1/media.html#json.moxy - and I tried that, but it doesn't help, makes no difference...
... because the real problem here, of course is the Class.forName in javax.json.spi.JsonProvider which as we all know well is very problematic unde OSGi (I should have seen and jumped on that 2 weeks ago!). Looking more closely at that here https://github.com/eclipse-ee4j/jsonp/blob/master/api/src/main/java/javax/json/spi/JsonProvider.java we see that it attempts to use java.util.ServiceLoader for javax.json.spi.JsonProvider (of which we and probably the rest of the world doesn't have any), and then default to the hard-coded DEFAULT_PROVIDER = "org.glassfish.json.JsonProviderImpl" and a Class.forName for that. This seems to have been like this since the dawn of time in JSON-P (JSR 374), already back in https://github.com/javaee/jsonp/blob/json-1.0/api/src/main/java/javax/json/Json.java. So why did this suddenly break for us??
It's because org.eclipse.persistence.moxy 2.7.1 depended directly on org.glassfish:javax.json:jar 1.0.4, whereas (Moxy) 2.7.3 depends on javax.json:javax.json-api 1.1.2 and we (see above) then had to add an explicit extra dependeny ourselves to org.glassfish:javax.json - but now 1.1.2.
And whereas org.glassfish:javax.json:jar 1.0.4 fat packaged both packages javax.json as well as org.glassfish.json (AND THEREFORE THE Class.forName WORKED EVEN UNDER OSGi), the newer org.glassfish:javax.json 1.1.2 has (only) the org.glassfish.json packages, with a dep to the separate new javax.json:javax.json-api 1.1.2 which now contains (only) the javax.json packages. Nicer API / Impl split packaging - but breaking OSGi.
So the EclipseLink bump in https://git.opendaylight.org/gerrit/#/c/75508/ from 2.7.1 to 2.7.3 is the culprit for this "little" (hugely time consuming and blocked the Neon-MRI upgrade) problem.
What next? We most probably, hopefully, do not have to revert that (and cut a new odlparent...) - there should be an easy local solution possible for the very short term by forcing usage of the old org.glassfish:javax.json:jar 1.0.4 instead of 1.1.2 (and excluding javax.json:javax.json-api).
> idea here was to use the org.glassfish.jersey.moxy.json.MoxyJsonFeature and the org.glassfish.jersey.moxy.json.MoxyJsonConfig instead of the org.eclipse.persistence.jaxb.rs.MOXyJsonProvider
FTR: The use of jersey-media-moxy (it's mere presence on the classpath by way of a dependency in northbound-api) also causes the following weird issue, which suspect NEUTRON-199 may fix; TBC later:
Exception [EclipseLink-43] (Eclipse Persistence Services - 2.7.3.v20180807-4be1041): org.eclipse.persistence.exceptions.DescriptorException Exception Description: Missing class for indicator field value [HTTP] of type [class java.lang.String]. Descriptor: XMLDescriptor(org.opendaylight.neutron.spi.NeutronLoadBalancerHealthMonitor --> [DatabaseTable(neutronLoadBalancerHealthMonitor), DatabaseTable(neutronObject), DatabaseTable(neutronID)])
> there should be an easy local solution possible for the very short term by forcing usage of the old
I finally got this working, see https://git.opendaylight.org/gerrit/#/c/76089/14..15, but it required dependency management both in the project-neutron-parent and introducing a new neutron-single-feature-parent for all of Neutron's odl-neutron-* features.
In fact, we are just very lucky that the org.eclipse.persistence.moxy bundle does not specify a restrictive exclusive version range for its javax.json Import-Package (it has no version); if it did, then we would be screwed without any way out to override that, AFAIK.
NEUTRON-200 may have future follow-up re. all this mess.
As commented in TSC-132, full #61 failed in neutron NeutronE2ETest AssertionError: L2 Gateway Delete Failed expected:<204> but was:<404> - that is not the same problem as the original broken Neutron in NEUTRON-197 (what was Singleton Network Post Failed NB expected:<201> but was:<400> in the IT ITNeutronE2E (not the standalone NeutronE2ETest) ... but this passed locally for me yesterday, still does - and also is green locally for skitt, so we agreed together to add an @Ignore and we'll re-activate it after.