[AAA-122] NPE in org.opendaylight.aaa.shiro.realm.MDSALDynamicAuthorizationFilter Created: 12/Apr/17  Updated: 21/Mar/19  Resolved: 05/May/17

Status: Resolved
Project: aaa
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Robert Varga Assignee: Ryan Goulding
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: File logs.tar    
External issue ID: 8214

 Description   

https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/627/archives/odl2_karaf.log.gz

2017-04-11 11:10:31,367 | WARN | tp1159369930-242 | ServletHandler | 44 - org.eclipse.jetty.aggregate.jetty-all-server - 8.1.19.v20160209 | Error Processing URI: /restconf/modules - (java.lang.NullPointerException) null
2017-04-11 11:10:33,534 | WARN | tp1159369930-242 | ServletHandler | 44 - org.eclipse.jetty.aggregate.jetty-all-server - 8.1.19.v20160209 |
javax.servlet.ServletException: java.lang.NullPointerException
at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)[238:org.apache.shiro.core:1.3.2]
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)[238:org.apache.shiro.core:1.3.2]
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)[238:org.apache.shiro.core:1.3.2]
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)[239:org.apache.shiro.web:1.3.2]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:69)[53:org.ops4j.pax.web.pax-web-jetty:3.2.9]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:240)[53:org.ops4j.pax.web.pax-web-jetty:3.2.9]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:75)[53:org.ops4j.pax.web.pax-web-jetty:3.2.9]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.Server.handle(Server.java:370)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)[44:org.eclipse.jetty.aggregate.jetty-all-server:8.1.19.v20160209]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
Caused by: java.lang.NullPointerException
at org.opendaylight.aaa.shiro.realm.MDSALDynamicAuthorizationFilter.getHttpAuthzContainer(MDSALDynamicAuthorizationFilter.java:52)[237:org.opendaylight.aaa.shiro:0.5.0.SNAPSHOT]
at org.opendaylight.aaa.shiro.realm.MDSALDynamicAuthorizationFilter.isAccessAllowed(MDSALDynamicAuthorizationFilter.java:74)[237:org.opendaylight.aaa.shiro:0.5.0.SNAPSHOT]
at org.opendaylight.aaa.shiro.realm.MDSALDynamicAuthorizationFilter.isAccessAllowed(MDSALDynamicAuthorizationFilter.java:61)[237:org.opendaylight.aaa.shiro:0.5.0.SNAPSHOT]
at org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(AccessControlFilter.java:162)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.filter.PathMatchingFilter.isFilterChainContinued(PathMatchingFilter.java:203)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.filter.PathMatchingFilter.preHandle(PathMatchingFilter.java:178)[239:org.apache.shiro.web:1.3.2]
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:131)[239:org.apache.shiro.web:1.3.2]
... 39 more

 Comments   
Comment by Peter Gubka [ 13/Apr/17 ]

I am switching this bug to blocker, because i am unable to move forward in DOMDataBroker testing if tell-based protocol is used.

Comment by Ryan Goulding [ 18/Apr/17 ]

Please add me to AAA bugs in the future.

https://github.com/opendaylight/aaa/blob/master/aaa-shiro/impl/src/main/java/org/opendaylight/aaa/impl/shiro/realm/MDSALDynamicAuthorizationFilter.java#L52

It looks like the dataBroker reference is null. I think I need to update the blueprint.xml to include a dataBroker reference so it is non-null when the AAAShiroProvider is loaded.

Comment by Ryan Goulding [ 19/Apr/17 ]

Actually I see now... NETCONF/RESTCONF doesn't have an OSGI dependency on AAA. Thats for a good reason, we never exposed an API for it. I would think the correct thing to do here would be to:

1) expose service api w/ filter method
2) have NETCONF/RESTCONF blueprint depend on AAA Service.

Make sense, Robert et al.?

Comment by Ryan Goulding [ 19/Apr/17 ]

1) is done through https://git.opendaylight.org/gerrit/#/c/55255/
2) is tricky since restconf utilizes CSS still

Comment by Ryan Goulding [ 19/Apr/17 ]

2) Draft patch in NETCONF

https://git.opendaylight.org/gerrit/55269

Comment by Ryan Goulding [ 27/Apr/17 ]

stable/carbon:
https://git.opendaylight.org/gerrit/#/c/56050/

master:
https://git.opendaylight.org/gerrit/#/c/55269

Comment by Peter Gubka [ 28/Apr/17 ]

Please have a look at https://jenkins.opendaylight.org/sandbox/job/bgpcep-csit-3node-bgpclustering-longevity-only-carbon/8/
I attach the logs from this job too.

NPE returned here:
https://logs.opendaylight.org/sandbox/jenkins091/bgpcep-csit-3node-bgpclustering-longevity-only-carbon/8/archives/log.html.gz#s1-s1-t3-k2-k1-k1-k6-k2-k1-k2-k1-k1-k3-k4-k1

It comes from node 2:
https://logs.opendaylight.org/sandbox/jenkins091/bgpcep-csit-3node-bgpclustering-longevity-only-carbon/8/archives/odl2_karaf.log.gz

Comment by Ryan Goulding [ 01/May/17 ]

Data store isn't accessible... is the node isolated?

Comment by Ryan Goulding [ 01/May/17 ]

https://git.opendaylight.org/gerrit/#/c/56330/

should handle the other NPE in this same section of code.

Comment by Peter Gubka [ 02/May/17 ]

I still hit the problem with NPE.
https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/2/archives/log.html.gz#s1-s14-t3-k2-k1-k1-k6-k1-k1-k2-k1-k1-k3-k4-k1

Here the suite restarts (kill and ./bin/start) all 3 odl nodes.

Comment by Ryan Goulding [ 02/May/17 ]

Okay I see... dataBroker is null which means that AAAShiroProvider.getInstance().getDataBroker() is returning null. My guess is that there is some sort of race or the container is restarted resulting on null dataBroker. I will try injecting the bean directly into the MDSALDynamicAuthorizationFilter instead and avoid the kludge of managing lifecycle in java code...

Comment by Viera Zelcamova [ 04/May/17 ]

Hi Ryan,
do you have any update for us? We are still hitting this error. Thanks.

Comment by Peter Gubka [ 04/May/17 ]

Stored logs from job mentioned in comment 10. Robot's log.html is not attached, because it is bigger tham 1MB.

Comment by Peter Gubka [ 04/May/17 ]

Attachment logs.tar has been added with description: stored logs from comment 10

Comment by Ryan Goulding [ 04/May/17 ]

I think this should fix it:

https://git.opendaylight.org/gerrit/#/c/56540/

This is a workaround due to AAA-118 which we need to address in Nitrogen.

Comment by Ryan Goulding [ 04/May/17 ]

https://git.opendaylight.org/gerrit/#/c/56548/

Comment by Ryan Goulding [ 04/May/17 ]

Leaving this as Waiting For Review until you verify it is fixed, Peter.

Comment by Vratko Polak [ 05/May/17 ]

https://git.opendaylight.org/gerrit/56548 is merged, so setting status to FIXED. Now it is time for Peter to look at the new results and reopen, or set VERIFIED.

Generated at Wed Feb 07 19:08:41 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.