[NETCONF-392] Transaction is already opened for another user Created: 04/Apr/17  Updated: 15/Mar/19  Resolved: 27/Apr/17

Status: Resolved
Project: netconf
Component/s: netconf
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Miroslav Kovac Assignee: Andrej Mak
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


External issue ID: 8152

 Description   

This happens in netconf-csit-3node-periodic-clustering-scale-only-carbon job.

2017-03-27 09:06:12,284 | ERROR | tp1576511659-502 | NetconfProxyDOMTransaction | 289 - org.opendaylight.netconf.topology-singleton - 1.2.0.SNAPSHOT | RemoteDevice

{17830-sim-device}

: Failed to open new transaction
java.lang.IllegalStateException: Transaction is already opened for another user
at org.opendaylight.netconf.topology.singleton.impl.RemoteOperationTxProcessorImpl.doOpenTransaction(RemoteOperationTxProcessorImpl.java:56)
at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfNodeActor.resolveProxyCalls(NetconfNodeActor.java:156)
at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfNodeActor.onReceive(NetconfNodeActor.java:128)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:497)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2017-03-27 09:06:12,286 | ERROR | tp1576511659-502 | ContainerResponse | 68 - com.sun.jersey.jersey-server - 1.17.0 | The RuntimeException could not be mapped to a response, re-throwing to the HTTP container
java.lang.IllegalStateException: Transaction is already opened for another user
at org.opendaylight.netconf.topology.singleton.impl.RemoteOperationTxProcessorImpl.doOpenTransaction(RemoteOperationTxProcessorImpl.java:56)
at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfNodeActor.resolveProxyCalls(NetconfNodeActor.java:156)
at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfNodeActor.onReceive(NetconfNodeActor.java:128)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:497)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2017-03-27 09:06:12,287 | WARN | tp1576511659-502 | ServletHandler | 44 - org.eclipse.jetty.aggregate.jetty-all-server - 8.1.19.v20160209 |
javax.servlet.ServletException: java.lang.IllegalStateException: Transaction is already opened for another user
at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:69)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:240)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:75)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:984)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1045)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:236)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Transaction is already opened for another user
at org.opendaylight.netconf.topology.singleton.impl.RemoteOperationTxProcessorImpl.doOpenTransaction(RemoteOperationTxProcessorImpl.java:56)
at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfNodeActor.resolveProxyCalls(NetconfNodeActor.java:156)
at org.opendaylight.netconf.topology.singleton.impl.actors.NetconfNodeActor.onReceive(NetconfNodeActor.java:128)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:497)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)



 Comments   
Comment by Tomas Cere [ 04/Apr/17 ]

which run did this happen on? This seems like multiple clients are trying to access the same device concurrently, which is currently not supported.

Comment by Miroslav Kovac [ 04/Apr/17 ]

https://logs.opendaylight.org/releng/jenkins092/netconf-csit-3node-periodic-clustering-scale-only-carbon/222/archives/odl3_karaf.log.gz

https://logs.opendaylight.org/releng/jenkins092/netconf-csit-3node-periodic-clustering-scale-all-carbon/218/archives/odl3_karaf.log.gz

Comment by Andrej Mak [ 19/Apr/17 ]

https://git.opendaylight.org/gerrit/#/c/55187/
https://git.opendaylight.org/gerrit/#/c/55192/

Current netconf device data broker supports only one opened write transaction per device. If user creates transaction and leave it open(e.g. akka message loss due to network problems, slave node failure before submit etc.), it will lock device forever.
This patches add configurable auto-cancellation after time idle, which unlocks device and allows it to be usable again.

Comment by Andrej Mak [ 25/Apr/17 ]

Carbon:
https://git.opendaylight.org/gerrit/#/c/55866/
https://git.opendaylight.org/gerrit/#/c/55975/
Boron:
https://git.opendaylight.org/gerrit/#/c/55873/
https://git.opendaylight.org/gerrit/#/c/55978/

Generated at Wed Feb 07 20:14:54 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.