[CONTROLLER-457] OSGi errors after starting controller and mininet of10 simulation: ERROR o.o.c.sal.binding.impl.NotifyTask - Unhandled exception thrown by listener: org.opendaylight.controller.sal.compatibility.InventoryAndReadAdapter$$Broker$ListenerInvoker@51f1a91e Created: 07/May/14  Updated: 19/May/16  Resolved: 19/May/16

Status: Resolved
Project: controller
Component/s: adsal
Affects Version/s: Helium
Fix Version/s: None

Type: Bug
Reporter: Madhusudhan Ananderi Assignee: Giovanni Meo
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Windows
Platform: PC


External issue ID: 965

 Description   

After starting the controller and mininet of10 simulation, I get this error:

1. Link to latest controller: https://jenkins.opendaylight.org/integration/job/integration-project-centralized-integration/lastSuccessfulBuild/artifact/distributions/base/target/distributions-base-0.1.2-SNAPSHOT-osgipackage.zip
2. Start the controller ./run.sh -of13 -Xmx1G
3. Start the mininet: sudo mn --topo tree,2 --controller 'remote,ip=10.125.136.38,port=6633'

2014-05-06 17:41:57.190 PDT [md-sal-binding-notification-1384] ERROR o.o.c.sal.binding.impl.NotifyTask - Unhandled exception thrown by listener: org.opendaylight.controller.sal.compatibility.InventoryAndReadAdapter$$Broker$ListenerInvoker@51f1a91e
java.lang.NullPointerException: Null values are not supported!
at org.infinispan.CacheImpl.assertValueNotNull(CacheImpl.java:193) ~[na:na]
at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:963) ~[na:na]
at org.infinispan.CacheImpl.replace(CacheImpl.java:957) ~[na:na]
at org.infinispan.CacheImpl.replace(CacheImpl.java:1361) ~[na:na]
at org.infinispan.CacheImpl.replace(CacheImpl.java:250) ~[na:na]
at org.opendaylight.controller.switchmanager.internal.SwitchManager.addNode(SwitchManager.java:1039) ~[na:na]
at org.opendaylight.controller.switchmanager.internal.SwitchManager.updateNode(SwitchManager.java:1137) ~[na:na]
at org.opendaylight.controller.sal.implementation.internal.Inventory.updateNode(Inventory.java:115) ~[na:na]
at org.opendaylight.controller.sal.compatibility.InventoryAndReadAdapter.publishNodeUpdate(InventoryAndReadAdapter.java:1035) ~[na:na]
at org.opendaylight.controller.sal.compatibility.InventoryAndReadAdapter.onNodeUpdated(InventoryAndReadAdapter.java:635) ~[na:na]
at org.opendaylight.controller.sal.compatibility.InventoryAndReadAdapter$$Broker$ListenerInvoker.onNotification(InventoryAndReadAdapter$$Broker$ListenerInvoker.java) ~[na:na]
at org.opendaylight.controller.sal.binding.impl.NotifyTask.call(NotifyTask.java:59) ~[na:na]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) ~[na:1.7.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:166) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_25]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]

2014-05-06 17:41:57.366 PDT [pool-15-thread-1] ERROR o.o.c.m.s.b.i.ForwardedBackwardsCompatibleDataBroker - Tx: DOM-2570 Rollback - Datastore commit failed.
2014-05-06 17:41:57.366 PDT [pool-15-thread-1] ERROR o.o.m.c.t.m.FlowCapableTopologyExporter - Topology export failed for Tx:DOM-2570
2014-05-06 17:41:57.993 PDT [nioEventLoopGroup-12-1] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection from (remote address): /10.125.136.39:33256 --> :6633
2014-05-06 17:41:57.994 PDT [nioEventLoopGroup-12-1] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection accepted - building pipeline
2014-05-06 17:41:58.002 PDT [nioEventLoopGroup-12-1] INFO o.o.o.p.i.c.ConnectionAdapterImpl - Hello received / branch
2014-05-06 17:41:58.049 PDT [pool-15-thread-1] INFO o.o.c.m.s.d.b.i.DOMDataBrokerImpl$CommitCoordination - Tx: DOM-2583 Did not pass canCommit phase.
2014-05-06 17:41:58.049 PDT [pool-15-thread-1] ERROR o.o.c.m.s.b.i.ForwardedBackwardsCompatibleDataBroker - Tx: DOM-2583 Rollback - Datastore commit failed.
2014-05-06 17:41:58.049 PDT [pool-15-thread-1] ERROR o.o.m.c.t.m.FlowCapableTopologyExporter - Topology export failed for Tx:DOM-2583

In the topology GUI, I see the following for the first time:

Node ID - Ports
MD_SAL|openflow:1 0
MD_SAL|openflow:3 3
MD_SAL|openflow:2 3

If I try to attempt again, everything works fine, even I could not see any OSGi errors as well.

Thanks,
Madhusudhan



 Comments   
Comment by Tony Tkacik [ 10/Jun/14 ]

After small analysis it seems SwitchManager implementation is not thread-safe,
and if two events occured for node at a same time, it is possible to create a race
in:

org.opendaylight.controller.switchmanager.internal.SwitchManager.addNode()

nodePropsCurr (retieved from nodeProps field) are null when method start executing

nodeProps.putIfAbsent(node,nodeProps) returns non-null (other thread updated nodeProps for particular key),
which triggers
nodeProps.replace(node,nodePropsCurr,nodeProps) with nodePropsCurr which was null at start and this leads to nullpointerexception.

Comment by Tony Tkacik [ 14/Jul/14 ]

Implementation of addNode is not thread-safe, it captures stated of shared concurrent map at begining of addNode and then assumes same state for that
map during whole call of method.. Failure is caused because other thread
also invoked addNode concurrently and during one run of addNode underlying map was modified.

Comment by Carol Sanders [ 04/May/15 ]

This bug is part of the project to Move all ADSAL associated component bugs to ADSAL

Comment by Robert Varga [ 19/May/16 ]

AD-SAL has been removed, this issue will not be fixed.

Generated at Wed Feb 07 19:53:04 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.