[OPNFLWPLUG-226] AD-SAL topology unstable Created: 02/Aug/14  Updated: 27/Sep/21  Resolved: 17/Sep/14

Status: Resolved
Project: OpenFlowPlugin
Component/s: General
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Luis Gomez Assignee: Luis Gomez
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Linux
Platform: PC

External issue ID: 1468

 Description   

If you see this log topology information gets earlier than the switch information and so it gets rejected by topology manager:

2014-07-31 20:30:46.181 UTC [nioEventLoopGroup-12-1] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection from (remote address): /192.168.198.27:53041 --> :6633
2014-07-31 20:30:46.181 UTC [nioEventLoopGroup-12-1] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection accepted - building pipeline
2014-07-31 20:30:46.183 UTC [nioEventLoopGroup-12-1] WARN o.o.o.o.m.c.s.SessionManagerOFImpl - context for invalidation not found
2014-07-31 20:30:46.592 UTC [nioEventLoopGroup-12-2] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection from (remote address): /192.168.198.27:53042 --> :6633
2014-07-31 20:30:46.592 UTC [nioEventLoopGroup-12-2] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection accepted - building pipeline
2014-07-31 20:30:46.597 UTC [nioEventLoopGroup-12-2] INFO o.o.o.p.i.c.ConnectionAdapterImpl - Hello received / branch
2014-07-31 20:30:46.601 UTC [nioEventLoopGroup-12-2] WARN o.o.o.p.i.c.ResponseExpectedRpcListener - Request for RpcResultKey [xid=6525, outputClazz=org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.protocol.rev130731.GetFeaturesOutput] did not receive a response
2014-07-31 20:30:46.635 UTC [nioEventLoopGroup-12-3] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection from (remote address): /192.168.198.27:53043 --> :6633
2014-07-31 20:30:46.636 UTC [nioEventLoopGroup-12-3] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection accepted - building pipeline
2014-07-31 20:30:46.637 UTC [nioEventLoopGroup-12-3] INFO o.o.o.p.i.c.ConnectionAdapterImpl - Hello received / branch
2014-07-31 20:30:46.641 UTC [nioEventLoopGroup-12-3] WARN o.o.o.p.i.c.ResponseExpectedRpcListener - Request for RpcResultKey [xid=6526, outputClazz=org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.protocol.rev130731.GetFeaturesOutput] did not receive a response
2014-07-31 20:30:46.657 UTC [pool-18-thread-1] WARN o.o.c.t.internal.TopologyManagerImpl - Ignore edge that contains invalid node connector: (OF|3@OF|00:00:00:00:00:00:00:02->OF|1@OF|00:00:00:00:00:00:00:01)
2014-07-31 20:30:46.657 UTC [pool-18-thread-1] ERROR o.o.c.s.c.t.TopologyCommitHandler - Exception caught
java.lang.NullPointerException: null
at org.opendaylight.controller.topologymanager.internal.TopologyManagerImpl.edgeUpdate(TopologyManagerImpl.java:655) ~[na:na]
at org.opendaylight.controller.topologymanager.internal.TopologyManagerImpl.edgeUpdate(TopologyManagerImpl.java:706) ~[na:na]
at org.opendaylight.controller.sal.implementation.internal.Topology.edgeUpdate(Topology.java:95) ~[na:na]
at org.opendaylight.controller.sal.compatibility.topology.TopologyCommitHandler.onDataChanged(TopologyCommitHandler.java:77) ~[na:na]
at org.opendaylight.controller.md.sal.binding.impl.ForwardedBackwardsCompatibleDataBroker$BackwardsCompatibleOperationalDataChangeInvoker.onDataChanged(ForwardedBackwardsCompatibleDataBroker.java:428) [bundlefile:na]
at org.opendaylight.controller.md.sal.binding.impl.AbstractForwardedDataBroker$TranslatingDataChangeInvoker.onDataChanged(AbstractForwardedDataBroker.java:198) [bundlefile:na]
at org.opendaylight.controller.md.sal.dom.store.impl.ChangeListenerNotifyTask.run(ChangeListenerNotifyTask.java:33) [bundlefile:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
2014-07-31 20:30:46.677 UTC [nioEventLoopGroup-12-4] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection from (remote address): /192.168.198.27:53044 --> :6633
2014-07-31 20:30:46.677 UTC [nioEventLoopGroup-12-4] INFO o.o.o.p.i.c.PublishingChannelInitializer - Incoming connection accepted - building pipeline
2014-07-31 20:30:46.679 UTC [nioEventLoopGroup-12-4] INFO o.o.o.p.i.c.ConnectionAdapterImpl - Hello received / branch
2014-07-31 20:30:46.681 UTC [nioEventLoopGroup-12-4] WARN o.o.o.p.i.c.ResponseExpectedRpcListener - Request for RpcResultKey [xid=6527, outputClazz=org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.protocol.rev130731.GetFeaturesOutput] did not receive a response
2014-07-31 20:30:46.695 UTC [pool-18-thread-1] WARN o.o.c.t.internal.TopologyManagerImpl - Ignore edge that contains invalid node connector: (OF|3@OF|00:00:00:00:00:00:00:03->OF|2@OF|00:00:00:00:00:00:00:01)
2014-07-31 20:30:46.696 UTC [pool-18-thread-1] ERROR o.o.c.s.c.t.TopologyCommitHandler - Exception caught
java.lang.NullPointerException: null
at org.opendaylight.controller.topologymanager.internal.TopologyManagerImpl.edgeUpdate(TopologyManagerImpl.java:655) ~[na:na]
at org.opendaylight.controller.topologymanager.internal.TopologyManagerImpl.edgeUpdate(TopologyManagerImpl.java:706) ~[na:na]
at org.opendaylight.controller.sal.implementation.internal.Topology.edgeUpdate(Topology.java:95) ~[na:na]
at org.opendaylight.controller.sal.compatibility.topology.TopologyCommitHandler.onDataChanged(TopologyCommitHandler.java:77) ~[na:na]
at org.opendaylight.controller.md.sal.binding.impl.ForwardedBackwardsCompatibleDataBroker$BackwardsCompatibleOperationalDataChangeInvoker.onDataChanged(ForwardedBackwardsCompatibleDataBroker.java:428) [bundlefile:na]
at org.opendaylight.controller.md.sal.binding.impl.AbstractForwardedDataBroker$TranslatingDataChangeInvoker.onDataChanged(AbstractForwardedDataBroker.java:198) [bundlefile:na]
at org.opendaylight.controller.md.sal.dom.store.impl.ChangeListenerNotifyTask.run(ChangeListenerNotifyTask.java:33) [bundlefile:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]

 Comments   
Comment by Kamal Rameshan [ 04/Aug/14 ]

Hi Luis,

Can you give more details about the distribution used to test and/or steps?

From the logs, it seems to be OF1.0 version.

I ran latest OFPlugin distribution against mininet OF1.0 Topo=tree,3.

I am not able to reproduce this defect, with both OF1.3 and OF1.0 .

thx
Kamal

Comment by Luis Gomez [ 04/Aug/14 ]

Hi Kamal,

Thanks for looking at this issue. This seems like an intermittent behavior regardless of OF10 or OF13 simulation and also the topology used. This issue is clearly detected in the automation when running several times the following sequence:

Pre-condition: start controller base edition (new plugin)

1) start mininet tree,2 (OF10 or OF13)
2) wait few secs
3) get AD-SAL topology or do pingall test (this fails too if links are not OK)
GET http://127.0.0.1:8080/controller/nb/v2/topology/default
4) stop mininet

You can take a look in existing automation report for a run that fails the topology or the ping like this one:

https://jenkins.opendaylight.org/integration/job/integration-master-csit-base-of13/2038/robot/report/log.html

But probably the easier way to reproduce locally is to download and run this robot test several times:

integration/test/csit/suites/base-of13/030__AD_SAL_NSF

There is a wiki on how to download and run system test:

https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Download_and_Run_System_Test

I can also guide you through all this in webex or hangout.

BR/Luis

Comment by Kamal Rameshan [ 05/Aug/14 ]

Luis in email , confirmed that he is not able to reproduce it manually and jenkins build is also not showing topology issues.

It was decided we would test out after a few days , and if the issue is not reproducible, we will close it.

Comment by Luis Gomez [ 08/Sep/14 ]

It still happens time to time in base distro, however I have not seen this issue yet with karaf packaging.

Comment by Abhijit Kumbhare [ 16/Sep/14 ]

Luis,

Any update? Should we move this to Lithium?

Thanks,
Abhijit

Comment by Luis Gomez [ 17/Sep/14 ]

It happens very rare now, lets close this bug.

Generated at Wed Feb 07 20:31:54 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.