[VTN-44] System test for VTN Manager fails. Created: 07/Oct/14 Updated: 19/Oct/17 Resolved: 30/Jun/15 |
|
| Status: | Resolved |
| Project: | vtn |
| Component/s: | VTN Manager |
| Affects Version/s: | unspecified |
| Fix Version/s: | None |
| Type: | Bug | ||
| Reporter: | Hideyuki Tai | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Operating System: All |
||
| Issue Links: |
|
||||||||
| External issue ID: | 2158 | ||||||||
| Description |
|
Continuous System Integration Test for VTN Manager features failed continuously. Tests from #165 to #174 of the above Jenkins job failed. Sometimes it failed to forward ping packets. Other times it failed to delete a virtual tenant network. |
| Comments |
| Comment by Hideyuki Tai [ 08/Oct/14 ] |
|
We've analyzed the failure of build #173 of the csit-karaf-vtn-all. The #173 succeeded to forward ping packets between h1 and h3 on a virtual bridge "vBridge1". We've also checked the log file of the #173. We've found out that TopologyManager received the notification of the edge addition (s3-eth2 <-> s1-eth2) before the notification of Node & NodeConnector (s3, s3-eth2, and s1-eth2) addition so that TopologyManager ignored the edge. 2014-10-06 08:58:02,394 | WARN | DOM-OPER-DCL-8 | TopologyManagerImpl | 359 - org.opendaylight.controller.topologymanager - 0.4.2.SNAPSHOT | Ignore edge that contains invalid node connector: (OF|3@OF|00:00:00:00:00:00:00:03->OF|2@OF|00:00:00:00:00:00:00:01) 2014-10-06 08:58:02,398 | WARN | DOM-OPER-DCL-8 | TopologyManagerImpl | 359 - org.opendaylight.controller.topologymanager - 0.4.2.SNAPSHOT | Ignore edge that contains invalid node connector: (OF|2@OF|00:00:00:00:00:00:00:01->OF|3@OF|00:00:00:00:00:00:00:03) 2014-10-06 08:58:02,401 | INFO | DOM-OPER-DCL-6 | VTNManagerImpl | 450 - org.opendaylight.vtn.manager.implementation - 0.2.0.SNAPSHOT | default: addNode: New node OF|00:00:00:00:00:00:00:03 2014-10-06 08:58:02,401 | INFO | DOM-OPER-DCL-6 | VTNManagerImpl | 450 - org.opendaylight.vtn.manager.implementation - 0.2.0.SNAPSHOT | default: addPort: New port: port=OF|2@OF|00:00:00:00:00:00:00:03, prop=PortProperty[name=s3-eth2,cost=1000,enabled] 2014-10-06 08:58:02,416 | INFO | DOM-OPER-DCL-6 | VTNManagerImpl | 450 - org.opendaylight.vtn.manager.implementation - 0.2.0.SNAPSHOT | default: addPort: New port: port=OF|2@OF|00:00:00:00:00:00:00:01, prop=PortProperty[name=s1-eth2,cost=1000,enabled] In this case, we think that VTN Manager would output an ERROR message saying a Path Fault occurred, and it would fail to forward ping packets between h1 and h3 as well. Further investigation is required for this bug. |
| Comment by Hideyuki Tai [ 09/Oct/14 ] |
|
I've also analyzed the failure of build #175 of the csit-karaf-vtn-all. https://jenkins.opendaylight.org/integration/job/integration-master-csit-karaf-vtn-all/175/ On the #175, the exactly same thing happened. It means that TopologyManager received the notification of the edge addition (s3-eth2 <-> s1-eth2) before the notification of Node & NodeConnector (s3, s3-eth2, and s1-eth2) addition so that TopologyManager ignored the edge. |
| Comment by Hideyuki Tai [ 10/Oct/14 ] |
|
https://jenkins.opendaylight.org/integration/job/integration-master-csit-karaf-vtn-all/174/ In the #174, the following eight tests failed. 1. Delete a vtn Tenant1. The last seven tests assume that the first test (Delete a vtn Tenant1) successes and it cleans up all VTN configuration. Therefore, I've focused on analyzing why it was failed to delete "Tenant1". The test client output the following error message for the failure of the deletion of "Tenant1". ConnectionError: [Errno 104] Connection reset by peer I think it means that the controller sent a RST packet to the test client. The #172 also failed due to the exactly same reason. Anyway, I think the root cause of the failure of the #174 is different from the root cause of #173. |
| Comment by Luis Gomez [ 27/Oct/14 ] |
|
Hi Hideyuki, After trying with my local device, I do not really see any ping failure BUT I do see the problem of deleting tenant time to time. ConnectionError: [Errno 104] Connection reset by peer BR/Luis |
| Comment by Hideyuki Tai [ 28/Oct/14 ] |
|
Hi Luis, Thank you for sharing the observation! We think the > ConnectionError: [Errno 104] Connection reset by peer I'm not sure why this happened. |
| Comment by Ed Warnicke [ 12/Nov/14 ] |
|
Is this a TCP RST being sent to a REST client? |
| Comment by Hideyuki Tai [ 19/Nov/14 ] |
|
(In reply to Ed Warnicke from comment #6) I think so. |
| Comment by Hideyuki Tai [ 01/Dec/14 ] |
|
I've submitted a patch for this issue into controller.git. |
| Comment by Kuldip [ 19/Dec/14 ] |
|
After starting my Karaf I see Karaf becoming very slow. The main packages installed are : odl-nsf-all, odl-openflowplugin-all, odl-adsal-compatibility-all, and odl-vtn-manager-all . Is there any conflict in this. The modules which are installed in karaf include: |
| Comment by Hideyuki Tai [ 06/Jan/15 ] |
|
Patches to the Topology Manager for the bug have been merged into the master branch and the stable/helium branch of the controller.git. [master] [stable/helium] |
| Comment by Hideyuki Tai [ 30/Jun/15 ] |
|
Recently, all CSIT jobs for VTN project works well. https://jenkins.opendaylight.org/releng/view/vtn/job/vtn-csit-1node-cds-manager-all-master/ I think the following patches also helped us. https://git.opendaylight.org/gerrit/#/c/22650/ (stable/lithium) |