In releng, sometimes CSIT test cases are failing in Mininet Ping test cases. Also failure in differnt functionality in each failing time. By analyzing log seems it's a path-fault issue.

And this is occurs both in Beryllium and Boron.

This issue occurs only in releng, it is not reproduced in local machine.

Below is one of the scenarios in which Mininet Ping Test cases are failing due path-fault issue.

Test scenario:
==============
CSIT for "VBR IF Flowfilter" with action "vtn-set-vlan-pcp-action"

Topology used
=============
Created topology in mininet for Network with hosts in different vlan.
https://wiki.opendaylight.org/view/OpenDaylight_Virtual_Tenant_Network_(VTN):Scripts:Mininet#Network_with_hosts_in_different_vlan

Following are the steps followed in the CSIT test script for both OF10 and OF13.

1. Create Tenant1
2. Create vBridge1
3. Create interfaces(if1 and if2) in vBridge1
4. Create portmapping for if1 and if2.
5. Add flow condition to match h1, h3 ip address
6. Add VTN flowfilter to set flow action "vtn-set-vlan-pcp-action")
7. Ping h1 and h3
8. Verify ping Success ---> Here test case fails.
9. Read the flow entries in mininet switch s2 to verify actions=mod_vlan_pcp is set or not. --> Due to ping failure in before step here flow entry is not installed with vlan-pcp , so here also tets failure.

The above failure is in the below link of jenkins releng,
https://logs.opendaylight.org/releng/jenkins092/vtn-csit-1node-manager-only-beryllium/505/archives/log.html.gz#s1-s1-s6

I have analyzed the karaf.log for the above failure which is in below link,

https://logs.opendaylight.org/releng/jenkins092/vtn-csit-1node-manager-only-beryllium/505/archives/karaf.log.gz

From the above log i have observed, path-fault issue occur which makes vbridge status to down. Which in turn gives error while we ping between hosts.

2016-07-21 07:56:54,660 | TRACE | Runner: VTN Main | VTNPassFilter | 177 - org.opendaylight.vtn.manager.implementation - 0.4.3.SNAPSHOT | vBridge-IF:Tenant1/vBridge1/if1%IN.1: Flow action was applied: VTNSetVlanPcpAction[pcp=6,order=1]
2016-07-21 07:56:54,660 | TRACE | Runner: VTN Main | VTNPassFilter | 177 - org.opendaylight.vtn.manager.implementation - 0.4.3.SNAPSHOT | vBridge-IF:Tenant1/vBridge1/if1%IN.1: Packet matched the condition: cond_1
2016-07-21 07:56:54,665 | TRACE | TN Flow Thread-0 | FlowAddContext | 177 - org.opendaylight.vtn.manager.implementation - 0.4.3.SNAPSHOT | Flow entry has been installed: flow=[id=7f5600000000002b-1, pri=10, timeout=(0,0), node=openflow:1, ingress=openflow:1:2, cond=

{DL_SRC=d6:6a:2d:b5:2e:96,DL_DST=2a:f9:1d:d3:fd:9e,DL_VLAN=200}

, actions=

{OUTPUT(port=openflow:1:1, len=65535)}

2016-07-21 07:56:54,666 | WARN | on-dispatcher-70 | VTenantManager | 177 - org.opendaylight.vtn.manager.implementation - 0.4.3.SNAPSHOT | vBridge:Tenant1/vBridge1: Path fault: openflow:1 -> openflow:2
2016-07-21 07:56:54,666 | INFO | on-dispatcher-70 | VTenantManager | 177 - org.opendaylight.vtn.manager.implementation - 0.4.3.SNAPSHOT | vBridge:Tenant1/vBridge1: vBridge status has been changed: old=

{state=UP, path-faults=0}

, new=

{state=DOWN, path-faults=1}

2016-07-21 07:56:54,667 | TRACE | TN Flow Thread-0 | FlowAddContext | 177 - org.opendaylight.vtn.manager.implementation - 0.4.3.SNAPSHOT | Flow entry has been installed: flow=[id=7f5600000000002b-0, pri=10, timeout=(300,0), node=openflow:2, ingress=openflow:2:3, cond=

{DL_SRC=d6:6a:2d:b5:2e:96,DL_DST=2a:f9:1d:d3:fd:9e,DL_VLAN=200}

, actions=

{OUTPUT(port=openflow:2:1, len=65535)}