[GENIUS-46] Leftovers in dispatcher table when unbind and ietf-state delete occurs simultaneously Created: 27/Dec/16  Updated: 19/May/17  Resolved: 19/May/17

Status: Resolved
Project: genius
Component/s: General
Affects Version/s: Carbon
Fix Version/s: None

Type: Bug
Reporter: Tali Ben-Meir Assignee: Faseela K
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

External issue ID: 7451

 Description   

If FlowBasedServicesConfigListener.remove() needs ietf-interfce from oper DS in order to delete T17 flow
FlowBasedServicesInterfaceStateListener.remove() needs the bound-service in order to delete T17 flows.
If deletion of both takes place in a short timeframe, there could be case where each listener won't be able to find the other model and the dispatcher flow will remain.
The consequences are beyond stale entries - it can affect future service bindings recycling the same lport-tag. New traffic may be hijacked by a stale entry having higher OF priority.

actions=write_metadata:0x40000000000/0xffffff0000000001,goto_table:17
cookie=0x8000000, duration=181.469s, table=0, n_packets=30, n_bytes=2632, priority=4,in_port=22 actions=write_metadata:0xc0000400000222e0/0xfffffffffffffffe,goto_table:19
cookie=0x8040000, duration=186.036s, table=17, n_packets=33, n_bytes=2758, priority=6,metadata=0xc000040000000000/0xffffff0000000000 actions=write_metadata:0xe00004138e000000/0xfffffffffffffffe,goto_table:48
cookie=0x8040000, duration=181.369s, table=17, n_packets=24, n_bytes=2164, priority=5,metadata=0xa000040000000000/0xffffff0000000000 actions=write_metadata:0xc000040000022370/0xfffffffffffffffe,goto_table:19
cookie=0x6900000, duration=68.916s, table=17, n_packets=0, n_bytes=0, priority=1,metadata=0x40000000000/0xffffff0000000000 actions=write_metadata:0xa000040000000000/0xfffffffffffffffe,goto_table:40

 Comments   
Comment by Alon Kochba [ 02/Apr/17 ]

This is blocking behavior, occurring a lot in CSIT automation lately - if another VM receives the same lport after this happens, it will NOT work.
Happening both on Carbon and on Boron.

example:
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-carbon/284/archives/log.html.gz#s1-s1-s1-t27-k3-k1-k2-k1-k15-k4

Comment by Tali Ben-Meir [ 02/Apr/17 ]

ELAN Binding removed from config DS when interface state still exist

2017-04-01 23:04:00,074 | INFO | nPool-1-worker-0 | FlowBasedServicesConfigListener | 331 - org.opendaylight.genius.interfacemanager-impl - 0.2.0.SNAPSHOT | Service Binding Entry removed for Interface: 206891615084733:br-physnet1-pa:1235, Data: BoundServices{getServiceName=elan.f84916de-6be3-4a5e-bdd5-17c8dd91b53e.206891615084733:br-physnet1-pa:1235, getServicePriority=9, getServiceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.servicebinding.rev160406.ServiceTypeFlowBased, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.genius.interfacemanager.servicebinding.rev160406.StypeOpenflow=StypeOpenflow{getFlowCookie=134479872, getFlowPriority=5, getInstruction=[Instruction{getInstruction=ApplyActionsCase{getApplyActions=ApplyActions{getAction=[Action{getAction=ServiceBindingNxActionRegLoadApplyActionsCase{getNxRegLoad=NxRegLoad{getDst=Dst{getDstChoice=DstNxRegCase{getNxReg=class org.opendaylight.yang.gen.v1.urn.opendaylight.openflowjava.nx.match.rev140421.NxmNxReg1, augmentations={}}, getEnd=19, getStart=0, augmentations={}}, getValue=1, augmentations={}}, augmentations={}}, getOrder=0, augmentations={}}], augmentations={}}, augmentations={}}, getOrder=2, augmentations={}}, Instruction{getInstruction=WriteMetadataCase{getWriteMetadata=WriteMetadata{getMetadata=83919634432, getMetadataMask=1099494850560, augmentations={}}, augmentations={}}, getOrder=1, augmentations={}}, Instruction{getInstruction=GoToTableCase{getGoToTable=GoToTable{getTableId=48, augmentations={}}, augmentations={}}, getOrder=3, augmentations={}}]}}}

Interface state deleted

2017-04-01 23:04:00,091 | INFO | nPool-1-worker-1 | IngressServicesStateUnbindHelper | 331 - org.opendaylight.genius.interfacemanager-impl - 0.2.0.SNAPSHOT | unbind all ingress services on interface 206891615084733:br-physnet1-pa:1235
2017-04-01 23:04:00,091 | TRACE | nPool-1-worker-1 | IngressServicesStateUnbindHelper | 331 - org.opendaylight.genius.interfacemanager-impl - 0.2.0.SNAPSHOT | bound services is empty for interface 206891615084733:br-physnet1-pa:1235

At the same time, FlowBasedIngressServicesConfigUnbindHelper.unbindService() triggered from config DS deletion can't find the state

2017-04-01 23:04:00,091 | INFO | nPool-1-worker-1 | ngressServicesConfigUnbindHelper | 331 - org.opendaylight.genius.interfacemanager-impl - 0.2.0.SNAPSHOT | Interface not operational, not unbinding Service for Interface: 206891615084733:br-physnet1-pa:1235

Comment by Faseela K [ 13/Apr/17 ]

Hi,
I have started working on this.
Will post updates shortly.
Thanks,
Faseela

Comment by Faseela K [ 16/May/17 ]

Patches up for review :

https://git.opendaylight.org/gerrit/#/c/55860/12

https://git.opendaylight.org/gerrit/#/c/55926/

second patch hitting some csit infra issues, hoping to get resolved soon

Comment by A H [ 17/May/17 ]

Can the GENIUS project update the status of this bug? With patch https://git.opendaylight.org/gerrit/#/c/56736/ successfully merged, should we update the status to fixed and resolved?

Comment by Sam Hague [ 17/May/17 ]

(In reply to A H from comment #5)
> Can the GENIUS project update the status of this bug? With patch
> https://git.opendaylight.org/gerrit/#/c/56736/ successfully merged, should
> we update the status to fixed and resolved?

An, there are two patches associated with this bug. The first is ready, but the second had issues with infra verification so we are retrying that now.

Comment by A H [ 17/May/17 ]

We are looking to build RC2 tomorrow 5/18 at 23:59 UTC time assuming there are no blocker bugs. Is there an ETA for when a fix can be merged and this bug resolved?

Generated at Wed Feb 07 19:59:46 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.