[NETVIRT-1363] FLOWs are getting recreated once the VM deleted and configured again even without subnet ip

Wed, 5 Sep 2018 12:57:46 +0000

Topology
VM1 VM2

L3VPN

Compute-node-1 ---------- Compute-node-2

Observation

FIB Entries and FLOWs are getting recreated once the VM deleted and configured again(even without subnet ipv6)

Steps to reproduce
1) configure two VM with different subnet IPv4/IPv6 Address(Via DHCP)

2) configure ipv6 subnet address on VM1 and verify traffic from VM2(traffic was working)

3) Delete the VMs(Flows were got deleted and FIB entries were removed)

4) Configure again the VM with same name( not configured the IPv6 Subnet)
Logs

Attached karaf for your reference

stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=45,
cookie=0x4000000, duration=730.411s, table=45, n_packets=1, n_bytes=70, priority=50,icmp6,metadata=0x1389000000/0xffff000000,icmp_type=133,icmp_code=0 actions=CONTROLLER:65535
cookie=0x4000000, duration=730.336s, table=45, n_packets=1, n_bytes=86, priority=50,icmp6,metadata=0x1389000000/0xffff000000,icmp_type=135,icmp_code=0,nd_target=2001:db8:0:2::1 actions=CONTROLLER:65535
cookie=0x4000000, duration=730.336s, table=45, n_packets=1, n_bytes=86, priority=50,icmp6,metadata=0x1389000000/0xffff000000,icmp_type=135,icmp_code=0,nd_target=fe80::f816:3eff:fe47:2ae8 actions=CONTROLLER:65535
cookie=0x4000000, duration=730.336s, table=45, n_packets=0, n_bytes=0, priority=50,icmp6,metadata=0x1389000000/0xffff000000,ipv6_src=2001:db8:0:2:f816:3eff:fe43:294,icmp_type=136,icmp_code=0 actions=resubmit(,17)
cookie=0x4000000, duration=730.336s, table=45, n_packets=2, n_bytes=172, priority=40,icmp6,metadata=0x1389000000/0xffff000000,ipv6_src=2001:db8:0:2::/64,icmp_type=136,icmp_code=0 actions=CONTROLLER:65535,resubmit(,17)
cookie=0x4000000, duration=881.760s, table=45, n_packets=16, n_bytes=1672, priority=0 actions=resubmit(,17)
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=21,
cookie=0x8000003, duration=735.108s, table=21, n_packets=0, n_bytes=0, priority=138,icmp6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::1,icmp_type=128,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]~~>NXM_OF_ETH_DST[],set_field:fa:16:3e:f9:b7:47~~>eth_src,move:NXM_NX_IPV6_SRC[]~~>NXM_NX_IPV6_DST[],set_field:2001:db8:0:3::1~~>ipv6_src,set_field:129->icmpv6_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=735.108s, table=21, n_packets=0, n_bytes=0, priority=138,icmp6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::1,icmp_type=128,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]~~>NXM_OF_ETH_DST[],set_field:fa:16:3e:47:2a:e8~~>eth_src,move:NXM_NX_IPV6_SRC[]~~>NXM_NX_IPV6_DST[],set_field:2001:db8:0:2::1~~>ipv6_src,set_field:129->icmpv6_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=734.996s, table=21, n_packets=0, n_bytes=0, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2:f816:3eff:fe43:294 actions=group:150000
cookie=0x8000003, duration=681.329s, table=21, n_packets=10, n_bytes=1180, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3:f816:3eff:fecd:7ea5 actions=set_field:0x186a9->tun_id,set_field:fa:16:3e:cd:7e:a5->eth_dst,load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=115.418s, table=21, n_packets=0, n_bytes=0, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::5050 actions=group:150000
cookie=0x8000003, duration=735.270s, table=21, n_packets=1, n_bytes=86, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::/64 actions=write_metadata:0x1389030d46/0xfffffffffe,goto_table:22
cookie=0x8000003, duration=681.429s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::/64 actions=write_metadata:0x138a030d46/0xfffffffffe,goto_table:22
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$

stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=21,
cookie=0x8000003, duration=797.378s, table=21, n_packets=10, n_bytes=1180, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3:f816:3eff:fecd:7ea5 actions=set_field:0x186a9->tun_id,set_field:fa:16:3e:cd:7e:a5->eth_dst,load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=797.478s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::/64 actions=write_metadata:0x138a030d46/0xfffffffffe,goto_table:22
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=21,
cookie=0x8000003, duration=906.446s, table=21, n_packets=10, n_bytes=1180, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3:f816:3eff:fecd:7ea5 actions=set_field:0x186a9->tun_id,set_field:fa:16:3e:cd:7e:a5->eth_dst,load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=906.547s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::/64 actions=write_metadata:0x138a030d46/0xfffffffffe,goto_table:22
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=45,
cookie=0x4000000, duration=965.730s, table=45, n_packets=1, n_bytes=70, priority=50,icmp6,metadata=0x1389000000/0xffff000000,icmp_type=133,icmp_code=0 actions=CONTROLLER:65535
cookie=0x4000000, duration=965.655s, table=45, n_packets=1, n_bytes=86, priority=50,icmp6,metadata=0x1389000000/0xffff000000,icmp_type=135,icmp_code=0,nd_target=2001:db8:0:2::1 actions=CONTROLLER:65535
cookie=0x4000000, duration=965.655s, table=45, n_packets=1, n_bytes=86, priority=50,icmp6,metadata=0x1389000000/0xffff000000,icmp_type=135,icmp_code=0,nd_target=fe80::f816:3eff:fe47:2ae8 actions=CONTROLLER:65535
cookie=0x4000000, duration=965.655s, table=45, n_packets=3, n_bytes=258, priority=40,icmp6,metadata=0x1389000000/0xffff000000,ipv6_src=2001:db8:0:2::/64,icmp_type=136,icmp_code=0 actions=CONTROLLER:65535,resubmit(,17)
cookie=0x4000000, duration=1117.079s, table=45, n_packets=16, n_bytes=1672, priority=0 actions=resubmit(,17)
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep 5050 --------> VM got deleted by this time
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$ openstack port show vpn6_net_1_port_1
------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Field

Value

------------------------------------------------------------------------------------------------------------------------------------------------------------------+

admin_state_up	UP
allowed_address_pairs	ip_address='2001:db8:0:2::5050', mac_address='fa:16:3e:43:02:94'
binding_host_id	openstack1
binding_profile
binding_vif_details	has_datapath_type_netdev='False', host_addresses='[u'openstack1']', support_vhost_user='False', uuid='5b4715b8-ce3d-453e-9447-f46b6b682319'
binding_vif_type	ovs
binding_vnic_type	normal
created_at	2018-07-03T17:53:09Z
data_plane_status	None
description
device_id	7fd7ffb9-017b-4544-8413-153be0e5d23d
device_owner	compute:nova
dns_assignment	None
dns_name	None
extra_dhcp_opts
fixed_ips	ip_address='2001:db8:0:2:f816:3eff:fe43:294', subnet_id='68d48d93-80c5-4773-81e6-108a26713d9c'
id	874d267d-6298-477f-9228-576f4f3b3e6e
ip_address	None
mac_address	fa:16:3e:43:02:94
name	vpn6_net_1_port_1
network_id	421f7b97-e0f2-4ed6-b0ef-251883776b39
option_name	None
option_value	None
port_security_enabled	True
project_id	3c684838d1fb4534888b478417e60ed9
qos_policy_id	None
revision_number	10
security_group_ids	a7df7988-3fcb-416b-84f9-93814d4215a0
status	ACTIVE
subnet_id	None
tags
trunk_details	None
updated_at	2018-07-03T18:14:17Z

------------------------------------------------------------------------------------------------------------------------------------------------------------------+
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=21,
cookie=0x8000003, duration=246.792s, table=21, n_packets=10, n_bytes=1180, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3:f816:3eff:fecd:7ea5 actions=set_field:0x186a9->tun_id,set_field:fa:16:3e:cd:7e:a5->eth_dst,load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=246.417s, table=21, n_packets=0, n_bytes=0, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::5050 actions=group:150003
cookie=0x8000003, duration=245.859s, table=21, n_packets=0, n_bytes=0, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2:f816:3eff:fe43:294 actions=group:150003
cookie=0x8000003, duration=246.849s, table=21, n_packets=0, n_bytes=0, priority=138,icmp6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::1,icmp_type=128,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]~~>NXM_OF_ETH_DST[],set_field:fa:16:3e:47:2a:e8~~>eth_src,move:NXM_NX_IPV6_SRC[]~~>NXM_NX_IPV6_DST[],set_field:2001:db8:0:2::1~~>ipv6_src,set_field:129->icmpv6_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=246.825s, table=21, n_packets=0, n_bytes=0, priority=138,icmp6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::1,icmp_type=128,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]~~>NXM_OF_ETH_DST[],set_field:fa:16:3e:f9:b7:47~~>eth_src,move:NXM_NX_IPV6_SRC[]~~>NXM_NX_IPV6_DST[],set_field:2001:db8:0:3::1~~>ipv6_src,set_field:129->icmpv6_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=246.849s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::/64 actions=write_metadata:0x1389030d46/0xfffffffffe,goto_table:22
cookie=0x8000003, duration=246.792s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::/64 actions=write_metadata:0x138a030d46/0xfffffffffe,goto_table:22
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=21, | grep 505
cookie=0x8000003, duration=259.355s, table=21, n_packets=0, n_bytes=0, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::5050 actions=group:150003 --> Even without subnet ip
stack@openstack1:/home/stack/new/fromgit$ sudo ovs-ofctl -OOpenflow13 dump-flows br-int | grep table=21,
cookie=0x8000003, duration=270.345s, table=21, n_packets=10, n_bytes=1180, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3:f816:3eff:fecd:7ea5 actions=set_field:0x186a9->tun_id,set_field:fa:16:3e:cd:7e:a5->eth_dst,load:0x200->NXM_NX_REG6[],resubmit(,220)
cookie=0x8000003, duration=269.970s, table=21, n_packets=0, n_bytes=0, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::5050 actions=group:150003
cookie=0x8000003, duration=269.412s, table=21, n_packets=0, n_bytes=0, priority=138,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2:f816:3eff:fe43:294 actions=group:150003
cookie=0x8000003, duration=270.402s, table=21, n_packets=0, n_bytes=0, priority=138,icmp6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::1,icmp_type=128,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]~~>NXM_OF_ETH_DST[],set_field:fa:16:3e:47:2a:e8~~>eth_src,move:NXM_NX_IPV6_SRC[]~~>NXM_NX_IPV6_DST[],set_field:2001:db8:0:2::1~~>ipv6_src,set_field:129->icmpv6_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=270.378s, table=21, n_packets=0, n_bytes=0, priority=138,icmp6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::1,icmp_type=128,icmp_code=0 actions=move:NXM_OF_ETH_SRC[]~~>NXM_OF_ETH_DST[],set_field:fa:16:3e:f9:b7:47~~>eth_src,move:NXM_NX_IPV6_SRC[]~~>NXM_NX_IPV6_DST[],set_field:2001:db8:0:3::1~~>ipv6_src,set_field:129->icmpv6_type,load:0->NXM_OF_IN_PORT[],resubmit(,21)
cookie=0x8000003, duration=270.402s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:2::/64 actions=write_metadata:0x1389030d46/0xfffffffffe,goto_table:22
cookie=0x8000003, duration=270.345s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30d46/0xfffffe,ipv6_dst=2001:db8:0:3::/64 actions=write_metadata:0x138a030d46/0xfffffffffe,goto_table:22
stack@openstack1:/home/stack/new/fromgit$ openstack port set --allowed-address ip-address=2001:db8:0:2::5151 874d267d-6298-477f-9228-576f4f3b3e6e
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$
stack@openstack1:/home/stack/new/fromgit$

ODL
opendaylight-user@root>
~
opendaylight-user@root>exception-displayfib-show
RD Prefix NextHop Label Origin
-------------------------------------------------------------------
1:1 2001:db8:0:2::1/128 0.0.0.0 100004 l
1:1 2001:db8:0:3::1/128 0.0.0.0 100005 l
1:1 2001:db8:0:3::/64 172.23.108.205 100008 c
1:1 2001:db8:0:3:f816:3eff:fecd:7ea5/128 172.23.108.205 100009 l
~
opendaylight-user@root>
opendaylight-user@root>fib-show
RD Prefix NextHop Label Origin
-------------------------------------------------------------------
1:1 2001:db8:0:2::/64 172.23.108.204 100012 c
1:1 2001:db8:0:2::1/128 0.0.0.0 100004 l
1:1 2001:db8:0:3::1/128 0.0.0.0 100005 l
1:1 2001:db8:0:2:0:0:0:5050/128 172.23.108.204 100013 s
1:1 2001:db8:0:2:f816:3eff:fe43:294/128 172.23.108.204 100011 l
1:1 2001:db8:0:3::/64 172.23.108.205 100008 c
1:1 2001:db8:0:3:f816:3eff:fecd:7ea5/128 172.23.108.205 100009 l
opendaylight-user@root>

stack@openstack1:~$ virsh list --all
Id Name State
----------------------------------------------------
162 instance-0000009a running

stack@openstack1:~$ virsh console 162
Connected to domain instance-0000009a
Escape character is ^]

$ ifconfig
eth0 Link encap:Ethernet HWaddr FA:16:3E:43:02:94
inet6 addr: 2001:db8:0:2:f816:3eff:fe43:294/64 Scope:Global
inet6 addr: fe80::f816:3eff:fe43:294/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:596 errors:0 dropped:143 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:69009 (67.3 KiB) TX bytes:1070 (1.0 KiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1020 (1020.0 B) TX bytes:1020 (1020.0 B)

stack@openstack2:~$ virsh list --all
Id Name State
----------------------------------------------------
51 instance-00000099 running

stack@openstack2:~$ virsh console 51
Connected to domain instance-00000099
Escape character is ^]

$
$ ifconfig
eth0 Link encap:Ethernet HWaddr FA:16:3E:CD:7E:A5
inet6 addr: 2001:db8:0:3:f816:3eff:fecd:7ea5/64 Scope:Global
inet6 addr: fe80::f816:3eff:fecd:7ea5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:445 errors:0 dropped:382 overruns:0 frame:0
TX packets:726 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:49564 (48.4 KiB) TX bytes:85902 (83.8 KiB)

OpenDaylight JIRA

[NETVIRT-1363] FLOWs are getting recreated once the VM deleted and configured again even without subnet ip