|
Tested on: Bery-RC1,openstack(kilo)
Steps to Reproduce:
===================
1.Stacked the Openstack Controller & compute node1,node2
2.Created 2 networks and spawn vm's on each compute node(s)/networks.
3.Associated the one Floating IP address on atleast one VM/networks (across compute node1,node2
4.East-West traffic(across compute node),North-South traffic(across compute node) working fine.
5.Unstack the compute node-2 and restack
6.Spawn VM-x on compute node-2 and associated floating IP address.
7.VM-x unable to reach VM on compute node-1(i.e created on step 3) using local ip address (i.e east-west traffic not reachable).However its reachable via Floating IP of VM-x to VM on compute node-1.
Workaround:
=============
If we reboot VM's (on compute node-1),then VM-x (on restacked compute node-2) manage to reach both local IP address (east-west) as well as floating ip address (north-south).
Openflow table entries captures on both compute node before & after reboot of VM.
Here: 3.3.3.5 (compute node-1 VM's local IP),201.1.1.5 is floating IP address.
Before reboot:
============
openflow@os-compute1:~/kilo/devstack$ sudo ovs-ofctl -O openflow13 dump-flows br-int | grep "3.3.3.5"
cookie=0x0, duration=546.107s, table=20, n_packets=0, n_bytes=0, priority=1024,arp,tun_id=0x446,arp_tpa=3.3.3.5,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:c9:60:51>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163ec96051->NXM_NX_ARP_SHA[],load:0x3030305->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=529.757s, table=30, n_packets=105, n_bytes=11706, priority=1024,ip,in_port=2,nw_dst=201.1.1.5 actions=set_field:3.3.3.5->ip_dst,load:0x446->NXM_NX_REG3[],goto_table:40
cookie=0x0, duration=546.107s, table=40, n_packets=132, n_bytes=14900, priority=36001,ip,in_port=1,dl_src=fa:16:3e:c9:60:51,nw_src=3.3.3.5 actions=goto_table:50
cookie=0x0, duration=551.677s, table=70, n_packets=110, n_bytes=12728, priority=1024,ip,tun_id=0x446,nw_dst=3.3.3.5 actions=set_field:fa:16:3e:c9:60:51->eth_dst,goto_table:80
cookie=0x0, duration=529.718s, table=100, n_packets=109, n_bytes=12698, priority=512,ip,tun_id=0x446,dl_dst=fa:16:3e:05:a1:e8,nw_src=3.3.3.5 actions=set_field:fa:16:3e:79:94:9d->eth_src,dec_ttl,set_field:00:01:e8:00:ee:ee->eth_dst,set_field:201.1.1.5->ip_src,output:2
openflow@os-compute1:~/kilo/devstack$ sudo ovs-ofctl -O openflow13 dump-flows br-int | grep "201.1.1.5"
cookie=0x0, duration=613.934s, table=20, n_packets=1, n_bytes=60, priority=1024,arp,in_port=2,arp_tpa=201.1.1.5,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:79:94:9d>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e79949d->NXM_NX_ARP_SHA[],load:0xc9010105->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=613.903s, table=30, n_packets=105, n_bytes=11706, priority=1024,ip,in_port=2,nw_dst=201.1.1.5 actions=set_field:3.3.3.5->ip_dst,load:0x446->NXM_NX_REG3[],goto_table:40
cookie=0x0, duration=613.864s, table=100, n_packets=109, n_bytes=12698, priority=512,ip,tun_id=0x446,dl_dst=fa:16:3e:05:a1:e8,nw_src=3.3.3.5 actions=set_field:fa:16:3e:79:94:9d->eth_src,dec_ttl,set_field:00:01:e8:00:ee:ee->eth_dst,set_field:201.1.1.5->ip_src,output:2
==========================
openflow@os-controller:~/kilo/devstack$ sudo ovs-ofctl -O openflow13 dump-flows br-int | grep "3.3.3.5"
cookie=0x0, duration=564.976s, table=20, n_packets=2, n_bytes=84, priority=1024,arp,tun_id=0x446,arp_tpa=3.3.3.5,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:c9:60:51>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163ec96051->NXM_NX_ARP_SHA[],load:0x3030305->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=565.007s, table=70, n_packets=5, n_bytes=1022, priority=1024,ip,tun_id=0x446,nw_dst=3.3.3.5 actions=set_field:fa:16:3e:c9:60:51->eth_dst,goto_table:80
openflow@os-controller:~/kilo/devstack$ sudo ovs-ofctl -O openflow13 dump-flows br-int | grep "201.1.1.3"
cookie=0x0, duration=3999.766s, table=20, n_packets=1, n_bytes=60, priority=1024,arp,in_port=2,arp_tpa=201.1.1.3,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:55:14:9a>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e55149a->NXM_NX_ARP_SHA[],load:0xc9010103->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=3999.684s, table=30, n_packets=113, n_bytes=13062, priority=1024,ip,in_port=2,nw_dst=201.1.1.3 actions=set_field:3.3.3.4->ip_dst,load:0x446->NXM_NX_REG3[],goto_table:40
cookie=0x0, duration=3999.679s, table=100, n_packets=108, n_bytes=12000, priority=512,ip,tun_id=0x446,dl_dst=fa:16:3e:05:a1:e8,nw_src=3.3.3.4 actions=set_field:fa:16:3e:55:14:9a->eth_src,dec_ttl,set_field:00:01:e8:00:ee:ee->eth_dst,set_field:201.1.1.3->ip_src,output:2
openflow@os-controller:~/kilo/devstack$
================================================================================
After reboot
=============
openflow@os-compute1:~/kilo/devstack$ sudo ovs-ofctl -O openflow13 dump-flows br-int | grep "201.1.1.5"
cookie=0x0, duration=694.066s, table=20, n_packets=1, n_bytes=60, priority=1024,arp,in_port=2,arp_tpa=201.1.1.5,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:79:94:9d>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e79949d->NXM_NX_ARP_SHA[],load:0xc9010105->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=694.035s, table=30, n_packets=105, n_bytes=11706, priority=1024,ip,in_port=2,nw_dst=201.1.1.5 actions=set_field:3.3.3.5->ip_dst,load:0x446->NXM_NX_REG3[],goto_table:40
cookie=0x0, duration=693.996s, table=100, n_packets=109, n_bytes=12698, priority=512,ip,tun_id=0x446,dl_dst=fa:16:3e:05:a1:e8,nw_src=3.3.3.5 actions=set_field:fa:16:3e:79:94:9d->eth_src,dec_ttl,set_field:00:01:e8:00:ee:ee->eth_dst,set_field:201.1.1.5->ip_src,output:2
openflow@os-compute1:~/kilo/devstack$ sudo ovs-ofctl -O openflow13 dump-flows br-int | grep "3.3.3.5"
cookie=0x0, duration=711.907s, table=20, n_packets=0, n_bytes=0, priority=1024,arp,tun_id=0x446,arp_tpa=3.3.3.5,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:c9:60:51>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163ec96051->NXM_NX_ARP_SHA[],load:0x3030305->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=695.557s, table=30, n_packets=105, n_bytes=11706, priority=1024,ip,in_port=2,nw_dst=201.1.1.5 actions=set_field:3.3.3.5->ip_dst,load:0x446->NXM_NX_REG3[],goto_table:40
cookie=0x0, duration=711.907s, table=40, n_packets=132, n_bytes=14900, priority=36001,ip,in_port=1,dl_src=fa:16:3e:c9:60:51,nw_src=3.3.3.5 actions=goto_table:50
cookie=0x0, duration=717.477s, table=70, n_packets=110, n_bytes=12728, priority=1024,ip,tun_id=0x446,nw_dst=3.3.3.5 actions=set_field:fa:16:3e:c9:60:51->eth_dst,goto_table:80
cookie=0x0, duration=695.518s, table=100, n_packets=109, n_bytes=12698, priority=512,ip,tun_id=0x446,dl_dst=fa:16:3e:05:a1:e8,nw_src=3.3.3.5 actions=set_field:fa:16:3e:79:94:9d->eth_src,dec_ttl,set_field:00:01:e8:00:ee:ee->eth_dst,set_field:201.1.1.5->ip_src,output:2
==================
openflow@os-controller:~/kilo/devstack$ sudo ovs-ofctl O openflow13 dump-flows br-int | grep "201.1.1.3" cookie=0x0, duration=4078.356s, table=20, n_packets=1, n_bytes=60, priority=1024,arp,in_port=2,arp_tpa=201.1.1.3,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:55:14:9a->eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e55149a->NXM_NX_ARP_SHA[],load:0xc9010103->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=4078.274s, table=30, n_packets=113, n_bytes=13062, priority=1024,ip,in_port=2,nw_dst=201.1.1.3 actions=set_field:3.3.3.4->ip_dst,load:0x446->NXM_NX_REG3[],goto_table:40
cookie=0x0, duration=4078.269s, table=100, n_packets=108, n_bytes=12000, priority=512,ip,tun_id=0x446,dl_dst=fa:16:3e:05:a1:e8,nw_src=3.3.3.4 actions=set_field:fa:16:3e:55:14:9a->eth_src,dec_ttl,set_field:00:01:e8:00:ee:ee->eth_dst,set_field:201.1.1.3->ip_src,output:2
openflow@os-controller:~/kilo/devstack$ sudo ovs-ofctl -O openflow13 dump-flows br-int | grep "3.3.3.5"
cookie=0x0, duration=688.149s, table=20, n_packets=2, n_bytes=84, priority=1024,arp,tun_id=0x446,arp_tpa=3.3.3.5,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:c9:60:51>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163ec96051->NXM_NX_ARP_SHA[],load:0x3030305->NXM_OF_ARP_SPA[],IN_PORT
cookie=0x0, duration=688.180s, table=70, n_packets=5, n_bytes=1022, priority=1024,ip,tun_id=0x446,nw_dst=3.3.3.5 actions=set_field:fa:16:3e:c9:60:51->eth_dst,goto_table:80
|