Operating System: All
Platform: All

Attachment karaf.log.xz has been added with description: karaflog_0

Attachment karaf.log.1.xz has been added with description: karaflog_1

Attachment karaf.log.2.xz has been added with description: karaflog_2

Attachment karaf.log.3.xz has been added with description: karaflog_3

The cause of this bug is that the information of deleted L2GW Node still left in ODL.

We found that the creation time can be reduced by deleting ODL data when this issue occurred.
The information left in ODL and the results of the creation time reduction by deleting it are below:
1. MD-SAL network-topology(CONFIGURATION)
Deleting this data will not reduce the creation time
2. MD-SAL itm(transport-zones)
Deleting this data will reduce the creation time
3. Memory cache(L2GWCONN, HA)
This data cannot be deleted from external side.

Hi Ran,
Can you give the exact steps that are followed.
like the restcalls that are used and the southbound config that is done.
Thanks,
Suneelu

Hi Suneelu

This issue also occurred in non-HA Cluster environment (single L2GW Node environment).

The following are the detailed commands we used to collect data and REST API used for deletion.

• Command used for data collection
  1. MD-SAL network-topology(CONFIGURATION/OPERATIONAL)
  curl -s -u admin:admin http://127.0.0.1:8080/restconf/config/network-topology:network-topology/ | python -mjson.tool > /root/vtep/$1.config
  curl -s -u admin:admin http://127.0.0.1:8080/restconf/operational/network-topology:network-topology/ | python -mjson.tool > /root/vtep/$1.operational
  2. MD-SAL itm(transport-zones)
  curl -s -u admin:admin http://127.0.0.1:8080/restconf/config/itm:transport-zones | python -mjson.tool > /root/vtep/$1.itm.transport-zones
  3. Memory cache(L2GWCONN, HA)
  for cache in L2GWCONN L2GW HA
  do
  echo "#### ${cache} ####"
  sshpass -p karaf ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p 8101 -l karaf ${LOCALHOST} l2gw:show-cache -cache ${cache}
  done >> /root/vtep/$1.cache

• RESTAPI
  1. MD-SAL network-topology(CONFIGURATION)
  curl -s -u admin:admin -X DELETE http://127.0.0.1:8181/restconf/config/network-topology:network-topology/topology/hwvtep%3A1/node/hwvtep%3A%2F%2Fuuid%2F${uuid} | python -mjson.tool

• For uuid, specify it as the uuid of node-id which obtained from hwvtep: 1
  2. MD-SAL itm(transport-zones)
  curl -s -u admin:admin -X DELETE http://127.0.0.1:8181/restconf/config/itm:transport-zones/transport-zone/${uuid}
• For uuid, specify it as the uuid obtained from the zone-name of * .itm.transport-zones

The contents of attached data file is listed as below:

• Attached data file list
  100.* Data of L2GW Node registration / deletion repeated near 100 times
  del1.* Data after deleted "MD-SAL network-topology (CONFIGURATION)" near 107 times
  150.* Data of L2GW Node registration / deletion repeated near 150 times
  del2.* Data after deleted "MD-SAL network-topology (CONFIGURATION)" near 151 times

The detailed reproduction steps are below:

• Detailed reproduction steps
  1. L2GW Node Initialization
  Delete L2GW GATEWAY / CONNECTION
  neutron l2-gateway-connection-delete `neutron l2-gateway-connection-list --segmentation_id=2222 -c id -f value`
  neutron l2-gateway-delete gw1

Stop OVS process and initialization:
pkill ovsdb-server
pkill ovs-vswitchd
ps -ef|grep ovs-vtep | grep -v grep | awk '

' | sh
ps -ef | grep ovs | grep -v grep
ps -ef | grep -v '[' | grep -v grep
rm -f /etc/openvswitch/ovs.db /etc/openvswitch/vtep.db
ovsdb-tool create /etc/openvswitch/ovs.db /usr/share/openvswitch/vswitch.ovsschema
ovsdb-tool create /etc/openvswitch/vtep.db /usr/share/openvswitch/vtep.ovsschema
ovs-dpctl del-flows system@ovs-system
rm -f /var/log/openvswitch/*

2. Network / subnet deletion
neutron net-delete test-nw

3. Network / subnet creation
neutron net-create test-nw
neutron subnet-create test-nw 192.168.0.0/24 --enable-dhcp

4. L2GW Node setting up
Create L2GW / LACP environment:
ulimit -n 65535
vtep=10.0.0.50
BRIDGE=ocata-l2gw1
ovsdb-server --pidfile --detach --log-file \
--remote=punix:/var/run/openvswitch/db.sock \
--remote=db:hardware_vtep,Global,managers \
--remote=ptcp:6632 /etc/openvswitch/ovs.db /etc/openvswitch/vtep.db
ovs-vswitchd --log-file --detach --pidfile \
unix:/var/run/openvswitch/db.sock
sleep 2
ovs-vsctl add-br $BRIDGE
ovs-vsctl add-port $BRIDGE eth2
vtep-ctl add-ps $BRIDGE
vtep-ctl set Physical_Switch $BRIDGE tunnel_ips=$vtep
sleep 2
/usr/share/openvswitch/scripts/ovs-vtep --log-file=/var/log/openvswitch/ovs-vtep.log \
--pidfile=/var/run/openvswitch/ovs-vtep.pid \
--detach $BRIDGE
vtep-ctl set-manager tcp:172.16.1.30:6640

Create L2GW GATEWAY / CONNECTION:
neutron l2-gateway-create gw1 --device name=ocata-l2gw1,interface_names=eth2
neutron l2-gateway-connection-create gw1 test-nw --default-segmentation-id 2222

5. Wait for creation completion
ssh root@172.16.1.50 'ovs-vsctl show'
ssh root@172.16.1.50 'ovsdb-client dump hardware_vtep'
*wait untill the tunnel I/F (vx1) in Logical Switch is created

6. Ping communication confirmation
ip netns exec qdhcp-`neutron net-show test-nw -c id -f value` ping -c 5 192.168.0.200
sleep 20

Attachment data.tar.gz has been added with description: Data file

Old bug, will reopen if seen again