https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/NETVIRT-770 netvirt <p>Issue：<br/> We repeated l2gw node addition/deletion to Open vSwitch HWVTEP Emulator a few times.<br/> The first 10 times, Logical Switch and tunnel I/F will be created in about 6 seconds after l2gw connection creation.<br/> After 100 times, it will takes more than 300 seconds to finish Logical Switch and tunnel I/F creation. </p> <p>Environment details:<br/> OpenStack Version：stable/ocata<br/> ODL Version：Carbon-FR + patch<br/> patch: <a href="https://git.opendaylight.org/gerrit/#/c/56773/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/56773/</a><br/> <a href="https://git.opendaylight.org/gerrit/#/c/56710/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/56710/</a><br/> <a href="https://git.opendaylight.org/gerrit/#/c/58787/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/58787/</a><br/> <a href="https://git.opendaylight.org/gerrit/#/c/59598/" class="external-link" target="_blank" rel="nofollow noopener">https://git.opendaylight.org/gerrit/#/c/59598/</a><br/> HWVTEP: Open vSwitch 2.6.1 HWVTEP Emulator<br/> : HA Cluster</p> <p>What we did (Steps):<br/> 1. L2GW Node Initialization:<br/> a. delete L2GW GATEWAY/CONNECTION <br/> b. stop OVS process and initialize<br/> 2. Network/subnet deletion:<br/> neutron net-delete test-nw<br/> 3. Network/subnet creation:<br/> neutron net-create test-nw<br/> neutron subnet-create test-nw 192.168.0.0/24 --enable-dhcp<br/> 4. L2GW Node configuration<br/> L2GW/LACP environment configuration<br/> L2GW GATEWAY/CONNECTION creation<br/> 5. Wait until creation is completed<br/> Wait until the VXLAN I/F is created in the OVS of the L2GW Node with a 3 seconds × 100 times loop <br/> 6. Communication confirmation (ping)<br/> 7. Go back to setp 1</p> <p>Please check the log files attached.</p> <p>Operating System: All<br/> Platform: All</p> NETVIRT-770

The time takes in Logical Switch creation and tunnel I/F creation will increase dramatically when repeating l2gw node addition/deletion.

Bug Medium Resolved Won't Do Unassigned Ran Xiao Fri, 7 Jul 2017 12:01:02 +0000 Fri, 8 Nov 2019 11:03:07 +0000 Fri, 8 Nov 2019 11:03:07 +0000 Carbon General 0 13 <p>Attachment karaf.log.xz has been added with description: karaflog_0</p> <p>Attachment karaf.log.1.xz has been added with description: karaflog_1</p> <p>Attachment karaf.log.2.xz has been added with description: karaflog_2</p> <p>Attachment karaf.log.3.xz has been added with description: karaflog_3</p> <p>The cause of this bug is that the information of deleted L2GW Node still left in ODL.</p> <p>We found that the creation time can be reduced by deleting ODL data when this issue occurred.<br/> The information left in ODL and the results of the creation time reduction by deleting it are below:<br/> 1. MD-SAL network-topology(CONFIGURATION)<br/> Deleting this data will not reduce the creation time<br/> 2. MD-SAL itm(transport-zones)<br/> Deleting this data will reduce the creation time<br/> 3. Memory cache(L2GWCONN, HA)<br/> This data cannot be deleted from external side.</p> <p>Hi Ran,<br/> Can you give the exact steps that are followed.<br/> like the restcalls that are used and the southbound config that is done.<br/> Thanks,<br/> Suneelu</p> <p>Hi Suneelu</p> <p>This issue also occurred in non-HA Cluster environment (single L2GW Node environment).</p> <p>The following are the detailed commands we used to collect data and REST API used for deletion.</p> <ul class="alternate" type="square"> <li>Command used for data collection<br/> 1. MD-SAL network-topology(CONFIGURATION/OPERATIONAL)<br/> curl -s -u admin:admin <a href="http://127.0.0.1:8080/restconf/config/network-topology:network-topology/" class="external-link" target="_blank" rel="nofollow noopener">http://127.0.0.1:8080/restconf/config/network-topology:network-topology/</a> | python -mjson.tool &gt; /root/vtep/$1.config<br/> curl -s -u admin:admin <a href="http://127.0.0.1:8080/restconf/operational/network-topology:network-topology/" class="external-link" target="_blank" rel="nofollow noopener">http://127.0.0.1:8080/restconf/operational/network-topology:network-topology/</a> | python -mjson.tool &gt; /root/vtep/$1.operational<br/> 2. MD-SAL itm(transport-zones)<br/> curl -s -u admin:admin <a href="http://127.0.0.1:8080/restconf/config/itm:transport-zones" class="external-link" target="_blank" rel="nofollow noopener">http://127.0.0.1:8080/restconf/config/itm:transport-zones</a> | python -mjson.tool &gt; /root/vtep/$1.itm.transport-zones<br/> 3. Memory cache(L2GWCONN, HA)<br/> for cache in L2GWCONN L2GW HA<br/> do<br/> echo "#### ${cache} ####"<br/> sshpass -p karaf ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -p 8101 -l karaf ${LOCALHOST} l2gw:show-cache -cache ${cache}<br/> done &gt;&gt; /root/vtep/$1.cache</li> </ul> <ul class="alternate" type="square"> <li>RESTAPI<br/> 1. MD-SAL network-topology(CONFIGURATION)<br/> curl -s -u admin:admin -X DELETE <a href="http://127.0.0.1:8181/restconf/config/network-topology:network-topology/topology/hwvtep%3A1/node/hwvtep%3A%2F%2Fuuid%2F$" class="external-link" target="_blank" rel="nofollow noopener">http://127.0.0.1:8181/restconf/config/network-topology:network-topology/topology/hwvtep%3A1/node/hwvtep%3A%2F%2Fuuid%2F$</a>{uuid} | python -mjson.tool</li> </ul> <ul> <li>For uuid, specify it as the uuid of node-id which obtained from hwvtep: 1<br/> 2. MD-SAL itm(transport-zones)<br/> curl -s -u admin:admin -X DELETE <a href="http://127.0.0.1:8181/restconf/config/itm:transport-zones/transport-zone/$" class="external-link" target="_blank" rel="nofollow noopener">http://127.0.0.1:8181/restconf/config/itm:transport-zones/transport-zone/$</a>{uuid}</li> <li>For uuid, specify it as the uuid obtained from the zone-name of * .itm.transport-zones</li> </ul> <p>The contents of attached data file is listed as below:</p> <ul class="alternate" type="square"> <li>Attached data file list<br/> 100.* Data of L2GW Node registration / deletion repeated near 100 times<br/> del1.* Data after deleted "MD-SAL network-topology (CONFIGURATION)" near 107 times<br/> 150.* Data of L2GW Node registration / deletion repeated near 150 times<br/> del2.* Data after deleted "MD-SAL network-topology (CONFIGURATION)" near 151 times</li> </ul> <p>The detailed reproduction steps are below:</p> <ul class="alternate" type="square"> <li>Detailed reproduction steps<br/> 1. L2GW Node Initialization<br/> Delete L2GW GATEWAY / CONNECTION<br/> neutron l2-gateway-connection-delete `neutron l2-gateway-connection-list --segmentation_id=2222 -c id -f value`<br/> neutron l2-gateway-delete gw1</li> </ul> <p> Stop OVS process and initialization:<br/> pkill ovsdb-server<br/> pkill ovs-vswitchd<br/> ps -ef|grep ovs-vtep | grep -v grep | awk '</p> {print "kill "$2} <p>' | sh<br/> ps -ef | grep ovs | grep -v grep<br/> ps -ef | grep -v '[' | grep -v grep<br/> rm -f /etc/openvswitch/ovs.db /etc/openvswitch/vtep.db<br/> ovsdb-tool create /etc/openvswitch/ovs.db /usr/share/openvswitch/vswitch.ovsschema<br/> ovsdb-tool create /etc/openvswitch/vtep.db /usr/share/openvswitch/vtep.ovsschema<br/> ovs-dpctl del-flows system@ovs-system<br/> rm -f /var/log/openvswitch/*</p> <p>2. Network / subnet deletion<br/> neutron net-delete test-nw</p> <p>3. Network / subnet creation<br/> neutron net-create test-nw<br/> neutron subnet-create test-nw 192.168.0.0/24 --enable-dhcp</p> <p>4. L2GW Node setting up <br/> Create L2GW / LACP environment:<br/> ulimit -n 65535<br/> vtep=10.0.0.50<br/> BRIDGE=ocata-l2gw1<br/> ovsdb-server --pidfile --detach --log-file \<br/> --remote=punix:/var/run/openvswitch/db.sock \<br/> --remote=db:hardware_vtep,Global,managers \<br/> --remote=ptcp:6632 /etc/openvswitch/ovs.db /etc/openvswitch/vtep.db<br/> ovs-vswitchd --log-file --detach --pidfile \<br/> unix:/var/run/openvswitch/db.sock<br/> sleep 2<br/> ovs-vsctl add-br $BRIDGE<br/> ovs-vsctl add-port $BRIDGE eth2<br/> vtep-ctl add-ps $BRIDGE<br/> vtep-ctl set Physical_Switch $BRIDGE tunnel_ips=$vtep<br/> sleep 2<br/> /usr/share/openvswitch/scripts/ovs-vtep --log-file=/var/log/openvswitch/ovs-vtep.log \<br/> --pidfile=/var/run/openvswitch/ovs-vtep.pid \<br/> --detach $BRIDGE<br/> vtep-ctl set-manager tcp:172.16.1.30:6640</p> <p> Create L2GW GATEWAY / CONNECTION:<br/> neutron l2-gateway-create gw1 --device name=ocata-l2gw1,interface_names=eth2<br/> neutron l2-gateway-connection-create gw1 test-nw --default-segmentation-id 2222</p> <p>5. Wait for creation completion<br/> ssh root@172.16.1.50 'ovs-vsctl show'<br/> ssh root@172.16.1.50 'ovsdb-client dump hardware_vtep' <br/> *wait untill the tunnel I/F (vx1) in Logical Switch is created</p> <p>6. Ping communication confirmation<br/> ip netns exec qdhcp-`neutron net-show test-nw -c id -f value` ping -c 5 192.168.0.200<br/> sleep 20</p> <p>Attachment data.tar.gz has been added with description: Data file</p> <p>Old bug, will reopen if seen again</p> Development External issue ID 8819 External issue URL Rank 0|i01tun: