https://jira.opendaylight.org This file is an XML representation of an issue en-us 8.20.10 820010 22-06-2022 https://jira.opendaylight.org/browse/OVSDB-455 ovsdb <p>During scale testing the OVSDB connection is dropped and eventually leads to hostconfig gets to fail. Current theory is that when the connection is removed, operational is cleared and this impacts the neutron northbound hostconfig queries from networking-odl. networking-odl does a get operational <a href="http://172.16.0.15:8081/restconf/operational/neutron:neutron/hostconfigs" class="external-link" target="_blank" rel="nofollow noopener">restconf/operational/neutron:neutron/hostconfigs</a> but neutron northbound does not return anything. At that point networking-odl marks the ODL l2-agent as down and does not try to use it anymore.</p> OVSDB-455

l2 agent marked down when no response to http://172.16.0.15:8081/restconf/operational/neutron:neutron/hostconfigs

Bug High Open Unresolved Unassigned Sam Hague ds Wed, 7 Mar 2018 15:23:58 +0000 Wed, 1 May 2019 23:54:28 +0000 Oxygen Fluorine Nitrogen-SR4 Fluorine-SR1 Southbound.Open_vSwitch 0 1 <p>When running Browbeat+Rally scenario with netcreate-boot scenario (i.e., we spawn a VM and verify that it becomes ACTIVE) with concurrency set to 5 and times set to 2000, we observed that after spawning some 1422 VMs all subsequent VMs start to fail (i.e, they go into an ERROR state).<br/>  <br/> When we looked at the logs, we observed that after spawning around 1200+ VMs the errors started to creep in. The errors are mostly related to "ODL L2 agent (i.e., neutron-odlagent-portbinding)" becoming inactive and hence port-binding fails.<br/>  <br/> Error reported in Rally: Binding failed for port 000f35ae-9db5-4a9d-9324-255fe7c668d0, please check neutron logs for more information.', u'code': 500, u'created': u'2017-11-30T14:37:43Z'}</p> <p>It looks like networking-odl marks "ODL L2" agent as down when the information is not returned as part of the following query </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> http:<span class="code-comment">//172.16.0.15:8081/restconf/operational/neutron:neutron/hostconfigs</span> </pre> </div></div> <p>After the browbeat test completed, we could see that all the "ODL L2" agents were down/inactive.<br/> On the compute nodes, we see that vswitchd connection to ODL Controller is flaky and this could be the reason why the agent info is not returned as part of hostconfig url.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> &lt;SNIP&gt; 2017-11-30T17:40:12.968Z\|01049\|rconn\|INFO\|br-int&lt;-&gt;tcp:172.16.0.17:6653: connected 2017-11-30T17:40:12.969Z\|01050\|rconn\|INFO\|br-int&lt;-&gt;tcp:172.16.0.17:6653: connection closed by peer 2017-11-30T17:40:12.969Z\|01051\|rconn\|INFO\|br-int&lt;-&gt;tcp:172.16.0.17:6653: continuing to retry connections in the background but suppressing further logging 2017-11-30T17:40:22.778Z\|01052\|rconn\|INFO\|br-int&lt;-&gt;tcp:172.16.0.17:6653: connected 2017-11-30T17:40:22.778Z\|01053\|rconn\|INFO\|br-int&lt;-&gt;tcp:172.16.0.17:6653: connection closed by peer &lt;SNIP&gt; </pre> </div></div> <p>Setup:<br/> OSP12 image<br/> ODL running in a non-cluster, non-container setup with 1 controller and 32 compute nodes.<br/> Port-status is disabled in this setup and inactivity_probe/max_backoff are appropriately configured.</p> <p>RPMs used:<br/> python-networking-odl-11.0.0-2.el7ost.noarch<br/> opendaylight-6.2.0-5.el7ost.noarch</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +--------------------------------------+----------------+------------------------------------+-------------------+-------+----------------+------------------------------+ \| id \| agent_type \| host \| availability_zone \| alive \| admin_state_up \| binary \| +--------------------------------------+----------------+------------------------------------+-------------------+-------+----------------+------------------------------+ \| 0201be96-3c80-451a-971e-dfdeb7ebbdd3 \| ODL L2 \| overcloud-compute-6.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 0283ed5d-b9c8-43c9-9c99-6c27b319b041 \| ODL L2 \| overcloud-compute-15.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 1ad29e22-cbf8-4384-b1e3-c14e3779a4ae \| ODL L2 \| overcloud-compute-18.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 29687ccb-aa71-4150-8059-75cdcc98b284 \| ODL L2 \| overcloud-compute-5.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 29b494c2-bfda-4cdc-b189-1713d66793b0 \| ODL L2 \| overcloud-compute-22.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 2e981caa-6d6e-4a0d-b716-d3f0d79c4968 \| ODL L2 \| overcloud-compute-12.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 3928ceb7-fc75-4309-a38d-2a8cad2b5666 \| ODL L2 \| overcloud-compute-14.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 4011168b-2bda-4bbc-8b3d-0458120b4c13 \| ODL L2 \| overcloud-compute-1.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 4404a197-73a2-41b7-a905-519a06318af0 \| ODL L2 \| overcloud-compute-30.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 4d148674-1e2f-4bcd-986a-505b8bc8f84f \| ODL L2 \| overcloud-compute-19.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 5623cb1c-0dcf-4126-b4f6-f19411e6ec5f \| ODL L2 \| overcloud-compute-26.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 5703d801-49dc-40d7-adda-e7949a6b6771 \| ODL L2 \| overcloud-compute-13.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 62d09361-4222-4a0a-8b91-d03dd324f1e7 \| ODL L2 \| overcloud-controller-0.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 66edd958-936d-48c9-99e3-4d9241315fe7 \| ODL L2 \| overcloud-compute-28.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 682cfb7e-9cd0-4cf8-a1c9-ea8c6450ec37 \| ODL L2 \| overcloud-compute-8.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 72292f55-6a4a-44f2-85a1-ac3640f1b9f0 \| ODL L2 \| overcloud-compute-9.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 75ae43a1-2dd8-4223-ae34-400bbd3b5ac8 \| ODL L2 \| overcloud-compute-11.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 7816271e-9c9c-49a6-9cc4-b56c945fd92b \| ODL L2 \| overcloud-compute-4.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 7b567e37-5ca3-4e3d-91de-31ae56191a2a \| ODL L2 \| overcloud-compute-25.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 7c7f5637-4242-4015-ac1d-ce0932346519 \| ODL L2 \| overcloud-compute-21.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 7f7c54d8-028e-458c-8231-c7774b86e1cf \| ODL L2 \| overcloud-compute-3.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 9270a276-9e0c-42a9-85f5-513bcb9e717a \| ODL L2 \| overcloud-compute-2.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 92b3383d-0fd3-4578-94e9-3e67239b86a9 \| ODL L2 \| overcloud-compute-24.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 94cc6e57-0ffe-4243-ac1d-46f17cee5ed7 \| ODL L2 \| overcloud-compute-31.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 9fc823e4-146b-417d-9c13-078b11d6e7b9 \| ODL L2 \| overcloud-compute-20.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| 9feaf78a-f2da-477d-8053-cc599b496aa8 \| ODL L2 \| overcloud-compute-27.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| a2b758db-06b7-490e-b6d0-d05938848a23 \| ODL L2 \| overcloud-compute-16.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| a68eb489-1ddd-4ab6-bbc9-931aa892282c \| ODL L2 \| overcloud-compute-17.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| ace04e9f-a647-471f-92c9-e3f10e036fd7 \| ODL L2 \| overcloud-compute-10.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| b5fc5492-edf0-4321-bf9f-41f1c541e381 \| ODL L2 \| overcloud-compute-0.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| bef93822-1da4-48b6-b80e-16733f02c858 \| Metadata agent \| overcloud-controller-0.localdomain \| \| :-) \| True \| neutron-metadata-agent \| \| c085c9cd-412c-4a28-8280-d5521de0099c \| ODL L2 \| overcloud-compute-29.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| f2fce8b9-7584-4255-ab7e-02475d82826f \| ODL L2 \| overcloud-compute-23.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| fc35e8f7-8e39-49df-9824-2447367bd59f \| ODL L2 \| overcloud-compute-7.localdomain \| \| xxx \| True \| neutron-odlagent-portbinding \| \| fdf5a3be-d458-461b-8e94-e99ecdf5e435 \| DHCP agent \| overcloud-controller-0.localdomain \| nova \| :-) \| True \| neutron-dhcp-agent \| +--------------------------------------+----------------+------------------------------------+-------------------+-------+----------------+------------------------------+| </pre> </div></div> Duplicate OVSDB-444 Relates NEUTRON-151 Development Rank 0|i03bbb: