[OVSDB-167] Updates to termination point configuration for existing termination points broken Created: 02/Jun/15  Updated: 19/Oct/17  Resolved: 13/Aug/15

Status: Resolved
Project: ovsdb
Component/s: Southbound.Open_vSwitch
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Ed Warnicke Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: File bug3545.out     File bug3545.sh     File bug3545.sh     File bug3545.sh     File createFloat.sh     Text File karaf.log    
External issue ID: 3545
Priority: High

 Description   

Because TerminationPointCreate/UpdateCommand are keying off creation/update in the config store, not the underlying OVSDB... attempt to create configuration for an existing termination point will result in an attempt to 'insert' an existing interface in ovsdb, which will call the entire transact to fail.

The solution to this is to do a single TerminationPointUpdateCommand in a manner similar to how its done for BridgeUpdateCommand.



 Comments   
Comment by Sam Hague [ 02/Jun/15 ]

Reproduce:
1. add port to bridge
2. Add data to port via ovs-vsctl
3. then later add extra config to the port via odl

Comment by Sharad Mishra [ 03/Jun/15 ]

I tried following on stable lithium this morning -

1. Create bretest2 bridge using postman.
2. Add port using - "ovs-vsctl add-port brtest2 vlan10 tag=10 – set Interface vlan10 type=internal"
3. operational shows following (cut to only the relevant section) -

"termination-point": [

{ "ovsdb:interface-type": "ovsdb:interface-type-internal", "ovsdb:interface-uuid": "c1666aa1-bd40-47ee-9f65-994b838d430f", "ovsdb:name": "vlan10", "ovsdb:ofport": 1, "ovsdb:port-uuid": "fa38cb93-c2f6-4850-8ce4-2a86bd108e6d", "ovsdb:vlan-tag": 10, "tp-id": "vlan10" }

]

4. Using postman, added trunks and vlan-mode to this termination-point. Operational changed to -

"ovsdb:interface-type": "ovsdb:interface-type-internal",
"ovsdb:interface-uuid": "c1666aa1-bd40-47ee-9f65-994b838d430f",
"ovsdb:name": "vlan10",
"ovsdb:ofport": 1,
"ovsdb:options": [

{ "option": "remote_ip", "value": "134.134.170.72" }

],
"ovsdb:port-external-ids": [

{ "external-id-key": "opendaylight-iid", "external-id-value": "/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://134.134.170.72:6640/bridge/brtest2']/network-topology:termination-point[network-topology:tp-id='vlan10']" }

],
"ovsdb:port-uuid": "fa38cb93-c2f6-4850-8ce4-2a86bd108e6d",
"ovsdb:trunks": [

{ "trunk": 2 }

,

{ "trunk": 3 }

],
"ovsdb:vlan-mode": "access",
"ovsdb:vlan-tag": 10,
"tp-id": "vlan10"
}

5. sudo ovs-vsctl list Port
_
_uuid : fa38cb93-c2f6-4850-8ce4-2a86bd108e6d
bond_active_slave : []
bond_downdelay : 0
bond_fake_iface : false
bond_mode : []
bond_updelay : 0
external_ids :

{opendaylight-iid="/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://134.134.170.72:6640/bridge/brtest2']/network-topology:termination-point[network-topology:tp-id='vlan10']"}

fake_bridge : false
interfaces : [c1666aa1-bd40-47ee-9f65-994b838d430f]
lacp : []
mac : []
name : "vlan10"
other_config : {}
qos : []
statistics : {}
status : {}
tag : 10
trunks : [2, 3]
vlan_mode : access

Comment by Sharad Mishra [ 03/Jun/15 ]

Postman collection - https://www.getpostman.com/collections/330c161d1f46d6273e17

Comment by Sharad Mishra [ 03/Jun/15 ]

I also tried

1. Create bridge using postman.
2. Add port using postman.
3. Add interface type using ovs-vsctl ( sudo ovs-vsctl – set Interface vlan10 type=internal)
4. $ sudo ovs-vsctl show
b31bca50-69da-4afe-8b14-5f8064cdf316
Manager "ptcp:6640"
is_connected: true
Bridge "brtest2"
Controller "tcp:134.134.170.72:6640"
Port "vlan10"
tag: 10
Interface "vlan10"
type: internal
Port "brtest2"
Interface "brtest2"
type: internal
ovs_version: "2.3.1-git3282e51"

4. Update TP with "options" and "trunk" using postman.

5. $ sudo ovs-vsctl show
b31bca50-69da-4afe-8b14-5f8064cdf316
Manager "ptcp:6640"
is_connected: true
Bridge "brtest2"
Controller "tcp:134.134.170.72:6640"
Port "vlan10"
tag: 10
trunks: [2, 3]
Interface "vlan10"
type: internal
options:

{remote_ip="134.134.170.72"}

Port "brtest2"
Interface "brtest2"
type: internal
ovs_version: "2.3.1-git3282e51"

Comment by Flavio Fernandes [ 24/Jun/15 ]

this is just a reference script. Refer to bug3545.sh for a handy
and easy script to reproduce this bug.

Comment by Flavio Fernandes [ 24/Jun/15 ]

Attachment createFloat.sh has been added with description: original script where bug3545.sh came from

Comment by Flavio Fernandes [ 24/Jun/15 ]
  1. steps to reproduce OVSDB-167
  1. 1) start ODL, and do: feature:install odl-ovsdb-openstack
  2. Make sure to wait for ODL to be fully up
  3. 2) copy this script to a system that ir running OVS. Make sure OVS is completely
  4. cleared. If using ubuntu, that involves something like:
    #
  5. #!/usr/bin/env bash
  6. sudo service openvswitch-switch stop
  7. sudo rm -rf /var/log/openvswitch/*
  8. sudo rm -rf /etc/openvswitch/conf.db
  9. sudo service openvswitch-switch start
    #
  10. 3) make sure you have curl installed in the system where this script is ran
    #
  11. 4) do "sudo ovs-vsctl show" and make sure there are no bridges and OVS is not
  12. connected to ODL yet. An example of what it should look like is below:
    #
  13. $ sudo ovs-vsctl show
  14. 5c012e23-b5a1-45d4-9a2e-865a3f81c59e
  15. ovs_version: "2.3.0"
  16. $
    #
  17. 5) change ODL_IP below to the ip address of where ODL is running
    #
  18. 6) run this script. It will:
    #
  19. - explicitly create bridge br-ex;
  20. - connect to ODL
  21. - create neutron network
  22. - add an ovs port
    #
  23. 7) if bug is reproduced, you will notive that while there is a patch-port in
  24. br-int, a patch port in br-ex is not created. Doing this will work around the bug
    #
  25. sudo ovs-vsctl add-port br-ex patch-int – set interface patch-int type=patch – set interface patch-int options:peer=patch-ext

===

vagrant@devstack-control:~$
vagrant@devstack-control:~$ sudo ovs-vsctl show
75234ef6-ef80-4899-8fda-2b26d1f7c199
Manager "tcp:192.168.50.1:6640"
is_connected: true
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Bridge br-int
Controller "tcp:192.168.50.1:6653"
is_connected: true
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port "tap79adcba5-19"
Interface "tap79adcba5-19"
type: internal
Port patch-ext
Interface patch-ext
type: patch
options:

{peer=patch-int}
ovs_version: "2.3.0"
vagrant@devstack-control:~$
vagrant@devstack-control:~$
vagrant@devstack-control:~$ sudo ovs-vsctl add-port br-ex patch-int – set interface patch-int type=patch – set interface patch-int options:peer=patch-ext
vagrant@devstack-control:~$ sudo ovs-vsctl show
75234ef6-ef80-4899-8fda-2b26d1f7c199
Manager "tcp:192.168.50.1:6640"
is_connected: true
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-ext}
Bridge br-int
Controller "tcp:192.168.50.1:6653"
is_connected: true
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port "tap79adcba5-19"
Interface "tap79adcba5-19"
type: internal
Port patch-ext
Interface patch-ext
type: patch
options: {peer=patch-int}

ovs_version: "2.3.0"
vagrant@devstack-control:~$

Comment by Flavio Fernandes [ 24/Jun/15 ]

Attachment bug3545.sh has been added with description: easy script for reproducing this bug

Comment by Sharad Mishra [ 02/Jul/15 ]

Attachment bug3545.out has been added with description: Output of shell script to create network and test this defect.

Comment by Sam Hague [ 07/Jul/15 ]

rc='ovs-vsctl: no bridge named br-int

Can you also attach the logs from ODL? Looks like the node never connected properly to ODL and netvirt did not create br-int.

This line should have enabled the connection: sudo ovs-vsctl set-manager tcp:localhost:6640

Comment by Flavio Fernandes [ 13/Jul/15 ]

I missed a step that would enable l3 in ovsdb net-virt

w/out that, no patch ports will be created (in any bridge).

Will re-submit updated patch and instructions.

https://github.com/opendaylight/ovsdb/blob/master/openstack/net-virt/src/main/java/org/opendaylight/ovsdb/openstack/netvirt/impl/BridgeConfigurationManagerImpl.java#L369

Comment by Flavio Fernandes [ 13/Jul/15 ]

Missed step 0: enable l3 forwarding in ovsdb netvirt:

echo 'ovsdb.l3.fwd.enabled=yes' >> ./etc/custom.properties

(In reply to Flavio Fernandes from comment #6)
> Created attachment 549 [details]
> easy script for reproducing this bug
>
> # steps to reproduce OVSDB-167
>
> # 1) start ODL, and do: feature:install odl-ovsdb-openstack
> # Make sure to wait for ODL to be fully up
> #

Comment by Flavio Fernandes [ 13/Jul/15 ]
  1. steps to reproduce OVSDB-167
  1. 0) enable L3 forwarding in ODL, so patch ports will be created upon neutron network creation:
  2. echo 'ovsdb.l3.fwd.enabled=yes' >> ./etc/custom.properties
    #
  3. 1) start ODL, and do: feature:install odl-ovsdb-openstack
  4. Make sure to wait for ODL to be fully up
    #
  5. 2) copy this script to a system that ir running OVS. Make sure OVS is completely
  6. cleared. If using ubuntu, that involves something like:
    #
  7. #!/usr/bin/env bash
  8. sudo service openvswitch-switch stop
  9. sudo rm -rf /var/log/openvswitch/*
  10. sudo rm -rf /etc/openvswitch/conf.db
  11. sudo service openvswitch-switch start
    #
  12. 3) make sure you have curl installed in the system where this script is ran
    #
  13. 4) do "sudo ovs-vsctl show" and make sure there are no bridges and OVS is not
  14. connected to ODL yet. An example of what it should look like is below:
    #
  15. $ sudo ovs-vsctl show
  16. 5c012e23-b5a1-45d4-9a2e-865a3f81c59e
  17. ovs_version: "2.3.0"
  18. $
    #
  19. 5) change ODL_IP below to the ip address of where ODL is running
    #
  20. 6) run this script. It will:
    #
  21. - explicitly create bridge br-ex;
  22. - connect to ODL
  23. - create neutron network
  24. - add an ovs port
    #
  25. 7) if bug is reproduced, you will notive that while there is a patch-port in
  26. br-int, a patch port in br-ex is not created. Doing this will work around the bug
    #
  27. sudo ovs-vsctl add-port br-ex patch-int – set interface patch-int type=patch – set interface patch-int options:peer=patch-ext
Comment by Flavio Fernandes [ 13/Jul/15 ]

Attachment bug3545.sh has been added with description: bug3545.sh

Comment by Flavio Fernandes [ 13/Jul/15 ]

I re-run the script and not I hit the same issue Sharad see:

br-int is not getting created at all in the node. I ended up with 2 br-int in mdsal's config:

{
"network-topology": {
"topology": [
{
"topology-id": "ovsdb:1",
"node": [
{
"node-id": "ovsdb://uuid/ddfb2be2-eafe-477d-be33-12e87f35bdb5/bridge/br-int",
"ovsdb:bridge-name": "br-int",
"ovsdb:managed-by": "/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/ddfb2be2-eafe-477d-be33-12e87f35bdb5']",
"ovsdb:controller-entry": [

{ "target": "tcp:192.168.50.1:6653" }

],
"ovsdb:protocol-entry": [

{ "protocol": "ovsdb:ovsdb-bridge-protocol-openflow-13" }

],
"ovsdb:fail-mode": "ovsdb:ovsdb-fail-mode-secure"
},
{
"node-id": "ovsdb://uuid/273cc9c1-1800-4af0-9c7c-bceb0454628c/bridge/br-int",
"ovsdb:bridge-name": "br-int",
"ovsdb:managed-by": "/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/273cc9c1-1800-4af0-9c7c-bceb0454628c']",
"ovsdb:controller-entry": [

{ "target": "tcp:192.168.50.1:6653" }

],
"ovsdb:protocol-entry": [

{ "protocol": "ovsdb:ovsdb-bridge-protocol-openflow-13" }

],
"ovsdb:fail-mode": "ovsdb:ovsdb-fail-mode-secure"
}
]
}
]
}
}

And I see br-ex (and nothing else) on the operational tree:

{
"network-topology": {
"topology": [

{ "topology-id": "netvirt:1" }

,

{ "topology-id": "flow:1" }

,
{
"topology-id": "ovsdb:1",
"node": [
{
"node-id": "ovsdb://uuid/ddfb2be2-eafe-477d-be33-12e87f35bdb5/bridge/br-ex",
"ovsdb:bridge-name": "br-ex",
"ovsdb:bridge-uuid": "4110352b-f9a7-4edc-a473-faf7e450b1ee",
"ovsdb:datapath-id": "00:00:2a:35:10:41:dc:4e",
"ovsdb:datapath-type": "ovsdb:datapath-type-system",
"ovsdb:managed-by": "/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/ddfb2be2-eafe-477d-be33-12e87f35bdb5']",
"ovsdb:bridge-external-ids": [

{ "bridge-external-id-key": "bridge-id", "bridge-external-id-value": "br-ex" }

],
"termination-point": [

{ "tp-id": "br-ex", "ovsdb:interface-type": "ovsdb:interface-type-internal", "ovsdb:port-uuid": "6d845492-99d1-4716-acde-0e32ad3ac278", "ovsdb:name": "br-ex", "ovsdb:interface-uuid": "c55dfd94-21a3-4163-b06c-afe7634907e5", "ovsdb:ofport": 65534 }

]
},
{
"node-id": "ovsdb://uuid/ddfb2be2-eafe-477d-be33-12e87f35bdb5",
"ovsdb:managed-node-entry": [

{ "bridge-ref": "/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://uuid/ddfb2be2-eafe-477d-be33-12e87f35bdb5/bridge/br-ex']" }

],
"ovsdb:connection-info":

{ "local-ip": "192.168.50.1", "remote-ip": "192.168.50.20", "remote-port": 47688, "local-port": 6640 }

,
"ovsdb:ovs-version": "2.3.2",
"ovsdb:openvswitch-external-ids": [

{ "external-id-key": "system-id", "external-id-value": "0b0f991e-5420-466f-aafd-492cdb2b130e" }

]
}
]
}
]
}
}

Comment by Flavio Fernandes [ 13/Jul/15 ]

this is karaf.log when br-int created twice in config tree, and never gets
in the operational tree.

Comment by Flavio Fernandes [ 13/Jul/15 ]

Attachment karaf.log has been added with description: karaf.log when br-int is not getting created

Comment by Flavio Fernandes [ 13/Jul/15 ]

(In reply to Flavio Fernandes from comment #11)
> Created attachment 567 [details]
> bug3545.sh
>
>
> # steps to reproduce OVSDB-167
>

revising step 0 to remove any previous state in ODL:

  1. 0.0) clean up ODL's previous state. But deleting data AND journal directories: rm -rf ./data ./journal
  2. 0.5) enable L3 forwarding in ODL, so patch ports will be created upon neutron network creation:
  3. echo 'ovsdb.l3.fwd.enabled=yes' >> ./etc/custom.properties
    #
  4. 1) start ODL, and do: feature:install odl-ovsdb-openstack
  5. Make sure to wait for ODL to be fully up
    #
    ...
Comment by Flavio Fernandes [ 13/Jul/15 ]
  • added bigger sleep between manager setup and creation of neutron net;
  • make sure no state from previous ODL run is carried over:
    rm -rf ./data ./journal

  1. steps to reproduce OVSDB-167
  1. 0.0) clean up ODL's previous state. But deleting data AND journal directories: rm -rf ./data ./journal
  2. 0.5) enable L3 forwarding in ODL, so patch ports will be created upon neutron network creation:
  3. echo 'ovsdb.l3.fwd.enabled=yes' >> ./etc/custom.properties
    #
  4. 1) start ODL, and do: feature:install odl-ovsdb-openstack
  5. Make sure to wait for ODL to be fully up
    #
  6. 2) copy this script to a system that ir running OVS. Make sure OVS is completely
  7. cleared. If using ubuntu, that involves something like:
    #
  8. #!/usr/bin/env bash
  9. sudo service openvswitch-switch stop
  10. sudo rm -rf /var/log/openvswitch/*
  11. sudo rm -rf /etc/openvswitch/conf.db
  12. sudo service openvswitch-switch start
    #
  13. 3) make sure you have curl installed in the system where this script is ran
    #
  14. 4) do "sudo ovs-vsctl show" and make sure there are no bridges and OVS is not
  15. connected to ODL yet. An example of what it should look like is below:
    #
  16. $ sudo ovs-vsctl show
  17. 5c012e23-b5a1-45d4-9a2e-865a3f81c59e
  18. ovs_version: "2.3.0"
  19. $
    #
  20. 5) change ODL_IP below to the ip address of where ODL is running
    #
  21. 6) run this script. It will:
    #
  22. - explicitly create bridge br-ex;
  23. - connect to ODL
  24. - sleep, so odl is given time to create br-int
  25. - create neutron network
  26. - add an ovs port
    #
  27. 7) if bug is reproduced, you will notive that while there is a patch-port in
  28. br-int, a patch port in br-ex is not created. Doing this will work around the bug
    #
  29. sudo ovs-vsctl add-port br-ex patch-int – set interface patch-int type=patch – set interface patch-int options:peer=patch-ext
Comment by Flavio Fernandes [ 13/Jul/15 ]

Attachment bug3545.sh has been added with description: bug3545.sh

Comment by Sharad Mishra [ 22/Jul/15 ]

draft patch uploaded at https://git.opendaylight.org/gerrit/#/c/24364/

Comment by Flavio Fernandes [ 12/Aug/15 ]

gerrit on stable/lithium: https://git.opendaylight.org/gerrit/#/c/25204/

Comment by Anil Vishnoi [ 13/Aug/15 ]

master : https://git.opendaylight.org/gerrit/#/c/24364/3

Generated at Wed Feb 07 20:35:41 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.