[OVSDB-162] NPE at org.opendaylight.ovsdb.openstack.netvirt.impl.SouthboundImpl.extractTerminationPointAugmentation(SouthboundImpl.java:387)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT] Created: 28/May/15  Updated: 19/Oct/17  Resolved: 01/Jun/15

Status: Resolved
Project: ovsdb
Component/s: Southbound.Open_vSwitch
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Flavio Fernandes Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: Text File karaf.log    
External issue ID: 3404

 Description   

while creating tenant VMs, I hit this NPE:

2015-05-27 23:24:18,092 | INFO | ntDispatcherImpl | OF13Provider | 278 - org.opendaylight.ovsdb.openstack.net-virt-providers - 1.1.0.SNAPSHOT | addTunnelPort enter: portName: vxlan-192.168.50.22
2015-05-27 23:24:18,092 | ERROR | ntDispatcherImpl | EventDispatcher | 277 - org.opendaylight.ovsdb.openstack.net-virt - 1.1.0.SNAPSHOT | Exception in dispatching event SouthboundEvent [type=PORT, action=UPDATE, augmentationData=OvsdbTerminationPointAugmentation{getInterfaceExternalIds=[InterfaceExternalIds{getExternalIdKey=iface-status, getExternalIdValue=active, augmentations={}}, InterfaceExternalIds{getExternalIdKey=attached-mac, getExternalIdValue=fa:16:3e:20:9d:8a, augmentations={}}, InterfaceExternalIds{getExternalIdKey=vm-id, getExternalIdValue=b3c31b99-271f-4923-ade0-ffd3232e093c, augmentations={}}, InterfaceExternalIds{getExternalIdKey=iface-id, getExternalIdValue=74f06b06-c969-4418-871c-0b352359feea, augmentations={}}], getInterfaceUuid=Uuid [_value=c437ae58-07d7-4b8f-adcb-6999b38379ee], getName=tap74f06b06-c9, getOfport=4, getPortUuid=Uuid [_value=22005364-ca1f-459c-9fef-f78ece4ca7a4]}, node=Node{getNodeId=Uri [_value=ovsdb://192.168.50.22:52848/bridge/br-int], getTerminationPoint=[TerminationPoint{getTpId=Uri [_value=tap74f06b06-c9], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation{getInterfaceExternalIds=[InterfaceExternalIds{getExternalIdKey=iface-status, getExternalIdValue=active, augmentations={}}, InterfaceExternalIds{getExternalIdKey=attached-mac, getExternalIdValue=fa:16:3e:20:9d:8a, augmentations={}}, InterfaceExternalIds{getExternalIdKey=vm-id, getExternalIdValue=b3c31b99-271f-4923-ade0-ffd3232e093c, augmentations={}}, InterfaceExternalIds{getExternalIdKey=iface-id, getExternalIdValue=74f06b06-c969-4418-871c-0b352359feea, augmentations={}}], getInterfaceUuid=Uuid [_value=c437ae58-07d7-4b8f-adcb-6999b38379ee], getName=tap74f06b06-c9, getOfport=4, getPortUuid=Uuid [_value=22005364-ca1f-459c-9fef-f78ece4ca7a4]}}}, TerminationPoint{getTpId=Uri [_value=tap0b238b22-a0], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation{getInterfaceExternalIds=[InterfaceExternalIds{getExternalIdKey=iface-status, getExternalIdValue=active, augmentations={}}, InterfaceExternalIds{getExternalIdKey=attached-mac, getExternalIdValue=fa:16:3e:2b:70:72, augmentations={}}, InterfaceExternalIds{getExternalIdKey=vm-id, getExternalIdValue=2bba76d1-2847-4bd8-9592-bfdb21b8a192, augmentations={}}, InterfaceExternalIds{getExternalIdKey=iface-id, getExternalIdValue=0b238b22-a018-44fa-9aab-489a9661f9d5, augmentations={}}], getInterfaceUuid=Uuid [_value=4dc2e98b-d914-4a27-ac04-716f5a6d0385], getName=tap0b238b22-a0, getOfport=2, getPortUuid=Uuid [_value=e7ffc99a-665b-49b6-8889-d6dd46ef3807]}}}, TerminationPoint{getTpId=Uri [_value=vxlan-192.168.50.20], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeVxlan, getInterfaceUuid=Uuid [_value=3bbe0d54-36a6-46bc-bcb9-8e654104dfea], getName=vxlan-192.168.50.20, getOfport=1, getOptions=[Options{getOption=local_ip, getValue=192.168.50.22, augmentations={}}, Options{getOption=key, getValue=flow, augmentations={}}, Options{getOption=remote_ip, getValue=192.168.50.20, augmentations={}}], getPortUuid=Uuid [_value=397bc2de-2cc8-452e-bcee-4409cf70c2cd]}}}, TerminationPoint{getTpId=Uri [_value=vxlan-192.168.50.21], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeVxlan, getInterfaceUuid=Uuid [_value=ecc8e438-1508-40e9-8feb-d2ddf38b1bf8], getName=vxlan-192.168.50.21, getOfport=3, getOptions=[Options{getOption=local_ip, getValue=192.168.50.22, augmentations={}}, Options{getOption=key, getValue=flow, augmentations={}}, Options{getOption=remote_ip, getValue=192.168.50.21, augmentations={}}], getPortUuid=Uuid [_value=ea723592-fcc2-4dbc-9269-c7d345505d29]}}}, TerminationPoint{getTpId=Uri [_value=br-int], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation

{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeInternal, getInterfaceUuid=Uuid [_value=c6e55756-a13b-4085-8a92-9bc1594d6543], getName=br-int, getOfport=65534, getPortUuid=Uuid [_value=c03db185-e9c1-4b98-a56a-c36f37d4d288]}

}}], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeAugmentation=OvsdbBridgeAugmentation{getBridgeExternalIds=[BridgeExternalIds{getBridgeExternalIdKey=opendaylight-iid, getBridgeExternalIdValue=/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb://192.168.50.22:52848/bridge/br-int'], augmentations={}}], getBridgeName=OvsdbBridgeName [_value=br-int], getBridgeOpenflowNodeRef=KeyedInstanceIdentifier

{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey [_topologyId=Uri [_value=ovsdb:1]]], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey [_nodeId=Uri [_value=ovsdb://192.168.50.1:6633]]]]}

, getBridgeUuid=Uuid [_value=7cdf423c-a2cb-450a-975b-6122eaa1aff2], getControllerEntry=[ControllerEntry{getControllerUuid=Uuid [_value=6f0cde7c-fc45-43e9-9051-a72712b72709], getTarget=Uri [_value=tcp:192.168.50.1:6633], isIsConnected=true, augmentations={}}], getDatapathId=DatapathId [_value=00:00:3e:42:df:7c:0a:45], getDatapathType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeSystem, getFailMode=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbFailModeSecure, getManagedBy=OvsdbNodeRef [_value=KeyedInstanceIdentifier

{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey [_topologyId=Uri [_value=ovsdb:1]]], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey [_nodeId=Uri [_value=ovsdb://192.168.50.22:52848]]]]}

], getProtocolEntry=[ProtocolEntry{getProtocol=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeProtocolOpenflow13, augmentations={}}]}}}]
java.lang.NullPointerException
at org.opendaylight.ovsdb.openstack.netvirt.impl.SouthboundImpl.extractTerminationPointAugmentation(SouthboundImpl.java:387)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.providers.openflow13.OF13Provider.addTunnelPort(OF13Provider.java:176)[278:org.opendaylight.ovsdb.openstack.net-virt-providers:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.providers.openflow13.OF13Provider.handleInterfaceUpdate(OF13Provider.java:1018)[278:org.opendaylight.ovsdb.openstack.net-virt-providers:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.handleInterfaceUpdate(SouthboundHandler.java:88)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.processPortUpdate(SouthboundHandler.java:303)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.processPortEvent(SouthboundHandler.java:291)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.processEvent(SouthboundHandler.java:236)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.impl.EventDispatcherImpl.dispatchEvent(EventDispatcherImpl.java:96)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.impl.EventDispatcherImpl.access$100(EventDispatcherImpl.java:30)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.impl.EventDispatcherImpl$1.run(EventDispatcherImpl.java:59)[277:org.opendaylight.ovsdb.openstack.net-virt:1.1.0.SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_67]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_67]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_67]



 Comments   
Comment by Flavio Fernandes [ 28/May/15 ]

https://gist.github.com/234db4f8614ed616140a

floatTest1.txt

log:set debug org.opendaylight.neutron
log:set trace org.opendaylight.ovsdb.openstack.netvirt.impl.NeutronL3Adapter

#==

source openrc admin admin

keystone tenant-create --name=tenant1 --enabled=true 2> /dev/null
keystone user-create --name=user1 --pass=user1 --email=user1@example.com 2> /dev/null
keystone user-role-add --user=user1 --role=member --tenant=tenant1 2> /dev/null

keystone tenant-create --name=tenant2 --enabled=true 2> /dev/null
keystone user-create --name=user2 --pass=user2 --email=user2@example.com 2> /dev/null
keystone user-role-add --user=user2 --role=member --tenant=tenant2 2> /dev/null

IMG_ID=$(nova image-list | grep 'cirros-0.3..-x86_64-uec\s' | awk '

{print $2}')
TNT1_ID=$(keystone tenant-list 2> /dev/null | grep '\s'tenant1'' | awk '{print $2}

')
TNT2_ID=$(keystone tenant-list 2> /dev/null | grep '\s'tenant2'' | awk '

{print $2}')

neutron net-create ext1 --router:external --tenant_id=${TNT1_ID} --provider:network_type flat \
--provider:physical_network physnetext1
neutron subnet-create --tenant_id=${TNT1_ID} \
--allocation-pool start=192.168.111.21,end=192.168.111.40 --gateway=192.168.111.254 \
--disable-dhcp --name subext1 ext1 192.168.111.0/24

neutron net-create ext2 --router:external --tenant_id=${TNT2_ID} --provider:network_type flat \
--provider:physical_network physnetext2
neutron subnet-create --tenant_id=${TNT2_ID} \
--allocation-pool start=192.168.111.41,end=192.168.111.60 --gateway=192.168.111.254 \
--disable-dhcp --name subext2 ext2 192.168.111.0/24

# –

unset SERVICE_TOKEN SERVICE_ENDPOINT
export OS_AUTH_URL=http://192.168.50.20:35357/v2.0/
export OS_REGION_NAME=RegionOne

####=================

export OS_USERNAME=user1
export OS_TENANT_NAME=tenant1
export OS_PASSWORD=user1
export PS1='[\u@\h \W(keystone_user1)]\$ '

rm -f id_rsa_demo* ; ssh-keygen -t rsa -b 2048 -N '' -f id_rsa_demo > /dev/null
nova keypair-add --pub-key id_rsa_demo.pub demo_key > /dev/null

neutron router-create rtr1
neutron router-gateway-set rtr1 ext1

## neutron net-create net1 --tenant-id $TNT1_ID --provider:network_type vxlan --provider:segmentation_id 256 > /dev/null
neutron net-create net1
neutron subnet-create net1 10.1.0.0/24 --name subnet1 --dns-nameserver 192.168.111.254
neutron router-interface-add rtr1 subnet1

neutron net-create net2
neutron subnet-create net2 10.2.0.0/24 --name subnet2 --dns-nameserver 192.168.111.254
neutron router-interface-add rtr1 subnet2

NET1_ID=$(neutron net-list | grep -w net1 | awk '{print $2}

') ; echo "net1 $NET1_ID"
NET2_ID=$(neutron net-list | grep -w net2 | awk '

{print $2}') ; echo "net2 $NET2_ID"

nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET1_ID} vm1
nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET1_ID} vm2
nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET2_ID} vm3
nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET2_ID} vm4

for VMNAME in vm1 vm2 vm3 vm4 ; do \
# Floating ip create
neutron floatingip-create ext1
FLOAT_IP=$(neutron floatingip-list | grep 192\.168\.111\. | grep -v 10\..\.0\. | head -1 | awk '{print $5}')
echo "Assigning floating ip ${FLOAT_IP} to ${VMNAME}"
nova floating-ip-associate ${VMNAME} ${FLOAT_IP}
done

##------

export OS_USERNAME=user2
export OS_TENANT_NAME=tenant2
export OS_PASSWORD=user2
export PS1='[\u@\h \W(keystone_user2)]\$ '

rm -f id_rsa_demo* ; ssh-keygen -t rsa -b 2048 -N '' -f id_rsa_demo > /dev/null
nova keypair-add --pub-key id_rsa_demo.pub demo_key > /dev/null

neutron router-create rtr2
neutron router-gateway-set rtr2 ext2

neutron net-create net10
neutron subnet-create net10 10.10.0.0/24 --name subnet10 --dns-nameserver 192.168.111.254
neutron router-interface-add rtr2 subnet10

neutron net-create net20
neutron subnet-create net20 10.20.0.0/24 --name subnet20 --dns-nameserver 192.168.111.254
neutron router-interface-add rtr2 subnet20

NET10_ID=$(neutron net-list | grep -w net10 | awk '{print $2}

') ; echo "net10 $NET10_ID"
NET20_ID=$(neutron net-list | grep -w net20 | awk '

{print $2}

') ; echo "net20 $NET20_ID"

nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET10_ID} vm5
nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET10_ID} vm6
nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET20_ID} vm7
nova boot --poll --flavor m1.nano --image $IMG_ID --key-name demo_key --nic net-id=${NET20_ID} vm8

for VMNAME in vm5 vm6 vm7 vm8 ; do \

  1. Floating ip create
    neutron floatingip-create ext2
    FLOAT_IP=$(neutron floatingip-list | grep 192\.168\.111\. | grep -v 10\...\.0\. | head -1 | awk ' {print $5}

    ')
    echo "Assigning floating ip ${FLOAT_IP} to ${VMNAME}"
    nova floating-ip-associate ${VMNAME} ${FLOAT_IP}
    done

Comment by Flavio Fernandes [ 28/May/15 ]

Yet another case of race condition?!?

dstBridgeNode is null.

So southbound.getBridgeNode(dstNode,
configurationService.getIntegrationBridgeName());

Did not find bridgeNode from dstNode.

for (Node dstNode : nodes.values()) {
InetAddress src = configurationService.getTunnelEndPoint(srcNode);
InetAddress dst = configurationService.getTunnelEndPoint(dstNode);
if ((src != null) && (dst != null)) {
Node dstBridgeNode = southbound.getBridgeNode(dstNode,
configurationService.getIntegrationBridgeName());
if (addTunnelPort(srcBridgeNode, networkType, src, dst))

{ programTunnelRules(networkType, segmentationId, dst, srcBridgeNode, intf, true); }

CRASH ==> if (addTunnelPort(dstBridgeNode, networkType, dst, src))

{ programTunnelRules(networkType, segmentationId, src, dstBridgeNode, intf, false); }

} else {

Comment by Flavio Fernandes [ 28/May/15 ]

Attachment karaf.log has been added with description: karaf.log

Comment by Flavio Fernandes [ 28/May/15 ]

gerrit: https://git.opendaylight.org/gerrit/#/c/21323/

Generated at Wed Feb 07 20:35:40 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.