[OVSDB-201] NPE while programming Port Security ACL Created: 05/Sep/15  Updated: 03/May/18  Resolved: 12/Jan/16

Status: Resolved
Project: ovsdb
Component/s: openstack.net-virt
Affects Version/s: unspecified
Fix Version/s: None

Type: Bug
Reporter: Anil Vishnoi Assignee: Flavio Fernandes
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


Attachments: File openstackTalk.cap    
External issue ID: 4265
Priority: High

 Description   

Getting following NPE in net-virt master branch code, when i spawn VM in the openstack :

2015-09-06 01:01:44,364 | ERROR | ntDispatcherImpl | OF13Provider | 260 - org.opendaylight.ovsdb.openstack.net-virt-providers - 1.2.1.SNAPSHOT | Exception in programming Local Rules for OvsdbTerminationPointAugmentation{getInterfaceExternalIds=[InterfaceExternalIds{getExternalIdKey=iface-id, getExternalIdValue=8a16117d-5a18-4df2-a0c0-7d01feb0591e, augmentations={}}, InterfaceExternalIds{getExternalIdKey=iface-status, getExternalIdValue=active, augmentations={}}, InterfaceExternalIds{getExternalIdKey=vm-id, getExternalIdValue=759b3724-8f0b-47aa-ba1b-43e7b680c5d8, augmentations={}}, InterfaceExternalIds{getExternalIdKey=attached-mac, getExternalIdValue=fa:16:3e:87:e3:34, augmentations={}}], getInterfaceUuid=Uuid [_value=f78dd6d1-9b2e-49c1-bdb4-532c6d330177], getName=tap8a16117d-5a, getOfport=2, getPortUuid=Uuid [_value=5f474252-5691-42a1-b623-3fbfa3897034]} on Node{getNodeId=Uri [_value=ovsdb:node4/bridge/br-int], getTerminationPoint=[TerminationPoint{getTpId=Uri [_value=br-int], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation

{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeInternal, getInterfaceUuid=Uuid [_value=9263b156-8c7d-4c80-b817-42b444627dca], getName=br-int, getOfport=65534, getPortUuid=Uuid [_value=36258dd6-65bd-4431-8009-8d5ba0c67b33]}

}}, TerminationPoint{getTpId=Uri [_value=vxlan-192.168.201.128], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation{getInterfaceType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.InterfaceTypeVxlan, getInterfaceUuid=Uuid [_value=cb814fc4-ea21-4f70-adf7-0e85f8a75f81], getName=vxlan-192.168.201.128, getOfport=1, getOptions=[Options{getOption=key, getValue=flow, augmentations={}}, Options{getOption=local_ip, getValue=192.168.201.129, augmentations={}}, Options{getOption=remote_ip, getValue=192.168.201.128, augmentations={}}], getPortExternalIds=[PortExternalIds{getExternalIdKey=opendaylight-iid, getExternalIdValue=/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb:node4/bridge/br-int']/network-topology:termination-point[network-topology:tp-id='vxlan-192.168.201.128'], augmentations={}}], getPortUuid=Uuid [_value=00557250-6626-4e21-82d6-6d3cb088bb45]}}}, TerminationPoint{getTpId=Uri [_value=tap8a16117d-5a], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbTerminationPointAugmentation=OvsdbTerminationPointAugmentation{getInterfaceExternalIds=[InterfaceExternalIds{getExternalIdKey=iface-id, getExternalIdValue=8a16117d-5a18-4df2-a0c0-7d01feb0591e, augmentations={}}, InterfaceExternalIds{getExternalIdKey=iface-status, getExternalIdValue=active, augmentations={}}, InterfaceExternalIds{getExternalIdKey=vm-id, getExternalIdValue=759b3724-8f0b-47aa-ba1b-43e7b680c5d8, augmentations={}}, InterfaceExternalIds{getExternalIdKey=attached-mac, getExternalIdValue=fa:16:3e:87:e3:34, augmentations={}}], getInterfaceUuid=Uuid [_value=f78dd6d1-9b2e-49c1-bdb4-532c6d330177], getName=tap8a16117d-5a, getOfport=2, getPortUuid=Uuid [_value=5f474252-5691-42a1-b623-3fbfa3897034]}}}], augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeAugmentation=OvsdbBridgeAugmentation{getBridgeExternalIds=[BridgeExternalIds{getBridgeExternalIdKey=opendaylight-iid, getBridgeExternalIdValue=/network-topology:network-topology/network-topology:topology[network-topology:topology-id='ovsdb:1']/network-topology:node[network-topology:node-id='ovsdb:node4/bridge/br-int'], augmentations={}}], getBridgeName=OvsdbBridgeName [_value=br-int], getBridgeUuid=Uuid [_value=1c367214-749f-4cc8-b777-d5d0794cbc0c], getControllerEntry=[ControllerEntry{getControllerUuid=Uuid [_value=172e3e00-7734-43cd-a635-6e64022f3eec], getTarget=Uri [_value=tcp:192.168.57.1:6653], isIsConnected=true, augmentations={}}], getDatapathId=DatapathId [_value=00:00:16:72:36:1c:c8:4c], getDatapathType=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.DatapathTypeSystem, getFailMode=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbFailModeSecure, getManagedBy=OvsdbNodeRef [_value=KeyedInstanceIdentifier

{targetType=interface org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node, path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.NetworkTopology, org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.Topology[key=TopologyKey [_topologyId=Uri [_value=ovsdb:1]]], org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang.network.topology.rev131021.network.topology.topology.Node[key=NodeKey [_nodeId=Uri [_value=ovsdb:node4]]]]}

], getProtocolEntry=[ProtocolEntry{getProtocol=class org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.ovsdb.rev150105.OvsdbBridgeProtocolOpenflow13, augmentations={}}]}}}
java.lang.NullPointerException
at org.opendaylight.ovsdb.openstack.netvirt.providers.openflow13.services.IngressAclService.programPortSecurityAcl(IngressAclService.java:64)[260:org.opendaylight.ovsdb.openstack.net-virt-providers:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.providers.openflow13.OF13Provider.programLocalSecurityGroupRules(OF13Provider.java:1037)[260:org.opendaylight.ovsdb.openstack.net-virt-providers:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.providers.openflow13.OF13Provider.programLocalRules(OF13Provider.java:765)[260:org.opendaylight.ovsdb.openstack.net-virt-providers:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.providers.openflow13.OF13Provider.handleInterfaceUpdate(OF13Provider.java:1057)[260:org.opendaylight.ovsdb.openstack.net-virt-providers:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.handleInterfaceUpdate(SouthboundHandler.java:87)[259:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.processPortUpdate(SouthboundHandler.java:300)[259:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.processPortEvent(SouthboundHandler.java:287)[259:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.SouthboundHandler.processEvent(SouthboundHandler.java:233)[259:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.impl.EventDispatcherImpl.dispatchEvent(EventDispatcherImpl.java:95)[259:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.impl.EventDispatcherImpl.access$200(EventDispatcherImpl.java:28)[259:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.SNAPSHOT]
at org.opendaylight.ovsdb.openstack.netvirt.impl.EventDispatcherImpl$1.run(EventDispatcherImpl.java:57)[259:org.opendaylight.ovsdb.openstack.net-virt:1.2.1.SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_80]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_80]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_80]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_80]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_80]



 Comments   
Comment by Anil Vishnoi [ 05/Sep/15 ]

Steps to recreate the issue:

1) Build ovsdb master code base and start the controller
2) Connect OpenStack compute/control/network node to ODL controller
3) Create network
4) Spawn VM and attach it to the network (i do it through Horizon and not CLI)

and you should see the above exception.

Comment by Anil Vishnoi [ 05/Sep/15 ]

L2 connectivity breaks because all the required flows are not getting installed because of this exception.

Comment by Aswin Suryanarayanan [ 07/Sep/15 ]

I am able to reproduce the issue with a custom security group, but not with the default. It seems to be a timing issue. The security group create is called along with a port create not when the SG is created from horizon. So there are chances that empty value is returned when port create tries to do retrieve the security group info in response to a Southbound Event. I am analyzing more on it. Will add a try catch to fix issue with other flows.

Comment by Flavio Fernandes [ 11/Sep/15 ]

steps to reproduce:

source openrc admin admin

keystone tenant-create --name=tenant1 --enabled=true 2> /dev/null

keystone user-create --name=user1 --pass=user1 --email=user1@example.com 2> /dev/null

keystone user-role-add --user=user1 --role=member --tenant=tenant1 2> /dev/null

TNT1_ID=$(keystone tenant-list 2> /dev/null | grep '\s'tenant1'' | awk '

{print $2}

')

neutron net-create ext1 --router:external --tenant_id=${TNT1_ID} \
--provider:network_type flat \
--provider:physical_network physnetext1

Comment by Flavio Fernandes [ 11/Sep/15 ]

POST /controller/nb/v2/neutron/security-group-rules HTTP/1.1
Host: 192.168.50.1:8080
Content-Length: 1699
Accept-Encoding: gzip, deflate
Accept: /
User-Agent: python-requests/2.7.0 CPython/2.7.6 Linux/3.13.0-24-generic
Connection: keep-alive
Content-Type: application/json
Authorization: Basic YWRtaW46YWRtaW4=

{
"security_group_rules": [

{ "remote_group_id": null, "direction": "egress", "remote_ip_prefix": null, "protocol": null, "ethertype": "IPv6", "tenant_id": "2640ee2ac2474bf3906e482047204fcb", "port_range_max": null, "port_range_min": null, "id": "35fb0f34-c8d3-416d-a205-a2c75f7b8e22", "security_group_id": "70f1b157-e79b-44dc-85a8-7de0fc9f2aab" }

,

{ "remote_group_id": null, "direction": "egress", "remote_ip_prefix": null, "protocol": null, "ethertype": "IPv4", "tenant_id": "2640ee2ac2474bf3906e482047204fcb", "port_range_max": null, "port_range_min": null, "id": "63814eed-bc12-4fe4-8b17-2af178224c71", "security_group_id": "70f1b157-e79b-44dc-85a8-7de0fc9f2aab" }

,

{ "remote_group_id": "70f1b157-e79b-44dc-85a8-7de0fc9f2aab", "direction": "ingress", "remote_ip_prefix": null, "protocol": null, "ethertype": "IPv6", "tenant_id": "2640ee2ac2474bf3906e482047204fcb", "port_range_max": null, "port_range_min": null, "id": "ccb9823e-559b-4d84-b656-2739f8e56d89", "security_group_id": "70f1b157-e79b-44dc-85a8-7de0fc9f2aab" }

,

{ "remote_group_id": "70f1b157-e79b-44dc-85a8-7de0fc9f2aab", "direction": "ingress", "remote_ip_prefix": null, "protocol": null, "ethertype": "IPv4", "tenant_id": "2640ee2ac2474bf3906e482047204fcb", "port_range_max": null, "port_range_min": null, "id": "fbc3f809-7378-40a4-822f-7a70f6ccba98", "security_group_id": "70f1b157-e79b-44dc-85a8-7de0fc9f2aab" }

]
}HTTP/1.1 500 Server Error
Content-Type: text/html;charset=ISO-8859-1
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 6292
Server: Jetty(8.1.15.v20140411)

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /controller/nb/v2/neutron/security-group-rules. Reason:
<pre> Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.NullPointerException
at org.opendaylight.neutron.transcriber.NeutronSecurityRuleInterface.updateSecGroupRuleInSecurityGroup(NeutronSecurityRuleInterface.java:80)
at org.opendaylight.neutron.transcriber.NeutronSecurityRuleInterface.addNeutronSecurityRule(NeutronSecurityRuleInterface.java:145)
at org.opendaylight.neutron.northbound.api.NeutronSecurityRulesNorthbound.createSecurityRules(NeutronSecurityRulesNorthbound.java:234)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1496)
at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:247)
at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:210)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:69)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:240)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:75)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
</pre>
<hr /><i><small>Powered by Jetty://</small></i><br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>

</body>
</html>

Comment by Flavio Fernandes [ 11/Sep/15 ]

Attachment openstackTalk.cap has been added with description: tcpdump that shows the crash

Comment by Flavio Fernandes [ 11/Sep/15 ]

https://git.opendaylight.org/gerrit/#/c/26870/

https://www.getpostman.com/collections/8e169cd17edfdb11dcb9

Generated at Wed Feb 07 20:35:46 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.