[OPNFLWPLUG-1025] Karaf reboot with shutdown -r, bundles are entering to Failure state and SB communication are lost Created: 10/Jul/18  Updated: 22/Oct/18  Resolved: 22/Oct/18

Status: Resolved
Project: OpenFlowPlugin
Component/s: openflowplugin
Affects Version/s: None
Fix Version/s: Fluorine-SR2, Neon

Type: Bug Priority: Low
Reporter: Srinivas Balajinaidu Assignee: Suja T
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ubuntu 16.04 operating system
windows 7


Attachments: Text File bundle_diag.txt     Text File karaf_restart_error.txt    
Issue Links:
Blocks
is blocked by CONTROLLER-1863 Bundles in error state after karaf re... Open

 Description   

Karaf reboot with shutdown -r -f, bundles are entering to Failure state and SB communication are lost.

build : karaf-0.9.0-20180710.005302-1532.zip
features installed :
featuresBoot=config,standard,region,package,kar,ssh,management,odl-netvirt-openstack,odl-restconf-all,odl-l2switch-all,odl-openflowplugin-all-he,odl-mdsal-apidocs,odl-restconf-all

#########################################################################
Logs:
#########################################################################
opendaylight-user@root>shutdown -r

opendaylight-user@root>bundle:list | grep -v Active
START LEVEL 100 , List Threshold: 50
ID State Lvl Version Name

113 Resolved 80 4.1.5 Apache Karaf :: Diagnostic :: Boot
202 Resolved 80 0.8.2.SNAPSHOT config-persister-directory-xml-adapter, Hosts: 204
203 Resolved 80 0.8.2.SNAPSHOT config-persister-file-xml-adapter, Hosts: 204
243 Failure 80 0.4.2.SNAPSHOT ODL :: genius :: alivenessmonitor-impl
244 GracePeriod 80 0.4.2.SNAPSHOT ODL :: genius :: alivenessmonitor-impl-protocols
246 GracePeriod 80 0.4.2.SNAPSHOT ODL :: genius :: arputil-impl
247 Failure 80 0.4.2.SNAPSHOT ODL :: genius :: fcapsapplication-impl
253 GracePeriod 80 0.4.2.SNAPSHOT ODL :: genius :: interfacemanager-impl
254 GracePeriod 80 0.4.2.SNAPSHOT ODL :: genius :: interfacemanager-shell
256 GracePeriod 80 0.4.2.SNAPSHOT ODL :: genius :: itm-impl
260 Failure 80 0.4.2.SNAPSHOT ODL :: genius :: mdsalutil-impl
288 Failure 80 0.7.2.SNAPSHOT hosttracker-impl
346 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: aclservice-impl
347 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: aclservice-shell
350 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: bgpmanager-impl
353 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: cloud-servicechain-impl
356 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: dhcpservice-impl
358 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: elanmanager-impl
360 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: fibmanager-impl
361 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: fibmanager-shell
363 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: ipv6service-impl
364 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: ipv6service-shell
367 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: natservice-impl
369 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: neutronvpn-impl
372 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: qosservice-impl
375 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: statemanager-impl
377 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: statistics-impl
379 GracePeriod 80 0.6.2.SNAPSHOT ODL :: netvirt :: vpnmanager-impl
387 GracePeriod 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.applications.forwardingrules-manager
389 Failure 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.applications.of-switch-config-pusher
391 GracePeriod 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.applications.topology-lldp-discovery
392 Failure 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.applications.topology-manager
393 GracePeriod 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.openflowplugin-blueprint-config
396 GracePeriod 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.openflowplugin-extension-nicira
397 GracePeriod 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.openflowplugin-extension-onf
403 GracePeriod 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.openflowjava-extension-nicira
405 GracePeriod 80 0.6.2.SNAPSHOT org.opendaylight.openflowplugin.openflowjava.openflowjava-blueprint-config

stack@pike-controller:/opt/stack/devstack$ show
043b676e-4739-44f0-93a8-d33fb42bb28b
Manager "tcp:192.168.100.201:6640"
is_connected: true
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Bridge br-int
Controller "tcp:192.168.100.201:6653"
Port "tap1d81df36-e8"
Interface "tap1d81df36-e8"
Port "tapa043d237-c2"
Interface "tapa043d237-c2"
type: internal
Port "tap9e51efe6-3c"
Interface "tap9e51efe6-3c"
Port br-int
Interface br-int
type: internal
Port "tun0ea884a4591"
Interface "tun0ea884a4591"
type: vxlan
options:

{key=flow, local_ip="192.168.100.202", remote_ip="192.168.100.203"}

Port "tapd0dfbfdf-38"
Interface "tapd0dfbfdf-38"
Port "tapcf0ed4c1-d8"
Interface "tapcf0ed4c1-d8"
type: internal
Port "tapfa65d267-83"
Interface "tapfa65d267-83"
type: internal
Port "tape244e18c-83"
Interface "tape244e18c-83"
Port "tap0d5ba1ac-95"
Interface "tap0d5ba1ac-95"
Port "tap2ce190ef-03"
Interface "tap2ce190ef-03"
type: internal
Port "tap54209f4d-20"
Interface "tap54209f4d-20"
type: internal
ovs_version: "2.9.0"



 Comments   
Comment by Gobinath Suganthan [ 20/Aug/18 ]

This seems to be related to https://jira.opendaylight.org/browse/CONTROLLER-1627 which was fixed in Boron. Similar exception was observed during the cluster restart scenario through Quarantine.

The fix was to restart the jvm itself rather than the karaf.

I'm not sure if the "shutdown -r" falls under this scenario.

I tried to manually enforce this by setting

 opendaylight-user@root>property karaf.restart.jvm true

 opendaylight-user@root>shutdown -r

in which case all the bundles got resolved with no issues. 

Comment by Gobinath Suganthan [ 18/Sep/18 ]

Avishnoi Have raise a Jira in the controller project as suggested

https://jira.opendaylight.org/browse/CONTROLLER-1863

 

Comment by Anil Vishnoi [ 22/Oct/18 ]

Bug is opened with controller project. Closing this bug.

Generated at Wed Feb 07 20:34:00 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.