[CONTROLLER-1431] "java.lang.OutOfMemoryError: GC overhead limit exceeded" when re-installing DIDM Created: 22/Oct/15  Updated: 19/Oct/17  Resolved: 05/Jul/17

Status: Resolved
Project: controller
Component/s: config
Affects Version/s: Beryllium
Fix Version/s: None

Type: Bug
Reporter: Linh Pham Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


External issue ID: 4516
Priority: Normal

 Description   

Installing DIDM and Openflowplugin seems to work well the first time after the first fresh start of Karaf. After the initial installation, different exceptions occur on uninstall and reinstall of the same set of Karaf features for DIDM and Openflowplugin causing Karaf to hang.

Steps to reproduce:

1. Download and install Karaf integration distribution (link to download: https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/distribution-karaf/0.4.0-SNAPSHOT/distribution-karaf-0.4.0-20151022.192517-1764.zip)

2. Install DIDM and openflowplugin:
feature:install odl-openflowplugin-all-li
feature:install odl-didm-identification
feature:install odl-didm-identification-api
feature:install odl-didm-drivers-api
feature:install odl-didm-hp3800

3. Uninstall DIDM and openflowplugin:
feature:uninstall odl-didm-hp3800
feature:uninstall odl-didm-drivers-api
feature:uninstall odl-didm-identification
feature:uninstall odl-didm-identification-api
feature:install odl-openflowplugin-all-li

4. Reinstall DIDM and openflowplugin (repeat step 2)

Different exceptions occur during step 3 and 4 (not seen on the the first fresh installation of Karaf).

Example 1: Karaf terminal capture

opendaylight-user@root>feature:install odl-openflowplugin-all-li
opendaylight-user@root>feature:install odl-didm-identification odl-didm-identification-api odl-didm-drivers-api odl-didm-hp3800
opendaylight-user@root>feature:uninstall odl-didm-hp3800 odl-didm-drivers-api odl-didm-identification odl-didm-identification-api odl-openflowplugin-all-li
Uncaught error from thread [opendaylight-cluster-data-akka.actor.default-dispatcher-174] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[opendaylight-cluster-data]
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.String.substring(String.java:1877)
at java.lang.Class.getSimpleName(Class.java:1172)
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:86)
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:534)
at akka.persistence.Recovery$State$class.process(Recovery.scala:30)
at akka.persistence.ProcessorImpl$$anon$2.process(Processor.scala:103)
at akka.persistence.ProcessorImpl$$anon$2.aroundReceive(Processor.scala:114)
at akka.persistence.Recovery$class.aroundReceive(Recovery.scala:265)
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(Eventsourced.scala:428)
at akka.persistence.Eventsourced$$anon$2.doAroundReceive(Eventsourced.scala:82)
at akka.persistence.Eventsourced$$anon$2.aroundReceive(Eventsourced.scala:78)
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:369)
at akka.persistence.UntypedPersistentActor.aroundReceive(Eventsourced.scala:428)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
at akka.dispatch.Mailbox.run(Mailbox.scala:221)
at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:1253)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1346)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
sdn@cell8-ODLC1:~/distribution-karaf-0.4.0-SNAPSHOT$>

Example 2: Exceptions found in karaf.log

2015-10-22 13:31:59,431 | ERROR | l for user karaf | ExtensibleBundleTracker | 114 - org.opendaylight.controller.config-manager - 0.4.0.SNAPSHOT | Failed to remove bundle org.opendaylight.didm.hp3800.hp-3800-impl_0.2.0.SNAPSHOT [273]
java.lang.IllegalStateException: Error while copying old configuration from ModuleInternalInfo [name=ModuleIdentifier

{factoryName='device-identification-manager', instanceName='device-identification-manager'}

] to org.opendaylight.yang.gen.v1.urn.opendaylight.params.xml.ns.yang.didm.identification.impl.rev150202.DeviceIdentificationManagerModuleFactory@22b7a92f
at org.opendaylight.controller.config.manager.impl.ConfigTransactionControllerImpl.copyExistingModule(ConfigTransactionControllerImpl.java:216)
at org.opendaylight.controller.config.manager.impl.ConfigTransactionControllerImpl.copyExistingModulesAndProcessFactoryDiff(ConfigTransactionControllerImpl.java:113)
at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.beginConfigInternal(ConfigRegistryImpl.java:190)
at org.opendaylight.controller.config.manager.impl.ConfigRegistryImpl.beginConfig(ConfigRegistryImpl.java:147)



 Comments   
Comment by Vratko Polak [ 23/Oct/15 ]

Feature uninstallation is not supported in Beryllium, specifically Config Subsystem does not know to unload configuration upon uninstall. That may explain the IllegalStateException.

On the other hand, OutOfMemoryError should not happen just because of feature uninstall. That is severe enough to be considered a Beryllium config subsystem Bug.

Comment by Robert Varga [ 13/Nov/15 ]

A memory dump is required to debug OOM errors.

Comment by Tom Pantelis [ 05/Jul/17 ]

This bug is old and I assume it's no longer an issue.

Generated at Wed Feb 07 19:55:32 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.