Operating System: All
Platform: All

See weather event: https://wiki.opendaylight.org/view/Weather#Clustered_Data_Store_Switch_over

See patch:
https://git.opendaylight.org/gerrit/#/c/14313/

> this would switch to clustering being on by default

This sounds like there will be a way for users to override this new default, and use InMemoryDataStore if they wish to (let us say for performance reasons).
Is there a wiki page (or other document) describing the way to apply such an override, so that we can test if it works?

Or should I read "on by default" as "unconditionally on"?

My original email below. If clustering is turned on by default and percolates to SFC it will most likely make it unusable.

It is very close to Lithium to make those changes. I think we need to have a larger discussion of who will report, chase and fix bugs.

=====

Some time ago I had the action item to test clustering with SFC and did that recently. I tried a simple single node as written in: https://wiki.opendaylight.org/view/Running_and_testing_an_OpenDaylight_Cluster

In a nutshell there are so many errors that I’m not even try to list them all. Even simple tests fail.

Anybody has feedback on actual ODL Application that has listeners, reads from data store, commit to the data store from executors, etc that has been tested successfully with clustering? The wiki says "You should be able to test the controller now as you would normally test a regular, single instance of the controller" but I wonder is something is amiss.

Thanks,

Reinaldo

Some examples of the 100s of messages I get

2015-03-24 00:40:35,081 | ERROR | DOM-CFG-DCL-4 | QueuedNotificationManager | 53 - org.opendaylight.yangtools.util - 0.7.0.SNAPSHOT | DataChangeListenerQueueMgr: Error notifying listener AbstractObjectRegistration

java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@56412b3 rejected from java.util.concurrent.ThreadPoolExecutor@7df7e573[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 2]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)[:1.7.0_60]
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)[:1.7.0_60]
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)[:1.7.0_60]
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)[:1.7.0_60]
at org.opendaylight.sfc.sbrest.provider.listener.SbRestSfEntryDataListener.onDataChanged(SbRestSfEntryDataListener.java:90)[280:org.opendaylight.sfc.sb-rest:0.1.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.binding.impl.AbstractForwardedDataBroker$TranslatingDataChangeInvoker.onDataChanged(AbstractForwardedDataBroker.java:134)[145:org.opendaylight.controller.sal-binding-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$1.invokeListener(InMemoryDOMDataStore.java:73)[148:org.opendaylight.controller.sal-inmemory-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.dom.store.impl.InMemoryDOMDataStore$1.invokeListener(InMemoryDOMDataStore.java:67)[148:org.opendaylight.controller.sal-inmemory-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$NotificationTask.notifyListener(QueuedNotificationManager.java:411)[53:org.opendaylight.yangtools.util:0.7.0.SNAPSHOT]
at org.opendaylight.yangtools.util.concurrent.QueuedNotificationManager$NotificationTask.run(QueuedNotificationManager.java:382)[53:org.opendaylight.yangtools.util:0.7.0.SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_60]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_60]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_60]
2015-03-24 00:40:35,571 | ERROR | lt-dispatcher-43 | DataChangeListener | 308 - org.opendaylight.controller.sal-distributed-datastore - 1.2.0.SNAPSHOT | Error notifying listener org.opendaylight.sfc.sfc_netconf.provider.listener.SfcNetconfNodeDataListener
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@61ada5f4 rejected from java.util.concurrent.ThreadPoolExecutor@7df7e573[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 2]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)[:1.7.0_60]
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)[:1.7.0_60]
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)[:1.7.0_60]
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)[:1.7.0_60]
at org.opendaylight.sfc.provider.api.SfcProviderServiceForwarderAPI.putServiceFunctionForwarderExecutor(SfcProviderServiceForwarderAPI.java:160)[220:org.opendaylight.sfc.provider:0.1.0.SNAPSHOT]
at org.opendaylight.sfc.sfc_netconf.provider.listener.SfcNetconfNodeDataListener.onDataChanged(SfcNetconfNodeDataListener.java:143)[227:org.opendaylight.sfc.netconf:0.1.0.SNAPSHOT]
at org.opendaylight.controller.md.sal.binding.impl.AbstractForwardedDataBroker$TranslatingDataChangeInvoker.onDataChanged(AbstractForwardedDataBroker.java:134)[145:org.opendaylight.controller.sal-binding-broker-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataChangeListener.dataChanged(DataChangeListener.java:66)[308:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.DataChangeListener.handleReceive(DataChangeListener.java:39)[308:org.opendaylight.controller.sal-distributed-datastore:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor.onReceive(AbstractUntypedActor.java:34)[300:org.opendaylight.controller.sal-clustering-commons:1.2.0.SNAPSHOT]
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)[293:com.typesafe.akka.actor:2.3.9]
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)[293:com.typesafe.akka.actor:2.3.9]
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)[293:com.typesafe.akka.actor:2.3.9]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)[293:com.typesafe.akka.actor:2.3.9]
at akka.actor.ActorCell.invoke(ActorCell.scala:487)[293:com.typesafe.akka.actor:2.3.9]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)[293:com.typesafe.akka.actor:2.3.9]
at akka.dispatch.Mailbox.run(Mailbox.scala:221)[293:com.typesafe.akka.actor:2.3.9]
at akka.dispatch.Mailbox.exec(Mailbox.scala:231)[293:com.typesafe.akka.actor:2.3.9]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[290:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[290:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[290:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[290:org.scala-lang.scala-library:2.10.4.v20140209-180020-VFINAL-b66a39653b]