Uploaded image for project: 'mdsal'
  1. mdsal
  2. MDSAL-195

ClusterSingletonService is not closed after Leader moves to IsolatedLeader

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Resolution: Done
    • None
    • None
    • None
    • None
    • Operating System: All
      Platform: All

    • 6540
    • High

    Description

      After isolation of one leader node, new leader is selected and start its services but the old leader does not close its services and becomes unresponsive

      2016-08-25 11:30:46,838 | WARN | ult-dispatcher-3 | ShardManager | 169 - org.opendaylight.controller.sal-distributed-datastore - 1.5.0.SNAPSHOT | Supervisor Strategy caught unexpected exception - resuming
      java.lang.IllegalArgumentException: Invalid combination of wasOwner: true, isOwner: true, hasOwner: true
      at com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)[38:com.google.guava:18.0.0]
      at org.opendaylight.mdsal.eos.common.api.EntityOwnershipChangeState.from(EntityOwnershipChangeState.java:100)[127:org.opendaylight.mdsal.eos-common-api:2.2.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerSupport.notifyListeners(EntityOwnershipListenerSupport.java:107)[169:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerSupport.notifyListeners(EntityOwnershipListenerSupport.java:100)[169:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipListenerSupport.notifyEntityOwnershipListeners(EntityOwnershipListenerSupport.java:89)[169:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipShard.lambda$notifyAllListeners$1(EntityOwnershipShard.java:290)[169:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipShard.searchForEntities(EntityOwnershipShard.java:475)[169:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipShard.notifyAllListeners(EntityOwnershipShard.java:272)[169:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.datastore.entityownership.EntityOwnershipShard.onStateChanged(EntityOwnershipShard.java:308)[169:org.opendaylight.controller.sal-distributed-datastore:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.raft.RaftActor.handleBehaviorChange(RaftActor.java:465)[164:org.opendaylight.controller.sal-akka-raft:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.raft.RaftActor.switchBehavior(RaftActor.java:398)[164:org.opendaylight.controller.sal-akka-raft:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:297)[164:org.opendaylight.controller.sal-akka-raft:1.5.0.SNAPSHOT]
      at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:29)[163:org.opendaylight.controller.sal-clustering-commons:1.5.0.SNAPSHOT]
      at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[157:com.typesafe.akka.persistence:2.4.7]
      at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:97)[163:org.opendaylight.controller.sal-clustering-commons:1.5.0.SNAPSHOT]
      at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[151:com.typesafe.akka.actor:2.4.7]
      at akka.actor.Actor$class.aroundReceive(Actor.scala:484)[151:com.typesafe.akka.actor:2.4.7]
      at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[157:com.typesafe.akka.persistence:2.4.7]
      at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:633)[157:com.typesafe.akka.persistence:2.4.7]
      at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:179)[157:com.typesafe.akka.persistence:2.4.7]
      at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[157:com.typesafe.akka.persistence:2.4.7]
      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[151:com.typesafe.akka.actor:2.4.7]
      at akka.actor.ActorCell.invoke(ActorCell.scala:495)[151:com.typesafe.akka.actor:2.4.7]
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[151:com.typesafe.akka.actor:2.4.7]
      at akka.dispatch.Mailbox.run(Mailbox.scala:224)[151:com.typesafe.akka.actor:2.4.7]
      at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[151:com.typesafe.akka.actor:2.4.7]
      at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[147:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
      at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[147:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
      at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[147:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
      at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[147:org.scala-lang.scala-library:2.11.8.v20160304-115712-1706a37eb8]
      2016-08-25 11:30:46,841 | WARN | lt-dispatcher-17 | OneForOneStrategy | 152 - com.typesafe.akka.slf4j - 2.4.7 | Invalid combination of wasOwner: true, isOwner: true, hasOwner: true
      2016-08-25 11:30:47,346 | WARN | lt-dispatcher-14 | Shard | 164 - org.opendaylight.controller.sal-akka-raft - 1.5.0.SNAPSHOT | member-3-shard-inventory-operational: At least 1 followers need to be active, Switching member-3-shard-inventory-operational from Leader to IsolatedLeader

      Attachments

        1. member3_karaf.log
          509 kB
          Vaclav Demcak
        2. member2_karaf.log
          441 kB
          Vaclav Demcak
        3. member1_karaf.log
          389 kB
          Vaclav Demcak

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              vdemcak@cisco.com Vaclav Demcak
              martin.mihalek@pantheon.sk Martin Mihálek
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: