Uploaded image for project: 'controller'
  1. controller
  2. CONTROLLER-1328

Clustering: Recovery misses flows installed

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Resolution: Done
    • Post-Helium
    • None
    • clustering
    • None
    • Operating System: All
      Platform: All

    • 3260
    • Highest

    Description

      Steps to reproduce
      1. Create an OFP distribution on master
      2. Start the controller install odl-openflowplugin-all
      3. Connect mininet - sudo mn --controller=remote,ip=<host> --topo tree,3
      4. Run config flow blaster a couple of times
      ./flow_config_blaster.py --auth --threads 5 --flows 5000 --no-delete
      5. Check the number of flows in config and note the number
      5. Stop controller gracefully.
      6. Stop mininet
      7. Start the controller
      8. Connect mininet
      9. Check the number of flows in config. Around 400 are missing.

      The log is filled with messages :

      2015-05-18 14:07:20,796 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | Recovery snapshot applied for member-1-shard-inventory-config in 2.119 s: snapshotIndex=39999, snapshotTerm=1, journal-size=0
      2015-05-18 14:07:20,805 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received ReplicatedLogEntry for recovery: index: 40322, size: 1445
      2015-05-18 14:07:20,806 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received ReplicatedLogEntry for recovery: index: 40323, size: 1441
      2015-05-18 14:07:20,806 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received ReplicatedLogEntry for recovery: index: 40324, size: 1441
      2015-05-18 14:07:20,806 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received ReplicatedLogEntry for recovery: index: 40325, size: 1445
      2015-05-18 14:07:20,807 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received apply journal entries for recovery, applying to state: 40000 to 40322
      2015-05-18 14:07:20,813 | ERROR | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | Log entry not found for index 40004
      2015-05-18 14:07:20,813 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received apply journal entries for recovery, applying to state: 40004 to 40323
      2015-05-18 14:07:20,814 | ERROR | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | Log entry not found for index 40004
      2015-05-18 14:07:20,814 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received apply journal entries for recovery, applying to state: 40004 to 40324
      2015-05-18 14:07:20,814 | ERROR | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | Log entry not found for index 40004
      2015-05-18 14:07:20,814 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received apply journal entries for recovery, applying to state: 40004 to 40325
      2015-05-18 14:07:20,814 | ERROR | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | Log entry not found for index 40004
      2015-05-18 14:07:20,814 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received ReplicatedLogEntry for recovery: index: 40326, size: 1441
      2015-05-18 14:07:20,814 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received apply journal entries for recovery, applying to state: 40004 to 40326
      2015-05-18 14:07:20,814 | ERROR | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | Log entry not found for index 40005
      2015-05-18 14:07:20,814 | INFO | lt-dispatcher-25 | Shard | 175 - org.opendaylight.controller.sal-akka-raft - 1.2.0.SNAPSHOT | member-1-shard-inventory-config: Received ReplicatedLogEntry for recovery: index: 40327, size: 1445

      The snapshot was taken at index=39999 with 0 unapplied entries.
      The persistent journal starts with the index=40322.
      Where did the 322 logs go?

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            kramesha@cisco.com Kamal Rameshan
            kramesha@cisco.com Kamal Rameshan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: