[CONTROLLER-1405] Provide ability to put content of multiple modules in a single shard Created: 18/Aug/15  Updated: 14/Nov/18  Resolved: 14/Nov/18

Status: Resolved
Project: controller
Component/s: clustering
Affects Version/s: Lithium
Fix Version/s: None

Type: Improvement
Reporter: Moiz Raja Assignee: Natarajan Dhiraviam
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All

Attachments: Text File JolokiaOutput.txt     Text File inv-lacp.log    
Issue Links:
Blocks
is blocked by CONTROLLER-1857 Migrate controller MD-SAL related cod... Resolved
Duplicate
duplicates CONTROLLER-918 Clustering : Implement Fine-Grained s... Resolved

 Description   

Currently we have 2 options. Create a shard per module and put the all the modules which do not have a configured shard into the default shard. It would be nice if we could have the data for some related modules stored together in one shard. This will help us distribute data better across a cluster without forcing us to configure one shard per module which has it's own overhead (RAFT related heartbeats and such)

 Comments   
Comment by Tom Pantelis [ 25/Aug/15 ]

Moiz notes via email:

A shard can contain,
a single modules data
a subset of a modules data
data for multiple modules
The original configuration was designed only for case 1 and 2.

So let’s say inventory could actually be split into 2 shards we would have a configuration as follows,

name=“inventory”
shards = [

{ name=“inventory-1”, replicas=[“member-1”, “member-2”] }

,

{ name=“inventory-2”, replicas=[“member-1”, “member-3”] }

]

The ShardStrategy would then be able to figure out if the data went in “inventory-1” or “inventory-2” shard.

So based on the initial design we would need to have the configuration as follows.

name=“inventory,topology”
shards = [

{ name=“inventory-topology”, replicas=[“member-1”, “member-2”] }

]

which basically says that inventory and topology data goes into the inventory-topology shard.

Comment by Natarajan Dhiraviam [ 12/Oct/15 ]

Conclusion from last discussion was :
+ to support data for related modules stored together in one shard (multiple modules in one shard)
Example : LACP extends node from inventory & it makes sense to have them
both part of same shard.

So initial configuration would like this:
name=“inventory,lacp”
shards = [

{ name=“inventory-lacp”, replicas=[“member-1”, “member-2”] }

]

+ Until micro-sharding strategy aspects are worked out, spanning a module across shards shall not be considered.

Comment by Natarajan Dhiraviam [ 12/Nov/15 ]

For multi-module sharding, tried aggregating inventory & lacp into a single shard, in a Single Node cluster setup. Was able to see inventory-lacp shard becoming Leader successfully.
Below was the modification in the conf file.
modules.conf:
-------------

{ name = "lacp" namespace = "urn:opendaylight:lacp:lacp-main" shard-strategy = "module" }

module-shards.conf:
-------------------
{
name = "inventory,lacp"
shards = [

{ name="inventory-lacp" replicas = [ "member-1" ] }

]
}

Attached is the karaf log extract, jolokia output

Comment by Natarajan Dhiraviam [ 12/Nov/15 ]

For multi-module sharding, tried aggregating inventory & lacp into a single shard, in a Single Node cluster setup. Was able to see inventory-lacp shard becoming Leader successfully.
Below was the modification in the conf file.
modules.conf:
-------------

{ name = "lacp" namespace = "urn:opendaylight:lacp:lacp-main" shard-strategy = "module" }

module-shards.conf:
-------------------
{
name = "inventory,lacp"
shards = [

{ name="inventory-lacp" replicas = [ "member-1" ] }

]
}

Attached is the karaf log extract, jolokia output

Comment by Natarajan Dhiraviam [ 12/Nov/15 ]

Attachment inv-lacp.log has been added with description: Karaf output inventory-lacp multi-module shard karaf log extract for single node cluster

Comment by Natarajan Dhiraviam [ 12/Nov/15 ]

Attachment JolokiaOutput.txt has been added with description: JolokiaOutput inventory-lacp multi-module shard Jolokia output for single node cluster

Generated at Wed Feb 07 19:55:28 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.