[MDSAL-196] In-Memory Data Store multi-sharding - controller dies when a large data set pushed into the data store Created: 29/Aug/16  Updated: 09/Mar/18  Resolved: 30/Aug/16

Status: Resolved
Project: mdsal
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug
Reporter: Jan Medved Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: All
Platform: All


External issue ID: 6570

 Description   

In the IMDS multi-sharding setup, the controller dies when a large data set is pushed into the data store. For example, pushing 1 million entries (a list of strings) into a single shard kills the controller. Pushing 500k entries per-shard into two shards kills the controller. But, pushing 250k entries per-shard into 4 shards works. By "kills the controller" i mean the CPU util goes to 100% and the controller stops responding.

To reproduce:
1. build coretutorials/clustering/shardingsimple in master
2. Start the controller build in Step 1 (./karaf/target/assembly/bin/karaf)
3. Point the browser to the controller (e.g. http://localhost:8181/index.html)
4. Go to Yangman on the selection menu on the left
5. Select the shardingsimple -> operations model
6. Enter the following JSON for "sent data:
{
"shard-test": {
"input":

{ "test-type": "ROUND-ROBIN", "data-store": "CONFIG", "shards": "2", "dataItems": "500000", "putsPerTx": "1", "listeners": "1", "precreate-data": "true", "validate-data": "false" }

}
}



 Comments   
Comment by Tomas Cere [ 30/Aug/16 ]

Karaf runs out of memory and the jvm is stuck in a permanent garbage collection, with this many entries you need to configure more heap for karaf. With 4G this seems to work fine.

Generated at Wed Feb 07 20:08:58 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.