[CONTROLLER-839] Clustering : Snapshotting uses up too much memory Created: 15/Sep/14  Updated: 06/Jan/15  Resolved: 06/Jan/15

Status: Resolved
Project: controller
Component/s: mdsal
Affects Version/s: Helium
Fix Version/s: None

Type: Bug
Reporter: Moiz Raja Assignee: Prem Sankar
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Operating System: Mac OS
Platform: PC

Issue Links:
Blocks
is blocked by CONTROLLER-874 Clustering: Improve CDS serialization... Resolved
is blocked by CONTROLLER-897 Clustering : Create a NormalizedNode ... Resolved
External issue ID: 1884
Priority: Normal

 Description   

Snapshotting the in-memory datastore within a shard can lead to out of memory when the in-memory datastore grows really large.

For example with 4GB memory allocated adding about 180K Flows into the config datastore exhausts all the memory.

From analysis it seems like the act of creating a snapshot is what exhausts the memory instead if retaining the bytes for the snapshot itself in memory. A snapshot is about a 10th of the size of the in-memory datastore.

To reproduce,

To reproduce,

1. Build the OpenFlowPlugin Karaf distribution
2. Run the controller with the following options
JAVA_MAX_MEM=4G JAVA_MAX_PERM_MEM=512m ./karaf
3. Install the odl-mdsal-clustering feature, also install jolokia
4. Run the attached script with the following options
/usr/local/bin/python flow_config_perf.py --nflows 500 --nthreads=5 --ncycles 60 --no-delete

 Comments   
Comment by Moiz Raja [ 14/Oct/14 ]

This will probably be fixed by the Stream Reader/Writer (see CONTROLLER-897). So hold off on working on this one.

Comment by Moiz Raja [ 06/Jan/15 ]

We do not store snapshots in memory

Generated at Wed Feb 07 19:54:00 UTC 2024 using Jira 8.20.10#820010-sha1:ace47f9899e9ee25d7157d59aa17ab06aee30d3d.