Clustering : Snapshotting uses up too much memory

Snapshotting the in-memory datastore within a shard can lead to out of memory when the in-memory datastore grows really large.

For example with 4GB memory allocated adding about 180K Flows into the config datastore exhausts all the memory.

From analysis it seems like the act of creating a snapshot is what exhausts the memory instead if retaining the bytes for the snapshot itself in memory. A snapshot is about a 10th of the size of the in-memory datastore.

To reproduce,

To reproduce,

1. Build the OpenFlowPlugin Karaf distribution
2. Run the controller with the following options
JAVA_MAX_MEM=4G JAVA_MAX_PERM_MEM=512m ./karaf
3. Install the odl-mdsal-clustering feature, also install jolokia
4. Run the attached script with the following options
/usr/local/bin/python flow_config_perf.py --nflows 500 --nthreads=5 --ncycles 60 --no-delete