[DAEXIM-14] Add an option to trigger a CDS snapshot after import Created: 12/Sep/19 Updated: 08/Dec/23 |
|
| Status: | Confirmed |
| Project: | daexim |
| Component/s: | General |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Medium |
| Reporter: | Robert Varga | Assignee: | Tibor Král |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
A daexim import really means overwriting the contents of the datastore, which is a large operation, which ends up being stored in journal. Doing multiple consecutive imports can cause the journal to grow, consuming both on-disk (leveldb file) space and in-memory (heap) space. Add an option whereby after a successful import the CDS leader is instructed to take a snapshot, hence purging the now-stale journal entries – controlling consumption of both resources. |
| Comments |
| Comment by Anton Ivanov [ 08/Oct/19 ] |
|
Full import in most databases is done with integrity and transaction support off during import and some (optional) checks at the end when the indexes are being rebuilt. IMHO taking the snapshot after import is treating the symptom instead of fixing the underlying issue that import is pumping all the data through the normal transaction engine and there is no way to turn on/off appropriate import knobs in it. |