Details
-
Improvement
-
Status: Resolved
-
High
-
Resolution: Done
-
None
Description
Heap analysis shows that io.atomix.storage.journal.FileChannelJournalSegmentWriter ends up allocating HeapByteBuffers with twice the size of maximum entry size. In our default full install, this means ODL ends up eating 306MiB of heap off the bat.
We also have a problem where the maximum entry size is limited, whereas we can have rather huge entries stored in the journal.
Introduce a revised journal version, which will seamlessly migrate when attached, so that we perform entry chunking – i.e. lower the maximum entry to something around 128KiB and split larger Akka journal entries into multiple file entries.
Attachments
| # | Subject | Branch | Project | Status | CR | V |
|---|---|---|---|---|---|---|
| 91718,14 | [WIP] Add entry fragmentation to segmented journal | master | controller | Status: ABANDONED | 0 | +1 |
| 92615,7 | Add a segmented DataJournal abstraction | master | controller | Status: MERGED | +2 | +1 |
| 92616,18 | WIP: Add DataJournalV1 | master | controller | Status: ABANDONED | 0 | -1 |
| 93097,4 | Update DataJournal interface | master | controller | Status: MERGED | +2 | +1 |
| 93099,5 | Rehost Chunked{ByteArray,InputStream,OutputStream} | master | controller | Status: MERGED | +2 | +1 |
| 93280,1 | Update DataJournal interface | 2.0.x | controller | Status: MERGED | +2 | +1 |
| 95328,4 | Use memory-mapped segmented journal by default | master | controller | Status: MERGED | +2 | +1 |
| 96014,3 | Use memory-mapped segmented journal by default | 3.0.x | controller | Status: MERGED | +2 | +1 |