An observation on JournalIndex: we maintain one of these for each segment file. For StorageLevel.MAPPED we could easily persist the index in memory, given a structure proposed in CONTROLLER-2110.

That calls for storing a size and a linear array of 64bit entries (well, perhaps as low as 60bits, maybe less if we play 'maxEntrySize' assumptions on possible 'size' delta, but let's not go there).

Persisting an index in the file, subject to maxSegmentSize() would mean that we would have fewer maximum entries, as the effective size of an entry includes its index, hence goes (length, crc, bytes) to (position, length, crc, bytes), i.e. from 9 to 17 bytes for the smallest possible entry.

The easiest thing we can do is to store the index at the end of each file, as an array growing downwards (mental picture: data = heap, index = stack in traditional C memory management sense). Once data and index consume maxSegmentSize, we move to the next segment.