Re: [Caml-list] Proposal: re-design of ocaml headers

[Florian Weimer <fw@deneb.enyo.de>]

* Yotam Barnoy:

> The pfbits (as shown below) are an efficient way of representing both
> floats and pointers in a data structure, at the cost of disallowing random
> access. From what I can gather, random access to this data is never needed,
> since the GC, Marshal module, and polymorphic comparison all process the
> whole data structure rather than referring to specific parts of it.

I wonder if it could make sense to get rid of polymorphic comparison
and make Marshal type-safe before changing the header.  If the header
is needed for GC only and for telling variants apart, it would not be
necessary to discriminate between strings, floats and custom memory
blocks in the header.  That would give room for 28 bits for the
tag/size combination or the string or value array length, with 2 bits
used by the GC colors and 2 bits for the type bits (telling apart
non-scanned blobs, scanned arrays of values, scanned tagged variants,
and unscanned tagged variants).  Arrays are separate, so 28 bits
should be plenty of room for both tag and variant record length.

This way, it would be possible to encode both string and array lengths
directly in the header, speeding up bounds checks.  The GC would have
check the type bits to compute the actual object size from the length
in the header, but mutator performance seems more important, and
perhaps we can come up with a clever way to decode the length without
branching.  It might even make sense to store the actual number of
elements of a double array in the header, again to speed up bounds
checks.

If the 32-bit 256 MB string size limit is too onerous, it should be
possible to bump it to 512 MB, but that will complicate header
decoding further.  But it's possible to index only 1 GB, so 256 MB
is already pretty close to that hard limit.