C binding and GC interaction: storing values outside the heap

[oleg@okmij.org]

Paolo Donadeo wrote: 
> The problem is that, for several good reasons, I need a copy, or a
> reference, to the OCaml value representing the lua_State *inside* the
> Lua state (I mean the C data structure).

Data structures with the mixture of OCaml values and non-OCaml
(unboxed) values are needed indeed, it seems. I had to emulate these
data structures since the captured native-code delimited continuation,
a part of the C stack, is exactly such mixed value. Leaking memory
can not be tolerated since continuations could be captured at high
rate (the delimcc distribution has several tests for that).

The solution was a custom GC scanning function. The mixed value is
allocated off the OCaml heap. All allocated values are linked.  The
pointer to the allocated value is wrapped into a OCaml custom block,
with a finalizer, which unlinks the value being finalized. The links
are not NOT heap pointers; they are not known to GC and so do not
prevent finalization. The tricky part is registering a GC scan
call-back. GC invokes the registered call-backs at the end.  The
call-back should go through all linked mixed values, and apply the GC
scanning action (passed as the argument to the callback) to all OCaml
values within the mixed values.

This solution is inspired by the implementation of thread contexts
in the native thread implementation, see the file
	otherlibs/systhreads/posix.c
in the OCaml distribution. The delimcc implementation is in the file
stacks-native.c of the delimcc distribution. 

The solution does not seem to be very efficient; it would be great if
OCaml supported mixed values directly (that is, permitted a custom
block with a custom GC scanning function). When GC scans such a custom
value, it invokes the user-provided function like a GC call-back.
The custom GC callback should know which parts of the mixed value are
OCaml values; it would do the GC action on those values.