[Caml-list] Fixing remaining problems with dynamically freeing code

[Nuutti Kotivuori <naked+caml@naked.iki.fi>]

Well, like mentioned in the patch - the problems that remain in having
dynamically loaded code garbage collected mainly concentrate around
the global table - eg. Module requires, provides and literals.

And I think I have come up with a solution.

We create one more custom block type - and that's a global
reference. It takes a global table slot number on creation, and the
only thing it ever does is go place the unit value in the global slot
on finalization.

Then, we make Symtable.patch_object create an array from the custom
block that references the actual code - and custom blocks for every
global the code uses - and we place this array into every closure
created. This array must not reference the modules that are provided
by the code block though. Instead, we place the references for those
globals in the ident lookup table for modules.

Some ascii art on how it would work.

  ,----------------------------------------------------------------------.
  v                            Symtable.global_table                     |
 +---------+------+----------+ +-----+-------------------+-----+         |
 | Closure | Code | Field(1) | | ... | Ident.t, int, ref | ... |         |
 +---------+------+----------+ +-----+----------------|--+-----+         |
                        |                             |                  |
  ,---------------------'                             `----------.       |
  v                                                              |       |
 +-------+----------+----------+----------+----------+           |       |
 | Array | Field(1) | Field(2) | Field(3) | Field(4) |           |       |
 +-------+----------+----------+----------+----------+           |       |
               |          |          ¦          |                |       |
        ,------'       ,--'          |          `--.             |       |
        v              v             v             v             v       |
 +-------------+ +-----------+ +-----------+ +-----------+ +-----------+ |
 | Staticalloc | | Globalref | | Globalref | | Globalref | | Globalref | |
 +-------------+ +-----------+ +-----------+ +-----------+ +-----------+ |
 Custom | blocks       |             |             |             |       |
        |              |             |             |             |       |
  ,-----'              `------.      `----.        `--.          `.      |
  |                           |           |           |           |      |
  v                           v           v           v           v      |
 +--------------+ +-----+-----------+-----------+-----------+-----------+|
 | Code         | | ... | Field(16) | Field(17) | Field(18) | Field(19) ||
 +--------------+ +-----+-----------+-----------+-----------+-----------+|
                  Global data                                     `------'

So, how this would work in garbage collection then would be that if
neither the ident lookup table, nor any other module by depending,
used the global provided by the code, then that global would be set to
unit - which in turn would mean that if no other closures are
anywhere, the last references to the array would drop, which would
then again allow freeing the code and let the globals referenced by
the code to freed in turn.

This should address the freeing problem entirely, for both module
references and literals. Then some sort of a simple solution would
have to be made for reusing freed slots in the global table - a free
list will probably do.

Anyhow, this builds on an awful lot of assumptions, and there's still
quite a bunch of open questions on the implementation - but I am quite
confident that those will get solved and the implementation can be done.

And if so, it's again a rather nice solution - minimal changes to the
core, no impact at all for normal code, only memory management
overhead for dynamically loaded code. After this, loading a module one
million times, private or not, should leave behind no overhead what so
ever - and yet it is safe, as any references will keep the blocks
alive.

Merry Christmas,
-- Naked

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners