Re: [Caml-list] Q: automatic forgetting cache, module Weak, Gc control

[Jacques GARRIGUE <garrigue@kurims.kyoto-u.ac.jp>]

From: Jan Kybic <kybic@fel.cvut.cz>
> 
>         I am implementing a complicated numerical computation
> algorithm in OCaml. (Basically, it is a Fast Mulipole Method
> accceleration for a Boundary Element Method.) There is a lot of
> intermediate results of several different types that are expensive to
> calculate but that are used repeatedly in subsequent
> calculations. Therefore, I am caching (memoizing) these intermediate
> results using a hash table, like this:
[...]
> This works fine for smaller problems and the calculation is much
> accelerated. However, when the problem gets big and the amount of
> memory needed to store the intermediate results exceeds the amount of
> available RAM, the operating system (Linux) starts to swap the cache
> memory to disk. At this point, it is no longer advantageous to cache
> the values, it is actually faster to recalculate them then to retrieve
> them from the disk. 

This seems a typical work for weak hashtables.

> 4) An ideal way would seem to be using the Weak module and the weak hash
>    table implemented there. Will that work as expected? From reading
>    the documentation I have the impression that the garbage collection
>    will probably collect the weak values too early, at first minor
>    collection. Is it correct? Is there a way to tune the garbage
>    collector to leave the weak values alone as long as the total
>    memory usage is below a certain threshold?

I don't think you can tweak it, but at least it seems that only weak
values inside the old generation are collected, i.e. only with the
major garbage collector. I don't know whether this is a feature.

In practice this should mean that your memoized values would stay in
memory long enough to be useful, and that you don't have to care about
memory management. If the above property fails, then you might need
another mechanism. For instance you could check Gc.quick_stat
regularly to see if you've got a major collection.

Jacques Garrigue

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners