From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id KAA26054; Thu, 1 Jul 2004 10:44:10 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id KAA25838 for ; Thu, 1 Jul 2004 10:44:09 +0200 (MET DST) Received: from kurims.kurims.kyoto-u.ac.jp (kurims.kurims.kyoto-u.ac.jp [130.54.16.1]) by concorde.inria.fr (8.12.10/8.12.10) with ESMTP id i618i6SH029172 for ; Thu, 1 Jul 2004 10:44:07 +0200 Received: from localhost (suiren.kurims.kyoto-u.ac.jp [130.54.16.25]) by kurims.kurims.kyoto-u.ac.jp (8.9.3p2-20030924/3.7W) with ESMTP id RAA15034; Thu, 1 Jul 2004 17:43:59 +0900 (JST) Date: Thu, 01 Jul 2004 17:43:58 +0900 (JST) Message-Id: <20040701.174358.116814367.garrigue@kurims.kyoto-u.ac.jp> To: kybic@fel.cvut.cz Cc: caml-list@pauillac.inria.fr Subject: Re: [Caml-list] Q: automatic forgetting cache, module Weak, Gc control From: Jacques GARRIGUE In-Reply-To: References: X-Mailer: Mew version 4.0.64 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Miltered: at concorde with ID 40E3CED6.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Loop: caml-list@inria.fr X-Spam: no; 0.00; caml-list:01 jacques:01 caching:01 memoizing:01 hash:01 retrieve:99 hashtables:01 hash:01 implemented:01 tweak:01 memoized:01 jacques:01 ocaml:01 garbage:01 garbage:01 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk From: Jan Kybic > > I am implementing a complicated numerical computation > algorithm in OCaml. (Basically, it is a Fast Mulipole Method > accceleration for a Boundary Element Method.) There is a lot of > intermediate results of several different types that are expensive to > calculate but that are used repeatedly in subsequent > calculations. Therefore, I am caching (memoizing) these intermediate > results using a hash table, like this: [...] > This works fine for smaller problems and the calculation is much > accelerated. However, when the problem gets big and the amount of > memory needed to store the intermediate results exceeds the amount of > available RAM, the operating system (Linux) starts to swap the cache > memory to disk. At this point, it is no longer advantageous to cache > the values, it is actually faster to recalculate them then to retrieve > them from the disk. This seems a typical work for weak hashtables. > 4) An ideal way would seem to be using the Weak module and the weak hash > table implemented there. Will that work as expected? From reading > the documentation I have the impression that the garbage collection > will probably collect the weak values too early, at first minor > collection. Is it correct? Is there a way to tune the garbage > collector to leave the weak values alone as long as the total > memory usage is below a certain threshold? I don't think you can tweak it, but at least it seems that only weak values inside the old generation are collected, i.e. only with the major garbage collector. I don't know whether this is a feature. In practice this should mean that your memoized values would stay in memory long enough to be useful, and that you don't have to care about memory management. If the above property fails, then you might need another mechanism. For instance you could check Gc.quick_stat regularly to see if you've got a major collection. Jacques Garrigue ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners