Re: [Caml-list] More re GC hanging

[Damien Doligez <damien.doligez@inria.fr>]

On 2010-08-15, at 12:45, Adrien wrote:

> First, remove all non-tail-rec functions: no more List.map, @ or
> List.concat. All lists were pretty short (definitely less than 1000
> elements) but maybe the amount of calls generated garbage or something
> like that: I couldn't get much infos about the problem so I tried what
> I could think of and being tail-rec couldn't be a bad thing anyway.
> The idea was to create less values since I first suspected a memory
> fragmentation issue (iirc I had thousands of fragments), so I also
> memoized some functions.

That has nothing to do with the GC getting huge counts.  Also, if you
have fragmentation problems, you can try changing the allocation
policy with this statement:

   Gc.set {(Gc.get ()) with Gc.allocation_policy = 1};;

I'm still waiting for feedback on that alternate allocation policy :-)

> Then, as Basile mentionned, call something like Gc.compact ()
> regularly. The overhead is actually pretty small, especially if ran
> regularly.

That's good for tracking down problems, but I wouldn't recommend it
for production code.

> Finally, C bindings. I created a few while not having access to the
> internet and they are quite dirty. I highly doubt they play perfectly
> well with the garbage collector: they seem ok but probably aren't
> perfect. That's definitely something to check, even if the bindings
> were written by someone else because working nicely with the GC can be
> quite hard.
> 
> Now, the problem seems to be gone but I can't say for sure. One really
> annoying thing was that adding a line like 'print_endline "pouet";'
> would make the out-of-memory problem go away! Same when getting stats
> from the GC.


That almost certainly indicates a problem with your C bindings: some
pointer gets garbled and the GC may or may not stumble upon it.

-- Damien