Re: [Caml-list] Netmcore problems.

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Freitag, den 13.01.2012, 17:30 +0100 schrieb Anders Fugmann:
> Hi,
> 
> I'm having a problem using netmcore from ocamlnet 3.4 library.
> 
> When I try to place a large data structure into Netmcore_array, the 
> controlling process dies with the following error:
> 
> [Fri Jan 13 16:21:38 2012] [netplex.controller] [alert] Process 28067 
> for service netmcore_0 terminated with signal 7

That's sigbus. You get it when the OS does not have memory anymore - but
not when the memory is allocated, but first when it is filled with data.
You can change this by disallowing that memory is overcommitted (set the
kernel param vm.overcommit_memory to 1 or even 2).

> The error occurs while trying to initialise a Netmcore_array from an 
> array of 4*10^6 strings all of length 150. This is done in the context 
> of the "first" process.
> 
> The controlling process initializes a memory pool by using
> 
>    let pool_id = Netmcore_mempool.create_mempool (1024 * 1024 * 1024 * 
> 10) in
> 
> Running the code on a smaller dataset seems to work.
> 
> I have increased kernel variables shmall and shmmax to very high numbers 
> (100Gb), but it does not solve the problem.

No, this cannot solve it. These variables only control System V shared
memory, but Netmcore uses POSIX shared memory. On Linux, you can change
the max of this memory by re-mounting /dev/shm, e.g.

mount -o remount,size=80% /dev/shm

The default is 50% of available RAM.

> All ideas are welcome.
> 
> On other question while I'm at it; am I allowed to create multiple 
> shared arrays from the same memory pool, or do I need to create one pool 
> for each shared array?

The pools can be shared.

Gerd

> 
> Regards
> Anders Fugmann
> 
> 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.
------------------------------------------------------------