Problems spawning threads

Problems spawning threads

[Edgar Friendly]

I've gone over the Computer Language Benchmarks Game[1] problems, and I
can't figure out how help the administrators of this problem to get the
OCaml code for the thread-ring benchmark (contributed by Charles Martin)
to work on their computer.

Here's the code:

let size = 503

let n = int_of_string Sys.argv.(1)

let run id ichan ochan =
  let rec loop () =
    let token = Event.sync (Event.receive ichan) in
    if token = n then (print_int id; print_newline (); exit 0)
    else (Event.sync (Event.send ochan (token + 1)); loop ())
  in Thread.create loop ()

let () =
  let channels =
    Array.init size
      (fun _ -> Event.new_channel ()) in
  let threads =
    Array.init size
      (fun i -> run (i + 1) channels.(i) channels.((i + 1) mod size)) in
  Event.sync (Event.send channels.(0) 0);
  Thread.join threads.(0)


Here's the compilation and execution:

~/tmp/testzone $ /usr/bin/ocamlopt -thread -noassert -unsafe
-I /usr/lib/ocaml/contrib -ccopt -O3 -ccopt -fomit-frame-pointer
-ccopt -march=pentium4 unix.cmxa threads.cmxa threadring.ml -o
threadring.ocaml_run

~/tmp/testzone $ ./threadring.ocaml_run 100
Fatal error: exception Sys_error("Thread.create: Cannot
allocate memory")


The code runs fine when compiled with ocamlc -vmthread ....  It works
fine on my Athlon with the same compilation options (except for
-march=athlon-tbird).

Interestingly enough, the administrator reports that the code works with
size=16, but fails for size=17.  Any ideas what's going on or how to fix
it?  Hopefully I'll have some confirmation soon that the (probably
useless) ccopts and other optimization options don't affect the error.

We also checked ulimit settings - his max user processes and stack size
are both higher than mine. (-u 4095, -s 160000k)  Any other ulimit
options that could interfere?  Last piece of information: his box runs
Gentoo Linux.

E.

Re: Problems spawning threads

[Edgar Friendly]

Edgar Friendly wrote:
> Hopefully I'll have some confirmation soon that the (probably
> useless) ccopts and other optimization options don't affect the error.

~/tmp/testzone $ ocamlopt -thread unix.cmxa threads.cmxa threadring.ml
-o threadring.ocaml_run

~/tmp/testzone $ ./threadring.ocaml_run 10
Fatal error: exception Sys_error("Thread.create: Cannot
allocate memory")


Still broken with simplest options for compilation.  :(

E.

Re: [Caml-list] Problems spawning threads

[Michael Wohlwend]

Am Dienstag, 18. Dezember 2007 06:18:34 schrieb Edgar Friendly:

> it?  Hopefully I'll have some confirmation soon that the (probably
> useless) ccopts and other optimization options don't affect the error.

the ccopts are for the c-compiler not for ocamlopt (it doesn't generate 
c-code)

 Michael

Re: Problems spawning threads

[Zheng Li]

Hi,

Edgar Friendly <thelema314@gmail.com> writes:
> Interestingly enough, the administrator reports that the code works with
> size=16, but fails for size=17.  Any ideas what's going on or how to fix
> it?  Hopefully I'll have some confirmation soon that the (probably
> useless) ccopts and other optimization options don't affect the error.

OCaml native code compiler uses system thread which is heavy, so you
shouldn't expect a large number of them running in parallel. In my box,
300+ threads will usually run out of virtual memory.

n=16 works and n=17 doesn't, probably means the n=16 version finishes
earlier so you haven't got chance to create enough threads to eat up
your memory, while n=17 version last longer so that the limit is
reached. You may try to add some printf to the thread creation function
to monitor the upper bound.

I don't know any workaround, in the absence of VM threads being able to
compile to native code (I asked a question on the topic once in the
beginner's list). Using user-land cooperative threads is probably
considered as cheating here.

-- 
Zheng Li
http://www.pps.jussieu.fr/~li

Re: [Caml-list] Re: Problems spawning threads

[Edgar Friendly]

Zheng Li wrote:
> Hi,
> 
> Edgar Friendly <thelema314@gmail.com> writes:
>> Interestingly enough, the administrator reports that the code works with
>> size=16, but fails for size=17.  Any ideas what's going on or how to fix
>> it?  Hopefully I'll have some confirmation soon that the (probably
>> useless) ccopts and other optimization options don't affect the error.
> 
> OCaml native code compiler uses system thread which is heavy, so you
> shouldn't expect a large number of them running in parallel. In my box,
> 300+ threads will usually run out of virtual memory.
> 
My laptop does a just fine job making and using 503 native threads with
512M ram (and lots of big programs already filling that up).  I don't
believe that their (likely beefier) benchmarking system hits a VM limit
mine doesn't.

> n=16 works and n=17 doesn't, probably means the n=16 version finishes
> earlier so you haven't got chance to create enough threads to eat up
> your memory, while n=17 version last longer so that the limit is
> reached. You may try to add some printf to the thread creation function
> to monitor the upper bound.
> 
The code should create 503 threads and 503 channels hooked together in a
ring, and *then* it starts a message (a simple counter) going around the
ring until it's counted high enough (normal test goes to 20,000,000
iirc), and then the last task to get the message prints its position.
None of the threads finish while thread creation continues.

E.

Re: Problems spawning threads

[Zheng Li]

Hi

Edgar Friendly <thelema314@gmail.com> writes:
> My laptop does a just fine job making and using 503 native threads with
> 512M ram (and lots of big programs already filling that up).  I don't
> believe that their (likely beefier) benchmarking system hits a VM limit
> mine doesn't.
Just made a test, the upper bound of my machine is 381 threads (without
event sync, just start threads and hold). You probably have to wrap some
C function to set pthread's stack size to ~ PTHREAD_STACK_MIN.

> The code should create 503 threads and 503 channels hooked together in a
> ring, and *then* it starts a message (a simple counter) going around the
> ring until it's counted high enough (normal test goes to 20,000,000
> iirc), and then the last task to get the message prints its position.
> None of the threads finish while thread creation continues.
My mistake.

-- 
Zheng Li
http://www.pps.jussieu.fr/~li

Re: [Caml-list] Re: Problems spawning threads

[Edgar Friendly]

Zheng Li wrote:
> Hi
> 
> Edgar Friendly <thelema314@gmail.com> writes:
>> My laptop does a just fine job making and using 503 native threads with
>> 512M ram (and lots of big programs already filling that up).  I don't
>> believe that their (likely beefier) benchmarking system hits a VM limit
>> mine doesn't.
> Just made a test, the upper bound of my machine is 381 threads (without
> event sync, just start threads and hold). You probably have to wrap some
> C function to set pthread's stack size to ~ PTHREAD_STACK_MIN.
> 
Okay, now things get really wierd...  I use a Makefile to build and
execute the test benchmark, and I run make from Emacs using tuareg's C-c
C-c macro.  When I do so, I have no problems running the test - it just
takes a while to complete.  But if I run the same make command from my
bash prompt inside gnome-terminal, I get the "Thread.create: Cannot
allocate memory" error...  hmmm...  Maybe there really is something to
this ulimit deal...


[root@iesyou shootout]# ulimit -s 1000
[root@iesyou shootout]# ./threadring.ocaml_run.opt 10000
444
[root@iesyou shootout]# ulimit -s 10000
[root@iesyou shootout]# ./threadring.ocaml_run.opt 10000
Fatal error: exception Sys_error("Thread.create: Cannot allocate memory")

I wonder what stack size gets inherited from emacs...  Anyway, this
looks like confirmation of Xavier's theory.  Thanks everyone.

E.

Re: [Caml-list] Problems spawning threads

[Xavier Leroy]

> Interestingly enough, the administrator reports that the code works with
> size=16, but fails for size=17.  Any ideas what's going on or how to fix
> it?  Hopefully I'll have some confirmation soon that the (probably
> useless) ccopts and other optimization options don't affect the error.
>
> We also checked ulimit settings - his max user processes and stack size
> are both higher than mine. (-u 4095, -s 160000k)

Such big stacks could be the source of the problem.  Assuming the
underlying POSIX thread implementation uses the "s" limit to determine
how much virtual memory space to allocate for the stack of each thread
(not an unreasonable assumption), 16 threads would eat up 2.5 Gb of
virtual address space, dangerously close to what's typically available
on a 32-bit architecture.

For this benchmark, you want much smaller stacks.  Or better yet, run
it with "ocamlc -vmthread": for silly benchmarks like this, it will
run much faster than the ocamlopt-generated code that uses POSIX
threads.

FYI, my 64-bit Linux installation has no problems with size = 10000...

- Xavier Leroy

Re: [Caml-list] Problems spawning threads

[Jon Harrop]

On Tuesday 18 December 2007 05:18, Edgar Friendly wrote:
> I've gone over the Computer Language Benchmarks Game[1] problems, and I
> can't figure out how help the administrators of this problem to get the
> OCaml code for the thread-ring benchmark (contributed by Charles Martin)
> to work on their computer.

This benchmark is subjective, ill-defined and trivially reducible. 
Consequently, your competitors aren't using real system threads but the 
shootout maintainers will probably reject your submission if you don't. You 
could pull in a more suitable "thread" library like LWT:

  http://www.ocsigen.org/lwt

However, once you pull in that code and write the benchmark you'll be able to 
trivially reduce it until it is just a "print" statement. At some point the 
shootout maintainers will choose to start rejecting your submissions and call 
you a "cheat" for writing better code.

The same is also true of binary-trees, chameneos-redux, mandelbrot, nsieve, 
nsieve-bits, partial-sums, pidigits and recursive.

In summary, this is a complete waste of time and effort. I would love to see 
simple tutorial examples of threading, concurrency and parallelism in OCaml. 
If you decide to embark on such a project, please choose examples that are 
objective, relevant and not trivially reducible.

I would be very interested to see parallel implementations of 
the "n"th-nearest neighbour example from my book, for example. I have been 
trying and failing to write an elegant functional implementation of that 
myself for some time now... :-)

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e