Re: [Caml-list] CamlMPI: sending marshalled objects

["Olivier Andrieu" <oandrieu@nerim.net>]

Hi,

On Jan 22, 2008 7:52 PM, Thomas Fischbacher <tf@functionality.de> wrote:
>
> Hello everybody,
>
> there is an old Caml/MPI interface, written by Xavier himself in 1998.
> This provides a pair of C functions caml_mpi_send/caml_mpi_receive,
> which may look a bit funny considering present day ML-C-interface coding
> conventions, but GC-wise is actually valid code as far as I can judge.
> For reference, I added them at the end of this email.

I've been using this interface some time ago (in 2006). I rewrote it a
bit to bring it up-to-date with the current conventions.

> As far as I can see, output_value_to_malloc() actually should be able to
> serialize virtually any ML object (which does not contain alloc_custom()
> blocks and similar stuff) to a buffer.

some custom blocks such as Bigarrays can be serialised too.

> Sending this over the net also
> should not be much of a problem, regardless of the data being sent. If,
> on the other hand, I manually serialized to a string and tried to send
> that, I would be bound by the 16 MB maximal string length limitation on
> 32-bit architectures.

right.

> So much the theory. But what I actually find in reality is that -- even
> using caml_mpi_send/caml_mpi_receive rather than intermediate strings --
> my code crashes when I try to pass around serialized ML data over MPI
> which is larger than 16 MB. I am not yet 100% sure that this really is
> the problem, but at present, I have every reason to believe so.
>
> So, (1) what is going on here, and (2) how can I repair it?
> Is there a quick fix?

I'm not sure what is going on, I was able to transmit large marshalled
values IIRC.

One serious problem in the current code is the treatment of errors.
Ocamlmpi registers an error handler that raises an OCaml exception.
However, should an error occur during MPI_Send, the handler is invoked
with the global mutex released since MPI_Send is between
enter/leave_blocking_section. Maybe that's what causing the crash.

-- 
  Olivier