From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=AWL,HTML_MESSAGE autolearn=disabled version=3.1.3 Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by yquem.inria.fr (Postfix) with ESMTP id 44E14BBCA for ; Sat, 10 May 2008 01:01:07 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtcCAAN1JEjRVcbsc2dsb2JhbACCNTaPGwEMAwQECQ8Fk2aFNg X-IronPort-AV: E=Sophos;i="4.27,462,1204498800"; d="scan'208";a="26015643" Received: from rv-out-0506.google.com ([209.85.198.236]) by mail4-smtp-sop.national.inria.fr with ESMTP; 10 May 2008 01:01:05 +0200 Received: by rv-out-0506.google.com with SMTP id f6so1565223rvb.3 for ; Fri, 09 May 2008 16:01:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; bh=5dfsx1OSO3sc81RrUS9P/sjeDpTPT81h/8TCaKy/Wbs=; b=varzArxCtmRGjF/P3iQHw995EgdMiMQrUqt0b3WETJVVuqmU1IcSvDekTvHtacTtYsdY1EYMDtfeCEYAdNRBItCOyXADr9bqdNu/WtDN+d8noyt5Xyi/IFbGWjzL/wEpfNbeN1OaiIe12t8WDd/XAqkq+5liZ9tZzYOcdvz+aN4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=spWrebmXxhIwCxghFdYJo5SkCcJX6t0uPTgwMn5o0oql0WLZsRH92bAWUVLhRm3kcC8AV1VrUXk4znQT/XhVQbT/UQpc2Z3MPF+C72c7wdUL9olBSn6ZQFza7/8JPdQOKeYDvaBzNkZqazoxnaiDxn0znQiaA81GQIXuOX+MtCw= Received: by 10.141.211.13 with SMTP id n13mr2413516rvq.184.1210374063930; Fri, 09 May 2008 16:01:03 -0700 (PDT) Received: by 10.140.193.3 with HTTP; Fri, 9 May 2008 16:01:03 -0700 (PDT) Message-ID: Date: Sat, 10 May 2008 01:01:03 +0200 From: "Berke Durak" To: "Richard Jones" Subject: Re: [Caml-list] Re: Why OCaml rocks Cc: "Till Varoquaux" , caml-list@yquem.inria.fr In-Reply-To: <20080509222603.GA21613@annexia.org> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5557_23960175.1210374063933" References: <200805090139.54870.jon@ffconsultancy.com> <200805090609.36123.jon@ffconsultancy.com> <1210331526.17578.32.camel@flake.lan.gerd-stolpmann.de> <200805091910.41381.jon@ffconsultancy.com> <1210365645.17578.88.camel@flake.lan.gerd-stolpmann.de> <9d3ec8300805091400q1ed60bf8x95e31814ebf62473@mail.gmail.com> <20080509222603.GA21613@annexia.org> X-Spam: no; 0.00; berke:01 durak:01 berke:01 durak:01 ocaml:01 0200,:01 bigarray:01 extern:01 parallelism:01 ad-hoc:01 0200,:01 bigarray:01 extern:01 parallelism:01 ad-hoc:01 X-Attachments: cset="UTF-8" cset="UTF-8" ------=_Part_5557_23960175.1210374063933 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Sat, May 10, 2008 at 12:26 AM, Richard Jones wrote: > On Fri, May 09, 2008 at 11:13:26PM +0200, Berke Durak wrote: > > - For sharing complex data, you can marshall into a shared Bigarray. > > > > If the speed of Marshal becomes a bottleneck, a specialized Marshal that > > skips most of the checks/byte-oriented, compact serialization things that > > extern.c currently does could speed > > things up. > > At the risk of sounding like a broken record ... OR you could use > Ancient which does the above, right. > I just looked at the interface and in spite of an initial difference in aims (sharing large, static data between processes), Ancient seems to be almost what's needed for parallelism. It could be used as an IPC mailbox, provided you map /dev/shm, come up with some cheap means of synchronizing your processes (possibly by using a futex?) and ensure that the creation/deletion functions do not have too much overhead. But you are saying in the README that values in the ancient heap have some limitations, namely no ad-hoc polymorphic primitives. That may be acceptable for sharing a large static database but won't cut the mustard for cheap message- passing. So we need a function to copy back an ancient value in the heap so that it can be used normally, which is what I was talking about: a simplified, cheaper Marshal module. -- Berke ------=_Part_5557_23960175.1210374063933 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Sat, May 10, 2008 at 12:26 AM, Richard Jones <rich@annexia.org> wrote:
On Fri, May 09, 2008 at 11:13:26PM +0200, Berke Durak wrote:
> - For sharing complex data, you can marshall into a shared Bigarray.
>
> If the speed of Marshal becomes a bottleneck, a specialized Marshal that
> skips most of the checks/byte-oriented, compact serialization things that
> extern.c currently does could speed
> things up.

At the risk of sounding like a broken record ...  OR you could use
Ancient which does the above, right.

I just looked at the interface and in spite of an initial difference in aims
(sharing large, static data between processes), Ancient seems to be almost
what's needed for parallelism.

It could be used as an IPC mailbox, provided you map /dev/shm, come up with
some cheap means of synchronizing your processes (possibly by using a futex?)
and ensure that the creation/deletion functions do not have too much overhead.

But you are saying in the README that values in the ancient heap have some
limitations, namely no ad-hoc polymorphic primitives.  That may be acceptable
for sharing a large static database but won't cut the mustard for cheap message-
passing.  So we need a function to copy back an ancient value in the heap so that
it can be used normally, which is what I was talking about: a simplified, cheaper
Marshal module.
--
Berke

------=_Part_5557_23960175.1210374063933--