From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <wojciech.meyer@gmail.com>
Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83])
	by sympa.inria.fr (Postfix) with ESMTPS id 005C7820A1
	for <caml-list@sympa.inria.fr>; Sat,  7 Sep 2013 21:13:21 +0200 (CEST)
Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender
  authenticity information available from domain of
  wojciech.meyer@gmail.com) identity=pra;
  client-ip=209.85.215.173;
  receiver=mail2-smtp-roc.national.inria.fr;
  envelope-from="wojciech.meyer@gmail.com";
  x-sender="wojciech.meyer@gmail.com";
  x-conformance=sidf_compatible
Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of
  wojciech.meyer@gmail.com designates 209.85.215.173 as
  permitted sender) identity=mailfrom;
  client-ip=209.85.215.173;
  receiver=mail2-smtp-roc.national.inria.fr;
  envelope-from="wojciech.meyer@gmail.com";
  x-sender="wojciech.meyer@gmail.com";
  x-conformance=sidf_compatible; x-record-type="v=spf1"
Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender
  authenticity information available from domain of
  postmaster@mail-ea0-f173.google.com) identity=helo;
  client-ip=209.85.215.173;
  receiver=mail2-smtp-roc.national.inria.fr;
  envelope-from="wojciech.meyer@gmail.com";
  x-sender="postmaster@mail-ea0-f173.google.com";
  x-conformance=sidf_compatible
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Am4BAGF5K1LRVdetk2dsb2JhbABAGoM/wlmBJRYOAQEBAQcLCwkUBCSCJQEBBAFAARsdAQMMBgULFhMSDwEEDxEBBQEiExuHVAEDCQYECDSmEYxRgwWDZAoZJw1kiBEBBQyPdAeEHQOXdYEvjk5BhEk
X-IPAS-Result: Am4BAGF5K1LRVdetk2dsb2JhbABAGoM/wlmBJRYOAQEBAQcLCwkUBCSCJQEBBAFAARsdAQMMBgULFhMSDwEEDxEBBQEiExuHVAEDCQYECDSmEYxRgwWDZAoZJw1kiBEBBQyPdAeEHQOXdYEvjk5BhEk
X-IronPort-AV: E=Sophos;i="4.90,860,1371074400"; 
   d="scan'208";a="31921407"
Received: from mail-ea0-f173.google.com ([209.85.215.173])
  by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 07 Sep 2013 21:13:21 +0200
Received: by mail-ea0-f173.google.com with SMTP id g10so2302306eak.4
        for <multiple recipients>; Sat, 07 Sep 2013 12:13:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=from:to:cc:subject:references:date:in-reply-to:message-id
         :user-agent:mime-version:content-type;
        bh=o7OS3+yLQoNomWXbZ4c3SDW63D2whWb9t0VLpnJccjk=;
        b=0kaQ3xOdVOw9heDNn5RvljlLmicpmRl9FYISzRxyuwG/H6bmQLPDnVu6B4zaPWupaT
         urBR7lUmentVfLyaOkoJf6ZARYdWh4L9IrZ7c9X+fCm4GoYQ4WetKeRrasLBOHHLTftl
         Zs/EjNStSqq+APD07E8JhE322ZRSlfBEYoaQkFts7h/VokcAwc23xd/CyU2LnOp2Sz2a
         PLx0L5Dgb20LDMyjlUExbHfPCUQrkLbmi3z8mGl458P10/Ncz9z1D/6KdNpnHQKhog5F
         TaPqiXp4InFHgQcG70LPD0lx+kYMYAViVVN94f1Ix428xrItUkDLz7Fl/wnlmzdO5nXt
         YVEw==
X-Received: by 10.14.172.133 with SMTP id t5mr15074986eel.35.1378581201104;
        Sat, 07 Sep 2013 12:13:21 -0700 (PDT)
Received: from localhost.danmey.org (89-77-178-93.dynamic.chello.pl. [89.77.178.93])
        by mx.google.com with ESMTPSA id z12sm7435013eev.6.1969.12.31.16.00.00
        (version=TLSv1.2 cipher=RC4-SHA bits=128/128);
        Sat, 07 Sep 2013 12:13:19 -0700 (PDT)
From: Wojciech Meyer <wojciech.meyer@gmail.com>
To: Xavier Leroy <Xavier.Leroy@inria.fr>
Cc: caml-list@inria.fr
References: <5229DEF9.7040706@inria.fr> <522A0336.7030203@inria.fr>
Date: Sat, 07 Sep 2013 20:13:17 +0100
In-Reply-To: <522A0336.7030203@inria.fr> (Xavier Leroy's message of "Fri, 06
	Sep 2013 18:30:46 +0200")
Message-ID: <87hadw1m42.fsf@gmail.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Subject: Re: [Caml-list] Accelerating compilation

Xavier Leroy <Xavier.Leroy@inria.fr> writes:

>> 3) Parallel compilation in Ocamlbuild
>> Of course it would help but it is not easy to implement so I'm just
>> putting it there to be exhaustive.
>
> As others said, ocamlbuild has some support for parallelism, but
> it's much less effective than "make -j".  One reason could be that
> ocamlbuild discovers dependencies as it goes, so it's doing online
> scheduling, in contrast with make's offline scheduling.  But within
> the online scheduling approach, there is certainly room for
> improvement.  Suggestions and patches welcome.

That's exactly what happens and is described here:

  http://caml.inria.fr/mantis/view.php?id=5754

Unfortunately it's not that easy to fix. Possible solutions:

- do the scheduling after the project was build for the first time and
  then:
  * for each of the targets note the products and store it in the
    database
  * when the target changes, re-build the database for this target and
    it's products

- provide simpler and less dynamic rules for ocamlbuild:
  * that actually don't run the Caml code, but do only invoke shell
    commands with some substitution
  * try to convert existing rules to the static ones

Both solutions guarantee perfect scheduling.

Wojciech