From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 4D656820A1 for ; Fri, 6 Sep 2013 17:27:56 +0200 (CEST) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.214.50; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.214.50 as permitted sender) identity=mailfrom; client-ip=209.85.214.50; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-bk0-f50.google.com) identity=helo; client-ip=209.85.214.50; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-bk0-f50.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AnIBAGPzKVLRVdYylGdsb2JhbABbgzxRrw+SRoEcCBYOAQEBAQcNCQkUBCSCJAEBBUABGxACCAMBAwwGBQsNDSEiAREBBQEKEgYTEoddAQMPDKBqjFCDBYQLChknAwpkiAwBBQyPPTMHhB0Dl3WBL45OGCmESjo X-IPAS-Result: AnIBAGPzKVLRVdYylGdsb2JhbABbgzxRrw+SRoEcCBYOAQEBAQcNCQkUBCSCJAEBBUABGxACCAMBAwwGBQsNDSEiAREBBQEKEgYTEoddAQMPDKBqjFCDBYQLChknAwpkiAwBBQyPPTMHhB0Dl3WBL45OGCmESjo X-IronPort-AV: E=Sophos;i="4.90,855,1371074400"; d="scan'208";a="25980904" Received: from mail-bk0-f50.google.com ([209.85.214.50]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 06 Sep 2013 17:27:55 +0200 Received: by mail-bk0-f50.google.com with SMTP id mz11so1345613bkb.23 for ; Fri, 06 Sep 2013 08:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=9TJLelM4U7+YpFXAnLUagz94G+UHS9+ZZYFeflfjJks=; b=cCcGxCtE0IEvhwNr9mJgSFYFoe4oyJNoTKy2W1RTLukbZGUuTdsIg45jOkQO5pPpSn IY9uUR8RWLJv5hZxmdB3iwzrKzqMKMbde15+0BHDhe7zaXM8XLeAmdgrmau89GIoEv83 HqOenzeBicu3SFXNXApQOpH2sqWMu/1aUALYygCmR3lEqvR+bRsvEUQjWvl3AqSfQvOT YHysQJBHpucaPtNAbgkByLRUetN+UAbx5D6w7rRjbcklDMuLJ7WjzMLO9+r7+Y/Phu9l UbWr9zbDD/kBRExwRkpMGwDTvLFT2kvoDXi0Gnjcs9H7mYNkFhtF6mtp4CewRlr8zQAG KOyw== X-Received: by 10.205.65.17 with SMTP id xk17mr2850601bkb.19.1378481274894; Fri, 06 Sep 2013 08:27:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.236.193 with HTTP; Fri, 6 Sep 2013 08:27:13 -0700 (PDT) In-Reply-To: <5229F284.5050806@inria.fr> References: <5229DEF9.7040706@inria.fr> <5229F284.5050806@inria.fr> From: Gabriel Scherer Date: Fri, 6 Sep 2013 17:27:13 +0200 Message-ID: To: Romain Bardou Cc: Markus Mottl , "caml-list@inria.fr" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Caml-list] Accelerating compilation One option that I think would be really help is a way to preserve separate compilation with ocamlopt, by not relying on .cmx of dependencies for optimization. That could preserve a good incremental development workflow and make most of those issues, I think, moot. (I also hear a suggestion that editors with incremental type feedback, such as Merlin and Typerex2, do help.) On Fri, Sep 6, 2013 at 5:19 PM, Romain Bardou wrot= e: > Le 06/09/2013 16:55, Markus Mottl a =E9crit : >> On Fri, Sep 6, 2013 at 9:56 AM, Romain Bardou w= rote: >>> 1) Separate typing and code generation, in ocamlc and in ocamlopt >>> >>> For instance, provide an option -typing-only which would mean "only >>> produce the .cmi but do not produce the .cmo or the .cmx". The compiler >>> would only need the .cmi of the dependencies, not their .cmo or .cmx. >>> This would make it possible to have a Makefile target, or an Ocamlbuild >>> option, to just type. >> >> This seems like an interesting suggestion. Code generation, >> especially for native code, can be quite expensive. I do use byte >> code compilation during development for that reason, which is somewhat >> faster. > > I considered doing that, but my project has some stubs which do not work > in bytecode for some reason (probably fixable). Also, it would mean that > I would have to compile both versions, as the program is too slow to be > used in bytecode, so the bytecode would only be used for quick > type-checking. > >> Note, however, that there is one problem with the approach as >> suggested: if you have both an .mli and an .ml file, the build system >> would have to know when to "type" the latter. In the suggested >> approach there would be no trace of this action, because the .cmi >> would come from the .mli file. We would need to generate a separate >> dummy file in that case to visibly record the fact that the .ml file >> unifies with the .cmi file. Or (see below) write out the typed >> abstract syntax tree of the .ml file, which would, of course, also >> have to unify with the .cmi file. > > Indeed the build system would need to be tweaked. Another approach would > be to consider that .cmi files depend on .ml files as well, maybe only > if an option -just-type is passed. I'm not sure of the implications. > >>> Also, provide an option -do-not-retype which would mean "if the .cmi >>> exists, load it instead of type-checking again". This would allow the >>> build process to first type-check (using -typing-only) and then generate >>> the code without type-checking again (using -do-not-retype). Of course >>> the build system should be very careful to ensure the .cmi is >>> up-to-date. This option could also help when compiling both in bytecode >>> and in native code. This option is not necessary to just find errors >>> quickly, though. >> >> The .cmi file does not contain enough information, because it only >> contains the signature. You need the typed AST for proper code >> generation, which might be quite big. I haven't looked into this, but >> I wouldn't be surprised if the size of that thing could be so large >> that you might prefer type checking again over writing to + reloading >> it from a file. > > Ah right I was thinking it contained the whole typed tree for some > reason, which is indeed not the case. > >>> 2) Be able to disable Ocamlbuild's digest mechanism and use dates and >>> file sizes instead >>> >>> If I am not mistaken, this is one of the main reasons why Ocamlbuild is >>> slower that make. It does help to prevent useless recompilation, but >>> what good does it make to prevent a useless recompilation once in a >>> while if it is at the cost of losing a lot of time in all other cases? >>> I'm sure it is project-dependent though so it should only be an option, >>> say, -do-not-hash, or -comparison-mode dateandsize. >> >> The problem here is that not all Unix-filesystems support sub-second >> resolution in timestamps. So if you build a file and change a >> character in it within one second, the change may go unnoticed, i.e. >> required recompilation doesn't happen. That's why hashing provides >> much stronger guarantees. But I think this is not a big problem in >> practice so I'd support such a flag in ocamlbuild. >> >>> 3) Parallel compilation in Ocamlbuild >>> >>> Of course it would help but it is not easy to implement so I'm just >>> putting it there to be exhaustive. >> >> I'm not sure what you are referring to, OCamlBuild does already >> support parallel builds. > > Does it? I actually thought the -j option was ignored. > > I just did a quick test and I gain about 5 seconds with -j on a 1min15 > build (I had cleaned, recompiled and recleaned before so that caching by > the file system would not impact the result too much), so it does seem > to be a *little* faster :) > > Cheers, > > -- > Romain Bardou > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs