From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 76A9E7F1AA for ; Wed, 9 Sep 2015 23:30:09 +0200 (CEST) IronPort-PHdr: 9a23:9smP+xPGRK1AxyUwjQAl6mtUPXoX/o7sNwtQ0KIMzox0Kfj+rarrMEGX3/hxlliBBdydsKIYzbOL+Pi5EUU7or+/81k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i760zceF13FOBZvIaytQ8iJ35jxjr35o8SbSj4LrQT+SIs6FA+xowTVu5teqqpZAYF19CH0pGBVcf9d32JiKAHbtR/94sCt4MwrqHwI6LoJvvRNWqTifqk+UacQTHF/azh0t/vQqALbQACTynwZW2QQ2loUUkmWpC39C7X2si2ynepn2TKGOtX2QKp8fD+L679qThKg3CUCPDc96GDRosp1kORWrA/341R0yovQKJyOOeBlNvfWdNYeAG5ARdp5VipbA4r6YZFZXMQbOuMNgID3vVoDqVOFDgmhHu71gmtHj3Xs3KA+lf8qEQzc0RYIENcHsXCSp9LwYvRBGdupxbXFmG2QJ8hd3i3wvc2VKx0= Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=None smtp.pra=gabriel.scherer@gmail.com; spf=Pass smtp.mailfrom=gabriel.scherer@gmail.com; spf=None smtp.helo=postmaster@mail-ig0-f174.google.com Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.213.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.213.174 as permitted sender) identity=mailfrom; client-ip=209.85.213.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ig0-f174.google.com) identity=helo; client-ip=209.85.213.174; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-ig0-f174.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0DnAQC5o/BVla7VVdFdDoM0NWkGgx6nVYFNhQeNYIV5AoE9BzwQAQEBAQEBAQEQAQEBAQcNCQkfMIIdggcBAQQSER0BGxIMAwwGBQsNAgIJHQICIgERAQUBChIGARISEId2AQMSDakMgTA+MYtBgWyCeYlrChknAwpWhBYBAQEBBgEBAQEYAQUOgRSFUYR7hROCaYFDBYV7DI9PhQqHcIISlxoSI4EXgimBP0A8M4hJAQEB X-IPAS-Result: A0DnAQC5o/BVla7VVdFdDoM0NWkGgx6nVYFNhQeNYIV5AoE9BzwQAQEBAQEBAQEQAQEBAQcNCQkfMIIdggcBAQQSER0BGxIMAwwGBQsNAgIJHQICIgERAQUBChIGARISEId2AQMSDakMgTA+MYtBgWyCeYlrChknAwpWhBYBAQEBBgEBAQEYAQUOgRSFUYR7hROCaYFDBYV7DI9PhQqHcIISlxoSI4EXgimBP0A8M4hJAQEB X-IronPort-AV: E=Sophos;i="5.17,499,1437429600"; d="scan'208";a="176751932" Received: from mail-ig0-f174.google.com ([209.85.213.174]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 09 Sep 2015 23:30:08 +0200 Received: by igcrk20 with SMTP id rk20so1259352igc.1 for ; Wed, 09 Sep 2015 14:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=PL2CnaA7KQAHmxxpRVyP5yI2DjCO5Sxrhm23mLKhHNg=; b=TZk6g4sa+KXmGzPoAIt3FyOGXwyF2HidGTOkjUxxZpZy7eedGzK+akRDhf4yYPTdKA 8Xk2yLTtTnlTXflpyOXgPDIv9u63gCI0Kx5HSvevA2wXrKzqH+dHJui9A6M6XHfXbG3S 4oBn0mt/6SKx3iiyK9Uy5LqMLxra8qYYhJzNIoNky3IcKyKKJtsS7ibg8kiwe4jXM5Pz AInazJEKuO+FrNXCTLTbxMlo/vDQAPxFgwhGLa2Ip1qr9hoAxm6GPSG4HGASZ2pPcwuH QLU99ltV8ipWcwjwP3Jo5lEYF+tWF4MO0+btH0s/2RlSMyUkMAYi7m5CClHBdEmWKGI1 sf1A== X-Received: by 10.50.112.231 with SMTP id it7mr10530611igb.37.1441834206961; Wed, 09 Sep 2015 14:30:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.19.68 with HTTP; Wed, 9 Sep 2015 14:29:27 -0700 (PDT) In-Reply-To: <55F09806.1030507@nmt.edu> References: <55F09806.1030507@nmt.edu> From: Gabriel Scherer Date: Wed, 9 Sep 2015 23:29:27 +0200 Message-ID: To: John Christopher McAlpine , caml users Content-Type: text/plain; charset=UTF-8 Subject: Re: [Caml-list] Use of unit testing frameworks in OCaml Could you be more specific in what kind of benefits you expect such a migration would bring? I'm possibly not familiar enough with these frameworks, but it is rather unclear to me what "significant improvement to the maintenability" we should expect. Note that many tests in the compiler testsuite are in fact rather integration tests than unit tests: they test a complete run of the compiler (or toplevel) on an input program, rather that a single function of the compiler codebase. If I had time to devote to improve the compiler testsuite, here are the two changes I would be interested in implementing (or getting implemented for me): - A way to collect coverage checking information (which parts of the compiler codebase are exercized by the tests?). I started working on it once by just adding Bisect somewhere in the compiler build system, but lost motivation before getting into a a work-out-of-the-box state (it generated many independent coverage files, one for each test, that would have to be aggregated). I think that should not be too hard. - A separation between the tests that are expected to be long-running and the others. Currently running the testsuite takes a few minutes on my machine. If I could get a subset of the tests that has the same coverage for changes that do not affect the code generation backend running in under one minute, that would improve the hack-build-test feedback loops. - A way to run in parallel the tests that are not sensitive to parallel execution (those that do not test performances or thread scheduling). All of these might be easier with a more elegant/declarative/structure test framework, but it is not immediately obvious to me that switching to oUnit would do the job. On Wed, Sep 9, 2015 at 10:35 PM, John Christopher McAlpine wrote: > From what I can tell, the tests for the OCaml compiler and standard > libraries are all done with a fairly small framework designed specifically > for testing the compiler and standard libraries. Is there any particular > reason not to use a tool such as OUnit or Broken for these tests? I am aware > that adding dependencies to the compiler is generally frowned upon, but > these libraries could significantly improve the maintainability of the core > language and libraries. > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs