From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by walapai.inria.fr (8.13.6/8.13.6) with ESMTP id pBB0leDL032641 for ; Sun, 11 Dec 2011 01:47:40 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtgAABH8405KfVI0imdsb2JhbABDqncIIgEBAQoJDQcSBiGBcgEBAQECARICExkBATIFAQQLCwsWJQ8BBA0TAQUBIhMZCYdmAgaYJgqKM4QcAY1AB4ttlHWKaYMIPYN5 X-IronPort-AV: E=Sophos;i="4.71,333,1320620400"; d="scan'208";a="134866887" Received: from mail-ww0-f52.google.com ([74.125.82.52]) by mail1-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-MD5; 11 Dec 2011 01:47:34 +0100 Received: by wgbdr12 with SMTP id dr12so10218420wgb.9 for ; Sat, 10 Dec 2011 16:47:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; bh=l4uwVOxM9DHclerw0TtDN6O0+xFSbh6EyAJgXhbFAkU=; b=v8UVQBror1GXZkrZmbHW2sROJg57BOPpM5hE/Uy3qSi8eYneoRsD9wCcmFRqq8BvbZ gjumED/TVGPpwI7QrzKPa9nQJtVMSBbxac6cMwuMLYkffOm5SQTbyddlkSpD4yG8vEMN Zbx65hr9+Q7zHnMiOILkdhzPxdHgk9AJ8RS84= Received: by 10.180.107.229 with SMTP id hf5mr16418622wib.35.1323564453599; Sat, 10 Dec 2011 16:47:33 -0800 (PST) Received: from spec-desktop.danmey.org (cpc1-cmbg12-0-0-cust201.5-4.cable.virginmedia.com. [86.9.116.202]) by mx.google.com with ESMTPS id ca18sm19525388wib.13.2011.12.10.16.47.31 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 10 Dec 2011 16:47:32 -0800 (PST) From: Wojciech Meyer To: Gabriel Scherer Cc: Wojciech Meyer , =?utf-8?B?SsOpcsOp?= =?utf-8?B?bWll?= Dimino , caml-list@inria.fr References: <55531934-37A5-4CC5-AB67-20CE4CCE8269@googlemail.com> <4EE37070.4010702@inria.fr> <1323541702.32136.8.camel@aurora> <1323550544.32136.19.camel@aurora> Date: Sun, 11 Dec 2011 00:47:12 +0000 In-Reply-To: (Gabriel Scherer's message of "Sun, 11 Dec 2011 00:34:37 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.91 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Caml-list] Camlp4/p5 type reflection [ Gabriel Scherer writes: > A summary to this lengthy mail: > (1) Why type-enriched Camlp4 is an unreasonable idea > (2) We should extract the typedtree; why it's hard > (3) A fictional narrative of the camlp4/camlp5 history > (4) Why you don't want to become Camlp4 maintainer > (5) How we could try not to use Camlp4 in the future > (6) Syntax extension survival advices > > # (1) Why type-enriched Camlp4 is an unreasonable idea > > Wojciech, your idea of having type information at the Camlp4 level is > absolutely unreasonable. You are not speaking about a "minor change" > here, but a major rewrite that would affect the compiler internals as > well. It would really be a new (and interesting) project. Hello Gabriel, I agree would be a serious changes, and I was thinking even of experimenting a bit with this kind of strictly typed meta programming. It's perfectly viable, as I've seen some good examples in my life. (and Template Haskell does it AFAIR). > > Camlp4 is, and I guess will remain, a syntax-level preprocessing > tool. You have to accept the fact that you can't use type information > at this level (but you can certainly "interact" in some way with the > type system by producing/transforming pieces of code in a way that you > know will have interesting typing effects; for example, you may want > to generate code that is purposedly ill-typed in some cases, to > disallow certain uses of your syntax extension). I'm not even sure > what it would mean to access type information at the camlp4 level, as > you're producing and transforming untyped AST; would you want > partially typed ASTs? How is the typer supposed to work on the part > that you haven't transformed yet, and therefore are not valid OCaml > syntax? I suppose you could have a "preprocessing and transformation" > tool at the typedtree level, but that would be a different tool with > different uses, distinct from the syntactic preprocessing part (though > you may develop "extensions" that act on both fronts). > > I'm not aware of so much Camlp4 situations that would really require > typing information. I would be interested in good examples if you have > some. One problem that I have had with Camlp4 is that you don't have > identifier resolution information (eg. you don't know if the > identifier "(@)" you're seeing is really list concatenation, or has > been redefined/shadowed in the context); this makes uses of Camlp4 for > inlining, for example, quirky and fragile. That's still a simpler > problem than type information. Of course worth to point out MetaOCaml and MetaML. They do runtime type safe staged meta programming. Good example would be: let t = (1,2,3,4) in map_tuple t (fun x -> x*2) (at this point I deliberately chosen closure and not additional syntax extension because you can pass closures and you can't pass code easily in Camlp4, because of the mentioned single stage meta programming). of course it's possible with dependent types and lists, but with meta programming you could just expand map_tuple to the code you want, *but* only when the type information is available. (or you can infer the type yourself, in the simplest case if the tuple was expression passed directly to map_tuple). Currently I believe that DSLs are the best way of achieving higher level abstractions, because you are no longer bound to the language syntax. And you can stay distant from the semantics of your target language. > # (6) Syntax extension survival advices > > To the reader considering use of a new syntax extension in his next project: > > - don't Personally I disagree with this statement, because: - syntax extension allows you to do certain things that are not available in a normal way, to enumerate: list comprehensions, pa_lwt, pa_where, pa_monad, pa_js. So they are clearly very useful in some cases. - combinators in particular monads allow certain syntactical abstractions, however there are runtime costs because of constructing closures and evaluating them later. These runtime costs can easily trimmed to some level with some compiler optimisations for instance good inlining (that we don't have as mentioned before in the thread) and efficiency of runtime (we do have a very good gc). - compilation is much easier than interpretation. You see the code generated and the code generated is restricted semantically by the target language. Handling environment is way lower level than just generating some let bindings in a quotation. - > > - if you really must, try to make sure that your code is also > reasonable to use *without* a syntax extension (eg. by producing > a library with a clean interface, making your extension desugar to > uses of it, but also making sure that it can be used by the human > user) > > - if you really must, try to get it in the form of a quotation; the > rest is fragile I agree that quotations are the best part of Camlp4. Right now I use them to generate a lot of ML code (more than hundreds of KLOC) out of data oriented DSL that parsing is detached from the official Camlp4 parser (I use Menhir for that task). It works pretty well and the code generator is pretty compact. > > - alternatively, try to branch yourself on existing "flexible" syntax > extensions such as Markus Mottl's 'type-conv' and Jeremy Yallop's > 'patterns': you are relatively safe if you don't write any line of > code modifying OCaml's syntax yourself. > > http://hg.ocaml.info/release/type-conv > http://code.google.com/p/ocaml-patterns/ > > - do not hesitate to send for code review and/or ask for help on the > list Additionally in case of if you need any help, I would volunteer to help. Cheers; Wojciech