Re: [Caml-list] proposal for finding, loading and composing ppx preprocessors

[Alain Frisch <alain@frisch.fr>]

Hi Benjamin,

This topic of how to specify which ppx processors to run, and avoiding 
multiple processes, is indeed still largely opened.

I don't see what's the benefit of restricting the processors to 
subtrees.  It's easy enough for each processor to traverse extension 
nodes it doesn't support (this is the behavior of the default mapper). 
And I don't think it's a good idea to introduce a composition model 
different from the successive application of different processors on the 
entire tree. I.e. function composition, which is quite well understood 
and easy to reason about.  In particular, you only need to understand 
the behavior of each processor to predict what the composition will do, 
not exactly how each processor is implemented (and which state it 
carries across the tree, internally).

With ocamlfind 1.5, requiring a package when compiling a file adds the 
required -ppx flags in addition to the -I flags.  If we want to avoid 
multiple process, one could create a meta ppx driver which dynamically 
loads and runs other drivers (specified as .cmxs files).  To be able to 
use such as meta driver from ocamlfind, ocamlfind needs to know about 
how to build each ppx processor (i.e. which libraries should be 
invoked).  Dynamic linking might not be the best solution, though: it is 
not available on all platforms, and it requires all libraries on which 
ppx processor depend to provide a corresponding .cmxs.  The alternative 
is to have ocamlfind link a specific meta driver statically (using its 
knowledge of package dependencies) for each actual combination of ppx to 
be used, relying on an internal cache to avoid linking the same driver 
repeatedly, of course.  The next step is to create not ppx drivers (that 
incorporate multiple precessors), but compiler drivers (to avoid 
completely extra process creations and marshaling of the AST).  If this 
is encapsulated in ocamlfind (or a similar tool), this can still be used 
by any build system which currently relies on ocamlfind.

Specifying ppx requirements in the source code is a different topic.  It 
might be a good direction, but then I don't see why this should be 
restricted to ppx requirements and not -I flags.  It would seem logical 
to specify package requirements, which would add both -I and -ppx flags 
(and maybe more).

Actually, it would have been more important to specify package 
requirements for Camlp4 processors, since this knowledge might be 
required by tools that are not under the control of your build system, 
such as your editor/IDE (to load the corresponding syntactic support). 
Since the concrete syntax doesn't change anymore with ppx processors, it 
seems less critical to specify them in the source code (I'd say, not 
more and not less than general package requirement).


-- Alain