[Caml-list] [ANN] visitors

[Caml-list] [ANN] visitors

[François Pottier]

Dear OCaml users,

I would like to announce the first release of the "visitors" package.
It is a syntax extension for OCaml, or more accurately, a plugin for
the "ppx_deriving" syntax extension.

By annotating a type definition with [@@deriving visitors { ... }],
one requests the automatic generation of visitor classes, which make
it easy to traverse and transform a data structure.

The documentation (which contains many examples) is here:

   http://gallium.inria.fr/~fpottier/visitors/manual.pdf

To install the package via opam, use the following commands:

   opam update
   opam install visitors

The project's repository is public:

   git clone git@gitlab.inria.fr:fpottier/visitors.git

Comments and feedback are welcome.

-- 
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] [ANN] visitors

[François Pottier]

Hello,

Le 27/01/2017 23:55, Drup a écrit :
> Very nice! I wanted this for quite a while and the documentation is
> excellent.

Thanks! And thanks for your comments and pointers.

> A big difference is that it load type definition from .cmi files directly.

Indeed, a limitation of the ppx approach is that it is purely syntatic:
a syntax tree (the type definition) is turned into a syntax tree (the
definition of the visitor classes). In particular, at the moment, "visitors"
cannot generate visitor classes for a pre-existing type. Perhaps one could
write a tool that reads a .cmi file, but I suppose it would have to be
explicitly invoked as an external tool. I will think about it.

> What do you think of visitor that translates one datatype into another ?

Well, the current package already allows this.

If the two data types are just two distinct instances of a single
parameterized data type, then a "map" visitor can do this, with very little
effort. See the examples in the manual when "ordinary expressions" are
converted to "hash-consed expressions" and back.

If the two data types are unrelated, then, in order to translate the type
"foo" to the type "bar", you need a "fold" visitor for the type "foo". You
then have to manually implement each of the "build_" methods, so as to 
explain
how each data constructor of the type "foo" should be translated.

> I wonder if, given some annotations, it would be doable to use similar
> techniques to generate visitors between different datatypes.

I think you have it already, but instead of placing "annotations" in the
type definition, you have to implement the "build_" methods.

--
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] [ANN] visitors

[François Pottier]

Le 30/01/2017 17:04, François Pottier a écrit :
> If the two data types are unrelated, then, in order to translate the type
> "foo" to the type "bar", you need a "fold" visitor for the type "foo". You
> then have to manually implement each of the "build_" methods, so as to
> explain how each data constructor of the type "foo" should be translated.

I forgot to add that the manual has an example of converting between two
entirely unrelated types: a user-defined type "crowd" is converted to
"(string * string) list".

--
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] [ANN] visitors

[whitequark]

On 2017-01-30 16:04, François Pottier wrote:
>> A big difference is that it load type definition from .cmi files 
>> directly.
> 
> Indeed, a limitation of the ppx approach is that it is purely syntatic:
> a syntax tree (the type definition) is turned into a syntax tree (the
> definition of the visitor classes). In particular, at the moment, 
> "visitors"
> cannot generate visitor classes for a pre-existing type. Perhaps one 
> could
> write a tool that reads a .cmi file, but I suppose it would have to be
> explicitly invoked as an external tool. I will think about it.

I wrote this tool long ago, it's called ppx_import[1]. In fact it 
motivated
some improvements to the compiler, chiefly the way to extract the set of 
-I paths.
It's even linked from ppx_deriving README[2].

[1]: https://github.com/whitequark/ppx_import
[2]: 
https://github.com/whitequark/ppx_deriving/#working-with-existing-types

-- 
whitequark

Re: [Caml-list] [ANN] visitors

[François Pottier]

Hello,

On 31/01/2017 10:30, whitequark wrote:
> I wrote this tool long ago, it's called ppx_import[1]. In fact it motivated
> some improvements to the compiler, chiefly the way to extract the set of
> -I paths.
> It's even linked from ppx_deriving README[2].

This is very cool, and I had missed it. Thanks for pointing it out.
I will refer to it in the visitors documentation.

By the way, the ppx_deriving README says that ppx_import is useful
"in the case where no attributes need to be attached",
but it is more useful than that,
since the "@with" form actually allows attaching attributes a posteriori.

-- 
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] [ANN] visitors

[whitequark]

On 2017-01-31 10:26, François Pottier wrote:
> Hello,
> 
> On 31/01/2017 10:30, whitequark wrote:
>> I wrote this tool long ago, it's called ppx_import[1]. In fact it 
>> motivated
>> some improvements to the compiler, chiefly the way to extract the set 
>> of
>> -I paths.
>> It's even linked from ppx_deriving README[2].
> 
> This is very cool, and I had missed it. Thanks for pointing it out.
> I will refer to it in the visitors documentation.
> 
> By the way, the ppx_deriving README says that ppx_import is useful
> "in the case where no attributes need to be attached",
> but it is more useful than that,
> since the "@with" form actually allows attaching attributes a 
> posteriori.

That paragraph predates [@with]. I've updated it now.

-- 
whitequark

Re: [Caml-list] [ANN] visitors

[François Pottier]

On 31/01/2017 10:30, whitequark wrote:
> I wrote this tool long ago, it's called ppx_import[1].

Is it possible to use ppx_import to import and copy not just one type 
definition, but a whole family of (possibly mutually recursive) type
definitions?

-- 
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] [ANN] visitors

[Emilio Jesús Gallego Arias]

Dear François,

François Pottier <francois.pottier@inria.fr> writes:

> Is it possible to use ppx_import to import and copy not just one type
> definition, but a whole family of (possibly mutually recursive) type
> definitions?

whitequark of course will know the exact details, but indeed ppx_import
seems capable to work in your use case, see:

https://github.com/ejgallego/coq-serapi/blob/master/serlib/ser_constrexpr.ml#L49

E.

Re: [Caml-list] [ANN] visitors

[François Pottier]

On 01/02/2017 16:50, Emilio Jesús Gallego Arias wrote:
> whitequark of course will know the exact details, but indeed ppx_import
> seems capable to work in your use case, see:
> https://github.com/ejgallego/coq-serapi/blob/master/serlib/ser_constrexpr.ml#L49

Thanks Emilio. This works indeed, but you have to write one line for 
every type that you want to import. When you have 86 mutually recursive
types, that seems silly :-)

Anyway, just wondering.

-- 
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] [ANN] visitors

[whitequark]

On 2017-02-01 16:48, François Pottier wrote:
> On 01/02/2017 16:50, Emilio Jesús Gallego Arias wrote:
>> whitequark of course will know the exact details, but indeed 
>> ppx_import
>> seems capable to work in your use case, see:
>> https://github.com/ejgallego/coq-serapi/blob/master/serlib/ser_constrexpr.ml#L49
> 
> Thanks Emilio. This works indeed, but you have to write one line for
> every type that you want to import. When you have 86 mutually
> recursive
> types, that seems silly :-)

Unfortunately Emilio has the shortest implementation that's possible 
right now.
I think this would be a fairly straightforward addition to ppx_import.

-- 
whitequark

Re: [Caml-list] [ANN] visitors

[Emilio Jesús Gallego Arias]

whitequark <whitequark@whitequark.org> writes:

> Unfortunately Emilio has the shortest implementation that's possible
> right now.

> I think this would be a fairly straightforward addition to ppx_import.

In the lines of François suggestion, it would be great if ppx_import
could some kind of "batch" import for modules. Imagine you want to add
show to a whole module, you could do:

[%import: M
  [@attach all [[@deriving show]]]]

or

module M = [%import M ....]

Another option is to teach deriving about modules:

module M = M
[@@deriving sexp]

but I am not sure what would be the best way to avoid all the
boilerplate I currently have [would had submitted an issue otherwise]

Best,
Emilio

Re: [Caml-list] [ANN] visitors

[Kakadu]

Can we have something like

type ('a,'b) glist = Nil | Cons ...
type 'a list = ('a, 'a list) glist [%import crazy hack to know
constructors of glist here]
?

Happy hacking,
Kakadu

On Thu, Feb 2, 2017 at 3:46 AM, Emilio Jesús Gallego Arias <e@x80.org> wrote:
> whitequark <whitequark@whitequark.org> writes:
>
>> Unfortunately Emilio has the shortest implementation that's possible
>> right now.
>
>> I think this would be a fairly straightforward addition to ppx_import.
>
> In the lines of François suggestion, it would be great if ppx_import
> could some kind of "batch" import for modules. Imagine you want to add
> show to a whole module, you could do:
>
> [%import: M
>   [@attach all [[@deriving show]]]]
>
> or
>
> module M = [%import M ....]
>
> Another option is to teach deriving about modules:
>
> module M = M
> [@@deriving sexp]
>
> but I am not sure what would be the best way to avoid all the
> boilerplate I currently have [would had submitted an issue otherwise]
>
> Best,
> Emilio
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] [ANN] visitors

[whitequark]

On 2017-03-16 11:12, Kakadu wrote:
> Can we have something like
> 
> type ('a,'b) glist = Nil | Cons ...
> type 'a list = ('a, 'a list) glist [%import crazy hack to know
> constructors of glist here]
> ?

I actually planned this feature, but scrapped it because I did not want
to implement module scoping. A PR would be warmly welcome.

> 
> Happy hacking,
> Kakadu
> 
> On Thu, Feb 2, 2017 at 3:46 AM, Emilio Jesús Gallego Arias <e@x80.org> 
> wrote:
>> whitequark <whitequark@whitequark.org> writes:
>> 
>>> Unfortunately Emilio has the shortest implementation that's possible
>>> right now.
>> 
>>> I think this would be a fairly straightforward addition to 
>>> ppx_import.
>> 
>> In the lines of François suggestion, it would be great if ppx_import
>> could some kind of "batch" import for modules. Imagine you want to add
>> show to a whole module, you could do:
>> 
>> [%import: M
>>   [@attach all [[@deriving show]]]]
>> 
>> or
>> 
>> module M = [%import M ....]
>> 
>> Another option is to teach deriving about modules:
>> 
>> module M = M
>> [@@deriving sexp]
>> 
>> but I am not sure what would be the best way to avoid all the
>> boilerplate I currently have [would had submitted an issue otherwise]
>> 
>> Best,
>> Emilio
>> 
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs

-- 
whitequark