[Caml-list] automatically resolving open?

[Caml-list] automatically resolving open?

[Kenichi Asai]

Would it be possible to transform an OCaml file to the one that does
not use open?  For example, if I have:

open List
let test = map (fun x -> x + 1) [1; 2; 3]

I want to obtain:

let test = List.map (fun x -> x + 1) [1; 2; 3]

where all the opened identifiers are prefixed with the opened module
names.

ocamlc -dtypedtree appears to produce a typed tree where all the
opened variable references are resolved.  Would it be possible to
transform this typed tree back to a source program?

-- 
Kenichi Asai

Re: [Caml-list] automatically resolving open?

[Francois Pottier]

Hello,

Le 23/04/2025 à 16:10, Kenichi Asai a écrit :
> Would it be possible to transform an OCaml file to the one that does
> not use open?

I don't know whether it is possible/easy to do this today,
but it would certainly interesting and useful to have such
a tool.

I note that the output that you expect cannot always be
produced, due to name shadowing issues. For example if
the program is

open List
module List = struct end
let test = map (fun x -> x + 1) [1; 2; 3]

then the best output that one can expect is

let map = List.map
module List = struct end
let test = map (fun x -> x + 1) [1; 2; 3]

That said (contradicting myself), one can actually obtain
better output if one is careful to always use absolute paths.
In this example one could write:

module List = struct end
let test = Stdlib.List.map (fun x -> x + 1) [1; 2; 3]

which relies on the fact that the name "Stdlib" is not shadowed.

As far as I know there is currently no syntax for absolute
paths in OCaml (every path is relative, and every name can
be shadowed). Maybe we should consider adding such a syntax?

-- 
François Pottier
francois.pottier@inria.fr
https://cambium.inria.fr/~fpottier/

Re: [Caml-list] automatically resolving open?

[BOBOT François]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

Le mercredi 23 avril 2025 à 16:32 +0200, Francois Pottier a écrit :
As far as I know there is currently no syntax for absolute
paths in OCaml (every path is relative, and every name can
be shadowed). Maybe we should consider adding such a syntax?


In a dune library, one can require the generation of a root module:

(root_module <module>)

This field instructs Dune to generate a module that will contain module aliases for every library specified in dependencies.
This is useful whenever a library is shadowed by a local module. The library may then still be accessible via this root module.

Best,

--
François Bobot

[-- Attachment #2: Type: text/html, Size: 1400 bytes --]

Re: [Caml-list] automatically resolving open?

[Ivan Gotovchits]

[-- Attachment #1: Type: text/plain, Size: 1812 bytes --]

An interesting corner case is
```
open struct let map f x = f x end
let () = map (fun _ -> ()) 0
```
since OCaml let's us open anonymous modules. In this case, -dtypedtree
generates an unqualified names. So it would be rather easy to remove all
opens to of the named modules, since OCaml will generate fully qualified
names, e.g., `Texp_ident "Stdlib!.List.map"`, but for anonymous modules, it
would be harder to do.

On Wed, Apr 23, 2025 at 10:32 AM Francois Pottier <francois.pottier@inria.fr>
wrote:

>
> Hello,
>
> Le 23/04/2025 à 16:10, Kenichi Asai a écrit :
> > Would it be possible to transform an OCaml file to the one that does
> > not use open?
>
> I don't know whether it is possible/easy to do this today,
> but it would certainly interesting and useful to have such
> a tool.
>
> I note that the output that you expect cannot always be
> produced, due to name shadowing issues. For example if
> the program is
>
> open List
> module List = struct end
> let test = map (fun x -> x + 1) [1; 2; 3]
>
> then the best output that one can expect is
>
> let map = List.map
> module List = struct end
> let test = map (fun x -> x + 1) [1; 2; 3]
>
> That said (contradicting myself), one can actually obtain
> better output if one is careful to always use absolute paths.
> In this example one could write:
>
> module List = struct end
> let test = Stdlib.List.map (fun x -> x + 1) [1; 2; 3]
>
> which relies on the fact that the name "Stdlib" is not shadowed.
>
> As far as I know there is currently no syntax for absolute
> paths in OCaml (every path is relative, and every name can
> be shadowed). Maybe we should consider adding such a syntax?
>
> --
> François Pottier
> francois.pottier@inria.fr
> https://cambium.inria.fr/~fpottier/
>
>

[-- Attachment #2: Type: text/html, Size: 2446 bytes --]

Re: [Caml-list] automatically resolving open?

[Jeremy Yallop]

On Wed, 23 Apr 2025 at 14:32, Francois Pottier
<francois.pottier@inria.fr> wrote:
> open List
> module List = struct end
> let test = map (fun x -> x + 1) [1; 2; 3]

On Wed, 23 Apr 2025 at 14:46, Ivan Gotovchits <ivg@ieee.org> wrote:
> open struct let map f x = f x end
> let () = map (fun _ -> ()) 0

We could perhaps add structure-level substitutions to handle these
kinds of cases:

module L := List
module List = struct end
let test = L.map (fun x -> x + 1) [1; 2; 3]

module M := struct let map f x = f x end
let () = M.map (fun _ -> ()) 0

As in signatures, the idea would be that names like 'L' and 'M' bound
in this way would be internal-only, and would not appear in the
signature.

Re: [Caml-list] automatically resolving open?

[Oleg]

First of all, to some small degree, such tool already exists: MetaOCaml

> open List
> let test = map (fun x -> x + 1) [1; 2; 3]
>
> I want to obtain:
>
> let test = List.map (fun x -> x + 1) [1; 2; 3]

   open List;;
   .<map (fun x -> x + 1) [1; 2; 3]>.;;

- : int list code = .<
Stdlib.List.map (fun x_1 -> x_1 + 1)
  (Stdlib.List.(::)
     (1, (Stdlib.List.(::) (2, (Stdlib.List.(::) (3, Stdlib.List.[]))))))>.

Everything is fully qualified: perhaps even more than
expected. MetaOCaml has functions to write the code to the file
(without the enclosing brackets). The obvious and major limitation is
that MetaOCaml brackets may only contain expressions (rather than
structures). Furtherfore, local modules etc. in those expressions are
not allowed (since it opens a huge can of worms for an uncertain
benefit: I'm not aware of any compelling example for allowing local
modules in brackets, which cannot be simply worked around in
traditional ways.)

> As far as I know there is currently no syntax for absolute
> paths in OCaml (every path is relative, and every name can
> be shadowed). Maybe we should consider adding such a syntax?

Actually, there is a hack for it in OCaml (OCaml source itself says
it's a hack.) See typing/typeclass.ml in OCaml repo and search for
"*predef*". It comes in the context of default arguments, which are
re-written during type checking into code using Some x and None. 
Those Some and None must be the pre-defined ones, rather than
locally re-defined. Thus the re-writing machinery produces
*predef*.Some and *predef*.None. The typing/env.ml and 
typing/persistent_env.ml has code to deal with
*predef*. Specifically, env.ml when asked to locale *predef*.l
looks up `l' in the initial environment.

It might be good to right this hack. The reason I care about it is
that naively pretty-printing Parsetree (converted from a Typedtree)
may produce identifiers like *predef*.None, which are not
syntactically valid. Therefore, I have to hack the pretty-printer to
ensure that the output is at least parseable, always. Incidentally,
the same typeclass.ml code also generates variable names like *sth*
and *opt* (I think typecore.ml has something similar). Now that we
have Ident.create_local, there is no need for such strange names.

[Caml-list] automatically resolving open?

[Virgile Prevosto]

Hello,

this is one of the built-in transformations offered by the recently advertised `ocamlmig` tool:
https://github.com/v-gb/ocamlmig/blob/main/doc/using.md#removing-opens
However, it comes with caveats, including that "when identifiers are requalified, not all possible shadowings are taken into account" (and I've checked that indeed in François' example we end up with a `List.map` below the `module List = struct end`...)

Best regards,

Re: [Caml-list] automatically resolving open?

[Ulysse Gérard]

Refactor open is actually a feature of Merlin which have long been exposed in the custom protocol:
https://github.com/ocaml/merlin/blob/main/doc/dev/PROTOCOL.md#refactor-open--postion-position--action-qualifyunqualify

- In the merlin.el mode the commands are: `merlin-refactor-open` and `merlin-refactor-open-qualify`.
- In the vim plugin it is `merlin#RefactorOpen`.
- In LSP based plugins, such as `ocaml-eglot` for emacs and `vscode-ocaml-platform`, it is available as a code action. It appears to be working in only one direction however: from the `open` statement it is possible to qualifying all identifiers coming from that statement.

As for other solutions, potential shadowing is not checked. This is a fairly obscur command of Merlin, please open an issue if you encounter any issue. Additionally, if someone wants to step in to improve the feature in LSP we will be happy to welcome their contribution :-)

Ulysse

> On 24 Apr 2025, at 08:39, Virgile Prevosto <virgile.prevosto@m4x.org> wrote:
> 
> Hello,
> 
> this is one of the built-in transformations offered by the recently advertised `ocamlmig` tool:
> https://github.com/v-gb/ocamlmig/blob/main/doc/using.md#removing-opens
> However, it comes with caveats, including that "when identifiers are requalified, not all possible shadowings are taken into account" (and I've checked that indeed in François' example we end up with a `List.map` below the `module List = struct end`...)
> 
> Best regards,

Re: [Caml-list] automatically resolving open?

[Kenichi Asai]

Thank you all for the information.  I realize the problem is much
harder than I first thought.  But the refactor open of Merlin appears
to be the closest to what I wanted.  I confirmed that I can add and
remove module names in emacs and VS Code.  Can I do the same thing
using a stand-alone command line tool?  If not, can I do it from the
VS Code extension programatically?

> It appears to be working in only one direction however: from the
> `open` statement it is possible to qualifying all identifiers coming
> from that statement.

In my environment, I could do both direction.

Sincerely,

-- 
Kenichi Asai


On Thu, Apr 24, 2025 at 11:16:36AM +0200,
 Ulysse Gérard wrote:

> Refactor open is actually a feature of Merlin which have long been exposed in the custom protocol:
> https://github.com/ocaml/merlin/blob/main/doc/dev/PROTOCOL.md#refactor-open--postion-position--action-qualifyunqualify
> 
> - In the merlin.el mode the commands are: `merlin-refactor-open` and `merlin-refactor-open-qualify`.
> - In the vim plugin it is `merlin#RefactorOpen`.
> - In LSP based plugins, such as `ocaml-eglot` for emacs and `vscode-ocaml-platform`, it is available as a code action. It appears to be working in only one direction however: from the `open` statement it is possible to qualifying all identifiers coming from that statement.
> 
> As for other solutions, potential shadowing is not checked. This is a fairly obscur command of Merlin, please open an issue if you encounter any issue. Additionally, if someone wants to step in to improve the feature in LSP we will be happy to welcome their contribution :-)
> 
> Ulysse
> 
> > On 24 Apr 2025, at 08:39, Virgile Prevosto <virgile.prevosto@m4x.org> wrote:
> > 
> > Hello,
> > 
> > this is one of the built-in transformations offered by the recently advertised `ocamlmig` tool:
> > https://github.com/v-gb/ocamlmig/blob/main/doc/using.md#removing-opens
> > However, it comes with caveats, including that "when identifiers are requalified, not all possible shadowings are taken into account" (and I've checked that indeed in François' example we end up with a `List.map` below the `module List = struct end`...)
> > 
> > Best regards,

Re: [Caml-list] automatically resolving open?

[Ulysse Gérard]

> On 24 Apr 2025, at 15:06, Kenichi Asai <asai@is.ocha.ac.jp> wrote:
> 
> Thank you all for the information.  I realize the problem is much
> harder than I first thought.  But the refactor open of Merlin appears
> to be the closest to what I wanted.  I confirmed that I can add and
> remove module names in emacs and VS Code.  Can I do the same thing
> using a stand-alone command line tool?  If not, can I do it from the
> VS Code extension programatically?

You can try to query manually to Merlin directly from the command line. For that you will still need to build your project so that the configuration is available. Then you will want to run something like this:

> ocamlmerlin single refactor-open -action qualify -position 4:6 -filename path/to/file.ml <path/to/file.ml

(note that you need to input the file content on stdin or the call will hang)

And then parse the output...

The commands are documented here: https://github.com/ocaml/merlin/blob/main/doc/dev/PROTOCOL.md
You can find examples in the test suite: https://github.com/ocaml/merlin/tree/main/tests/test-dirs/refactor-open

> 
>> It appears to be working in only one direction however: from the
>> `open` statement it is possible to qualifying all identifiers coming
>> from that statement.
> 
> In my environment, I could do both direction.

That's good to know !