[Caml-list] glob-ing dependencies for a custom ocamlbuild rule

[Caml-list] glob-ing dependencies for a custom ocamlbuild rule

[Christoph Höger]

Dear all,

I want to write a custom ocamlbuild rule that basically combines all
source files from a given (sub-)directory and thus depends on these files.

However, the following does not work:

        rule "normalize (preprocess) a Modelica library"
          ~prods:["%.modlib.impl"; "%.modlib.sign"]
          ~deps:["%.modlib/**/*.mo"; "%.modlib.depends"]
          ~doc:"Preprocesses a Modelica library"
          modlibc_command

It seems that the rule does not expand the pattern in the dependency. Is
there a way to achieve something similar? It would be a major PITA to
add all these files manually...

regards,

Christoph
-- 
Christoph Höger

Technische Universität Berlin
Fakultät IV - Elektrotechnik und Informatik
Übersetzerbau und Programmiersprachen

Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin

Tel.: +49 (30) 314-24890
E-Mail: christoph.hoeger@tu-berlin.de

Re: [Caml-list] glob-ing dependencies for a custom ocamlbuild rule

[Gabriel Scherer]

Ocamlbuild distinguishes "static dependencies", which are expressed in
the ~deps argument of the rule declaration, and "dynamic dependencies"
which are files required by the rule command.

The notion of pattern accepted for static dependencies is indeed
rather rigid (less expressive than the patterns used in the _tags
file), and what you have in mind does not work. The supported patterns
are deterministic: given one file, there is a unique way a pattern can
match, and a pattern is instantiated into a single file by any
substitution environment. There is a good reason for that: it is not
clear what the semantics of your proposal would be. Indeed, ocamlbuild
tries to recursively build all static dependencies before deciding to
apply a rule, but what does it mean to build all **/*.mo? Listing the
one that exist in the source directory is easy, but how would you
guess which one to try to build recursively using other rules?

Notions such as "all files matching P in this directory" are not
stable to the replacement of a source file by an intermediate target:
for example if tomorrow you decide that one of the .mo file will be
itself pre-processed from a .mo.pp file, it won't be seen by a
existing-file globbing technique anymore. You have a lot of
flexibility in the behaviors you implement in the build command, so
you can easily do the matching there and "build" those files, but it
will result in a relatively fragile rule; this is you decision to
make, and not in the path of least resistance.


On Tue, Apr 12, 2016 at 8:00 AM, Christoph Höger
<christoph.hoeger@tu-berlin.de> wrote:
> Dear all,
>
> I want to write a custom ocamlbuild rule that basically combines all
> source files from a given (sub-)directory and thus depends on these files.
>
> However, the following does not work:
>
>         rule "normalize (preprocess) a Modelica library"
>           ~prods:["%.modlib.impl"; "%.modlib.sign"]
>           ~deps:["%.modlib/**/*.mo"; "%.modlib.depends"]
>           ~doc:"Preprocesses a Modelica library"
>           modlibc_command
>
> It seems that the rule does not expand the pattern in the dependency. Is
> there a way to achieve something similar? It would be a major PITA to
> add all these files manually...
>
> regards,
>
> Christoph
> --
> Christoph Höger
>
> Technische Universität Berlin
> Fakultät IV - Elektrotechnik und Informatik
> Übersetzerbau und Programmiersprachen
>
> Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin
>
> Tel.: +49 (30) 314-24890
> E-Mail: christoph.hoeger@tu-berlin.de
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] glob-ing dependencies for a custom ocamlbuild rule

[Christoph Höger]

I should have mentioned it, but I am not particularly interested in
letting ocamlbuild build one of the <dir>/**/*.mo files, I just want to
make sure that when it comes to building <dir>.modlib.impl or not,
ocamlbuild considers _all_ source files in <dir> and rebuilds if one has
changed. Is that covered by "dynamic dependencies"?

Am 12.04.2016 um 14:48 schrieb Gabriel Scherer:
> Ocamlbuild distinguishes "static dependencies", which are expressed in
> the ~deps argument of the rule declaration, and "dynamic dependencies"
> which are files required by the rule command.
> 
> The notion of pattern accepted for static dependencies is indeed
> rather rigid (less expressive than the patterns used in the _tags
> file), and what you have in mind does not work. The supported patterns
> are deterministic: given one file, there is a unique way a pattern can
> match, and a pattern is instantiated into a single file by any
> substitution environment. There is a good reason for that: it is not
> clear what the semantics of your proposal would be. Indeed, ocamlbuild
> tries to recursively build all static dependencies before deciding to
> apply a rule, but what does it mean to build all **/*.mo? Listing the
> one that exist in the source directory is easy, but how would you
> guess which one to try to build recursively using other rules?
> 
> Notions such as "all files matching P in this directory" are not
> stable to the replacement of a source file by an intermediate target:
> for example if tomorrow you decide that one of the .mo file will be
> itself pre-processed from a .mo.pp file, it won't be seen by a
> existing-file globbing technique anymore. You have a lot of
> flexibility in the behaviors you implement in the build command, so
> you can easily do the matching there and "build" those files, but it
> will result in a relatively fragile rule; this is you decision to
> make, and not in the path of least resistance.
> 
> 
> On Tue, Apr 12, 2016 at 8:00 AM, Christoph Höger
> <christoph.hoeger@tu-berlin.de> wrote:
>> Dear all,
>>
>> I want to write a custom ocamlbuild rule that basically combines all
>> source files from a given (sub-)directory and thus depends on these files.
>>
>> However, the following does not work:
>>
>>         rule "normalize (preprocess) a Modelica library"
>>           ~prods:["%.modlib.impl"; "%.modlib.sign"]
>>           ~deps:["%.modlib/**/*.mo"; "%.modlib.depends"]
>>           ~doc:"Preprocesses a Modelica library"
>>           modlibc_command
>>
>> It seems that the rule does not expand the pattern in the dependency. Is
>> there a way to achieve something similar? It would be a major PITA to
>> add all these files manually...
>>
>> regards,
>>
>> Christoph
>> --
>> Christoph Höger
>>
>> Technische Universität Berlin
>> Fakultät IV - Elektrotechnik und Informatik
>> Übersetzerbau und Programmiersprachen
>>
>> Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin
>>
>> Tel.: +49 (30) 314-24890
>> E-Mail: christoph.hoeger@tu-berlin.de
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 


-- 
Christoph Höger

Technische Universität Berlin
Fakultät IV - Elektrotechnik und Informatik
Übersetzerbau und Programmiersprachen

Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin

Tel.: +49 (30) 314-24890
E-Mail: christoph.hoeger@tu-berlin.de

Re: [Caml-list] glob-ing dependencies for a custom ocamlbuild rule

[Ashish Agarwal]

[-- Attachment #1: Type: text/plain, Size: 4116 bytes --]

On Tue, Apr 12, 2016 at 8:59 AM, Christoph Höger <
christoph.hoeger@tu-berlin.de> wrote:

_all_ source files in <dir>


I think that is Gabriel's point. The definition of _all_ depends on when
you think this pattern will be evaluated.


It would be a major PITA to add all these files manually...
>

Not really. The nice thing about ocamlbuild is that you can write arbitrary
OCaml code, so just compute deps by traversing the directory yourself.




Am 12.04.2016 um 14:48 schrieb Gabriel Scherer:
> > Ocamlbuild distinguishes "static dependencies", which are expressed in
> > the ~deps argument of the rule declaration, and "dynamic dependencies"
> > which are files required by the rule command.
> >
> > The notion of pattern accepted for static dependencies is indeed
> > rather rigid (less expressive than the patterns used in the _tags
> > file), and what you have in mind does not work. The supported patterns
> > are deterministic: given one file, there is a unique way a pattern can
> > match, and a pattern is instantiated into a single file by any
> > substitution environment. There is a good reason for that: it is not
> > clear what the semantics of your proposal would be. Indeed, ocamlbuild
> > tries to recursively build all static dependencies before deciding to
> > apply a rule, but what does it mean to build all **/*.mo? Listing the
> > one that exist in the source directory is easy, but how would you
> > guess which one to try to build recursively using other rules?
> >
> > Notions such as "all files matching P in this directory" are not
> > stable to the replacement of a source file by an intermediate target:
> > for example if tomorrow you decide that one of the .mo file will be
> > itself pre-processed from a .mo.pp file, it won't be seen by a
> > existing-file globbing technique anymore. You have a lot of
> > flexibility in the behaviors you implement in the build command, so
> > you can easily do the matching there and "build" those files, but it
> > will result in a relatively fragile rule; this is you decision to
> > make, and not in the path of least resistance.
> >
> >
> > On Tue, Apr 12, 2016 at 8:00 AM, Christoph Höger
> > <christoph.hoeger@tu-berlin.de> wrote:
> >> Dear all,
> >>
> >> I want to write a custom ocamlbuild rule that basically combines all
> >> source files from a given (sub-)directory and thus depends on these
> files.
> >>
> >> However, the following does not work:
> >>
> >>         rule "normalize (preprocess) a Modelica library"
> >>           ~prods:["%.modlib.impl"; "%.modlib.sign"]
> >>           ~deps:["%.modlib/**/*.mo"; "%.modlib.depends"]
> >>           ~doc:"Preprocesses a Modelica library"
> >>           modlibc_command
> >>
> >> It seems that the rule does not expand the pattern in the dependency. Is
> >> there a way to achieve something similar? It would be a major PITA to
> >> add all these files manually...
> >>
> >> regards,
> >>
> >> Christoph
> >> --
> >> Christoph Höger
> >>
> >> Technische Universität Berlin
> >> Fakultät IV - Elektrotechnik und Informatik
> >> Übersetzerbau und Programmiersprachen
> >>
> >> Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin
> >>
> >> Tel.: +49 (30) 314-24890
> >> E-Mail: christoph.hoeger@tu-berlin.de
> >>
> >> --
> >> Caml-list mailing list.  Subscription management and archives:
> >> https://sympa.inria.fr/sympa/arc/caml-list
> >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> >> Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
>
>
> --
> Christoph Höger
>
> Technische Universität Berlin
> Fakultät IV - Elektrotechnik und Informatik
> Übersetzerbau und Programmiersprachen
>
> Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin
>
> Tel.: +49 (30) 314-24890
> E-Mail: christoph.hoeger@tu-berlin.de
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 6479 bytes --]

Re: [Caml-list] glob-ing dependencies for a custom ocamlbuild rule

[Gabriel Scherer]

For the record, below is the code of a small plugin that will list
files in %.modlib/*.mo in the source dir, import them all in the build
directory, and also build a %.modlib.list file (in the build
directory) containing the list of imported files.

open Ocamlbuild_plugin

let modlibc_command env build =
  let dir = env "%.modlib" in
  let dir_in_source = Pathname.concat Pathname.pwd dir in
  let files = (* filenames found in %.modlib/*.mo *)
    Pathname.readdir dir_in_source |> Array.to_list
    |> List.filter (fun path -> Pathname.check_extension path "mo") in
  let () =
    (* "build" all these files; imports them in the build directory *)
    List.map (fun file -> [Pathname.concat dir file]) files
    |> build
    |> List.iter Outcome.ignore_good in
  (* then populate %.modlib.list with the list of files we found *)
  Echo (List.map (fun s -> s ^ "\n") files, env "%.modlib.list")

let _ = dispatch begin function
  | After_rules ->
     rule "modelica listing: %.modlib/ => %.modlib.list"
          ~prods:["%.modlib.list"]
          ~doc:"list the files of a Modelica library"
          modlibc_command;
  | _ -> ()
end

On Tue, Apr 12, 2016 at 9:15 AM, Ashish Agarwal <agarwal1975@gmail.com> wrote:
> On Tue, Apr 12, 2016 at 8:59 AM, Christoph Höger
> <christoph.hoeger@tu-berlin.de> wrote:
>
>> _all_ source files in <dir>
>
>
> I think that is Gabriel's point. The definition of _all_ depends on when you
> think this pattern will be evaluated.
>
>
>> It would be a major PITA to add all these files manually...
>
>
> Not really. The nice thing about ocamlbuild is that you can write arbitrary
> OCaml code, so just compute deps by traversing the directory yourself.
>
>
>
>
>> Am 12.04.2016 um 14:48 schrieb Gabriel Scherer:
>> > Ocamlbuild distinguishes "static dependencies", which are expressed in
>> > the ~deps argument of the rule declaration, and "dynamic dependencies"
>> > which are files required by the rule command.
>> >
>> > The notion of pattern accepted for static dependencies is indeed
>> > rather rigid (less expressive than the patterns used in the _tags
>> > file), and what you have in mind does not work. The supported patterns
>> > are deterministic: given one file, there is a unique way a pattern can
>> > match, and a pattern is instantiated into a single file by any
>> > substitution environment. There is a good reason for that: it is not
>> > clear what the semantics of your proposal would be. Indeed, ocamlbuild
>> > tries to recursively build all static dependencies before deciding to
>> > apply a rule, but what does it mean to build all **/*.mo? Listing the
>> > one that exist in the source directory is easy, but how would you
>> > guess which one to try to build recursively using other rules?
>> >
>> > Notions such as "all files matching P in this directory" are not
>> > stable to the replacement of a source file by an intermediate target:
>> > for example if tomorrow you decide that one of the .mo file will be
>> > itself pre-processed from a .mo.pp file, it won't be seen by a
>> > existing-file globbing technique anymore. You have a lot of
>> > flexibility in the behaviors you implement in the build command, so
>> > you can easily do the matching there and "build" those files, but it
>> > will result in a relatively fragile rule; this is you decision to
>> > make, and not in the path of least resistance.
>> >
>> >
>> > On Tue, Apr 12, 2016 at 8:00 AM, Christoph Höger
>> > <christoph.hoeger@tu-berlin.de> wrote:
>> >> Dear all,
>> >>
>> >> I want to write a custom ocamlbuild rule that basically combines all
>> >> source files from a given (sub-)directory and thus depends on these
>> >> files.
>> >>
>> >> However, the following does not work:
>> >>
>> >>         rule "normalize (preprocess) a Modelica library"
>> >>           ~prods:["%.modlib.impl"; "%.modlib.sign"]
>> >>           ~deps:["%.modlib/**/*.mo"; "%.modlib.depends"]
>> >>           ~doc:"Preprocesses a Modelica library"
>> >>           modlibc_command
>> >>
>> >> It seems that the rule does not expand the pattern in the dependency.
>> >> Is
>> >> there a way to achieve something similar? It would be a major PITA to
>> >> add all these files manually...
>> >>
>> >> regards,
>> >>
>> >> Christoph
>> >> --
>> >> Christoph Höger
>> >>
>> >> Technische Universität Berlin
>> >> Fakultät IV - Elektrotechnik und Informatik
>> >> Übersetzerbau und Programmiersprachen
>> >>
>> >> Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin
>> >>
>> >> Tel.: +49 (30) 314-24890
>> >> E-Mail: christoph.hoeger@tu-berlin.de
>> >>
>> >> --
>> >> Caml-list mailing list.  Subscription management and archives:
>> >> https://sympa.inria.fr/sympa/arc/caml-list
>> >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> >> Bug reports: http://caml.inria.fr/bin/caml-bugs
>> >
>>
>>
>> --
>> Christoph Höger
>>
>> Technische Universität Berlin
>> Fakultät IV - Elektrotechnik und Informatik
>> Übersetzerbau und Programmiersprachen
>>
>> Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin
>>
>> Tel.: +49 (30) 314-24890
>> E-Mail: christoph.hoeger@tu-berlin.de
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>