* [Caml-list] a question about "ocamlopt" and "ocamldep" @ 2012-03-13 18:19 Matej Košík 2012-03-13 18:34 ` Matthias Puech 0 siblings, 1 reply; 8+ messages in thread From: Matej Košík @ 2012-03-13 18:19 UTC (permalink / raw) To: caml-list Hi, The "ocamldep" tool generates Makefile dependencies for both situations: - when we use "ocamlc" - as well as when we use "ocamlopt" Dependencies, generated for "*.cmo" files, are corresponding "*.cmi" files. This is not surprising. However, dependencies, generated for "*.cmx" files, are always other "*.cmx" files. This is surprising. Why "*.cmx" files do not depend on "*.cmi" files? I have noticed this in a bigger project but this phenomenon appear to happen for arbitrarily small projects. Consider the following ocamldep-generated couple of rules: src/ml2c/typing/printtyp.cmo: src/ml2c/typing/types.cmi \ src/ml2c/typing/primitive.cmi src/ml2c/typing/predef.cmi \ src/ml2c/typing/path.cmi src/ml2c/typing/outcometree.cmi \ src/ml2c/typing/oprint.cmi src/ml2c/utils/misc.cmi \ src/ml2c/parsing/longident.cmi src/ml2c/typing/ident.cmi \ src/ml2c/typing/env.cmi src/ml2c/typing/ctype.cmi \ src/ml2c/utils/clflags.cmi src/ml2c/typing/btype.cmi \ src/ml2c/parsing/asttypes.cmi src/ml2c/typing/printtyp.cmi src/ml2c/typing/printtyp.cmx: src/ml2c/typing/types.cmx \ src/ml2c/typing/primitive.cmx src/ml2c/typing/predef.cmx \ src/ml2c/typing/path.cmx src/ml2c/typing/outcometree.cmx \ src/ml2c/typing/oprint.cmx src/ml2c/utils/misc.cmx \ src/ml2c/parsing/longident.cmx src/ml2c/typing/ident.cmx \ src/ml2c/typing/env.cmx src/ml2c/typing/ctype.cmx \ src/ml2c/utils/clflags.cmx src/ml2c/typing/btype.cmx \ src/ml2c/parsing/asttypes.cmx src/ml2c/typing/printtyp.cmi The second rule seems to be unnecessarily demanding (unless it makes no sense to compile *.cmi files if we use ocamlopt). It should read: src/ml2c/typing/printtyp.cmx: src/ml2c/typing/types.cmi \ src/ml2c/typing/primitive.cmi src/ml2c/typing/predef.cmi \ src/ml2c/typing/path.cmi src/ml2c/typing/outcometree.cmi \ src/ml2c/typing/oprint.cmi src/ml2c/utils/misc.cmi \ src/ml2c/parsing/longident.cmi src/ml2c/typing/ident.cmi \ src/ml2c/typing/env.cmi src/ml2c/typing/ctype.cmi \ src/ml2c/utils/clflags.cmi src/ml2c/typing/btype.cmi \ src/ml2c/parsing/asttypes.cmi src/ml2c/typing/printtyp.cmi Shouldn't it? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] a question about "ocamlopt" and "ocamldep" 2012-03-13 18:19 [Caml-list] a question about "ocamlopt" and "ocamldep" Matej Košík @ 2012-03-13 18:34 ` Matthias Puech 2012-03-13 19:26 ` Gabriel Scherer 0 siblings, 1 reply; 8+ messages in thread From: Matthias Puech @ 2012-03-13 18:34 UTC (permalink / raw) To: caml-list; +Cc: 5764c029b688c1c0d24a2e97cd764f Hello, This is consistent with how ocamlc/ocamlopt work: separate compilation is ensured the way you think by bytecode .cmo compilation: to build a module, you only need the *interfaces* of its dependencies, but it is unfortunately not ensured when compiling to native code, because of the global (inter-modules) optimizations performed (inlining AFAIK). Thus, to build a .cmx module, you need to be aware of the actual *code* of its dependencies. I wonder now if it would be theoretically possible to do these optimization, not at compile-time, but delay them until link-time, when the code is fully known... Cheers, -m Le 03/13/2012 07:19 PM, Matej Košík a écrit : > Hi, > > The "ocamldep" tool generates Makefile dependencies for both situations: > - when we use "ocamlc" > - as well as when we use "ocamlopt" > > Dependencies, generated for "*.cmo" files, > are corresponding "*.cmi" files. > > This is not surprising. > > However, dependencies, generated for "*.cmx" files, > are always other "*.cmx" files. > > This is surprising. > > Why "*.cmx" files do not depend on "*.cmi" files? > > I have noticed this in a bigger project but this phenomenon appear to > happen for arbitrarily small projects. > > Consider the following ocamldep-generated couple of rules: > > src/ml2c/typing/printtyp.cmo: src/ml2c/typing/types.cmi \ > src/ml2c/typing/primitive.cmi src/ml2c/typing/predef.cmi \ > src/ml2c/typing/path.cmi src/ml2c/typing/outcometree.cmi \ > src/ml2c/typing/oprint.cmi src/ml2c/utils/misc.cmi \ > src/ml2c/parsing/longident.cmi src/ml2c/typing/ident.cmi \ > src/ml2c/typing/env.cmi src/ml2c/typing/ctype.cmi \ > src/ml2c/utils/clflags.cmi src/ml2c/typing/btype.cmi \ > src/ml2c/parsing/asttypes.cmi src/ml2c/typing/printtyp.cmi > src/ml2c/typing/printtyp.cmx: src/ml2c/typing/types.cmx \ > src/ml2c/typing/primitive.cmx src/ml2c/typing/predef.cmx \ > src/ml2c/typing/path.cmx src/ml2c/typing/outcometree.cmx \ > src/ml2c/typing/oprint.cmx src/ml2c/utils/misc.cmx \ > src/ml2c/parsing/longident.cmx src/ml2c/typing/ident.cmx \ > src/ml2c/typing/env.cmx src/ml2c/typing/ctype.cmx \ > src/ml2c/utils/clflags.cmx src/ml2c/typing/btype.cmx \ > src/ml2c/parsing/asttypes.cmx src/ml2c/typing/printtyp.cmi > > The second rule seems to be unnecessarily demanding (unless it makes no > sense to compile *.cmi files if we use ocamlopt). It should read: > > src/ml2c/typing/printtyp.cmx: src/ml2c/typing/types.cmi \ > src/ml2c/typing/primitive.cmi src/ml2c/typing/predef.cmi \ > src/ml2c/typing/path.cmi src/ml2c/typing/outcometree.cmi \ > src/ml2c/typing/oprint.cmi src/ml2c/utils/misc.cmi \ > src/ml2c/parsing/longident.cmi src/ml2c/typing/ident.cmi \ > src/ml2c/typing/env.cmi src/ml2c/typing/ctype.cmi \ > src/ml2c/utils/clflags.cmi src/ml2c/typing/btype.cmi \ > src/ml2c/parsing/asttypes.cmi src/ml2c/typing/printtyp.cmi > > Shouldn't it? > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] a question about "ocamlopt" and "ocamldep" 2012-03-13 18:34 ` Matthias Puech @ 2012-03-13 19:26 ` Gabriel Scherer 0 siblings, 0 replies; 8+ messages in thread From: Gabriel Scherer @ 2012-03-13 19:26 UTC (permalink / raw) To: Matthias Puech; +Cc: caml-list, 5764c029b688c1c0d24a2e97cd764f > to build a module, > you only need the *interfaces* of its dependencies, but it is unfortunately > not ensured when compiling to native code It is actually possible to have separate compilation for .cmx, just like for .cmo: when ocamlopt looks for the .cmx for a given dependency, if it doesn't find it, it doesn't optimize, and doesn't record it as a dependency. So by selectively removing .cmx from your compilation environment, you can ensure to have only interface-dependency on some modules -- at the cost of disabled optimizations, of course. (I seem to remember that may happen in particular when you compile against certain libraries that do not distribute the .cmx separately.) > I wonder now if it would be theoretically possible to do these optimization, > not at compile-time, but delay them until link-time, when the code is fully > known... Well, you could always recompile just before you want to link. What is usually called "link-time optimizations" are optimizations that are convenient to perform on the compiled objects directly (those that the linkers see). The high-level inlining or memory representation prediction optimizations are really part of "compilation" (but *when* you compile in the lifetime of your project is up to you; you could always distribute source only and compile just before running the program, without saving the result). On Tue, Mar 13, 2012 at 7:34 PM, Matthias Puech <puech@cs.unibo.it> wrote: > Hello, > > This is consistent with how ocamlc/ocamlopt work: separate compilation is > ensured the way you think by bytecode .cmo compilation: to build a module, > you only need the *interfaces* of its dependencies, but it is unfortunately > not ensured when compiling to native code, because of the global > (inter-modules) optimizations performed (inlining AFAIK). Thus, to build a > .cmx module, you need to be aware of the actual *code* of its dependencies. > > I wonder now if it would be theoretically possible to do these optimization, > not at compile-time, but delay them until link-time, when the code is fully > known... > > Cheers, > -m > > Le 03/13/2012 07:19 PM, Matej Košík a écrit : > >> Hi, >> >> The "ocamldep" tool generates Makefile dependencies for both situations: >> - when we use "ocamlc" >> - as well as when we use "ocamlopt" >> >> Dependencies, generated for "*.cmo" files, >> are corresponding "*.cmi" files. >> >> This is not surprising. >> >> However, dependencies, generated for "*.cmx" files, >> are always other "*.cmx" files. >> >> This is surprising. >> >> Why "*.cmx" files do not depend on "*.cmi" files? >> >> I have noticed this in a bigger project but this phenomenon appear to >> happen for arbitrarily small projects. >> >> Consider the following ocamldep-generated couple of rules: >> >> src/ml2c/typing/printtyp.cmo: src/ml2c/typing/types.cmi \ >> src/ml2c/typing/primitive.cmi src/ml2c/typing/predef.cmi \ >> src/ml2c/typing/path.cmi src/ml2c/typing/outcometree.cmi \ >> src/ml2c/typing/oprint.cmi src/ml2c/utils/misc.cmi \ >> src/ml2c/parsing/longident.cmi src/ml2c/typing/ident.cmi \ >> src/ml2c/typing/env.cmi src/ml2c/typing/ctype.cmi \ >> src/ml2c/utils/clflags.cmi src/ml2c/typing/btype.cmi \ >> src/ml2c/parsing/asttypes.cmi src/ml2c/typing/printtyp.cmi >> src/ml2c/typing/printtyp.cmx: src/ml2c/typing/types.cmx \ >> src/ml2c/typing/primitive.cmx src/ml2c/typing/predef.cmx \ >> src/ml2c/typing/path.cmx src/ml2c/typing/outcometree.cmx \ >> src/ml2c/typing/oprint.cmx src/ml2c/utils/misc.cmx \ >> src/ml2c/parsing/longident.cmx src/ml2c/typing/ident.cmx \ >> src/ml2c/typing/env.cmx src/ml2c/typing/ctype.cmx \ >> src/ml2c/utils/clflags.cmx src/ml2c/typing/btype.cmx \ >> src/ml2c/parsing/asttypes.cmx src/ml2c/typing/printtyp.cmi >> >> The second rule seems to be unnecessarily demanding (unless it makes no >> sense to compile *.cmi files if we use ocamlopt). It should read: >> >> src/ml2c/typing/printtyp.cmx: src/ml2c/typing/types.cmi \ >> src/ml2c/typing/primitive.cmi src/ml2c/typing/predef.cmi \ >> src/ml2c/typing/path.cmi src/ml2c/typing/outcometree.cmi \ >> src/ml2c/typing/oprint.cmi src/ml2c/utils/misc.cmi \ >> src/ml2c/parsing/longident.cmi src/ml2c/typing/ident.cmi \ >> src/ml2c/typing/env.cmi src/ml2c/typing/ctype.cmi \ >> src/ml2c/utils/clflags.cmi src/ml2c/typing/btype.cmi \ >> src/ml2c/parsing/asttypes.cmi src/ml2c/typing/printtyp.cmi >> >> Shouldn't it? >> > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/wws/info/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] a question about "ocamlopt" and "ocamldep"
@ 2012-03-14 10:31 Matej Košík
2012-03-14 11:23 ` Gabriel Scherer
0 siblings, 1 reply; 8+ messages in thread
From: Matej Košík @ 2012-03-14 10:31 UTC (permalink / raw)
To: Caml List
On 03/13/2012 06:34 PM, Matthias Puech wrote:
> This is consistent with how ocamlc/ocamlopt work: separate compilation
> is ensured the way you think by bytecode .cmo compilation: to build a
> module, you only need the *interfaces* of its dependencies, but it is
> unfortunately not ensured when compiling to native code, because of the
> global (inter-modules) optimizations performed (inlining AFAIK). Thus,
> to build a .cmx module, you need to be aware of the actual *code* of its
> dependencies.
:-(
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] a question about "ocamlopt" and "ocamldep" 2012-03-14 10:31 Matej Košík @ 2012-03-14 11:23 ` Gabriel Scherer 2012-03-14 12:38 ` Matej Košík 0 siblings, 1 reply; 8+ messages in thread From: Gabriel Scherer @ 2012-03-14 11:23 UTC (permalink / raw) To: Matej Košík; +Cc: Caml List > :-( I don't understand. Why is it sad to have the *ability* to perform cross-module implementation-dependent optimizations (at the inevitable cost of locally damaging separate compilation) *if* you wish? On Wed, Mar 14, 2012 at 11:31 AM, Matej Košík <5764c029b688c1c0d24a2e97cd764f@gmail.com> wrote: > On 03/13/2012 06:34 PM, Matthias Puech wrote: >> This is consistent with how ocamlc/ocamlopt work: separate compilation >> is ensured the way you think by bytecode .cmo compilation: to build a >> module, you only need the *interfaces* of its dependencies, but it is >> unfortunately not ensured when compiling to native code, because of the >> global (inter-modules) optimizations performed (inlining AFAIK). Thus, >> to build a .cmx module, you need to be aware of the actual *code* of its >> dependencies. > > :-( > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/wws/info/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] a question about "ocamlopt" and "ocamldep" 2012-03-14 11:23 ` Gabriel Scherer @ 2012-03-14 12:38 ` Matej Košík 2012-03-14 13:19 ` Virgile Prevosto 0 siblings, 1 reply; 8+ messages in thread From: Matej Košík @ 2012-03-14 12:38 UTC (permalink / raw) To: Gabriel Scherer; +Cc: Caml List On 03/14/2012 11:23 AM, Gabriel Scherer wrote: >> :-( > > I don't understand. Why is it sad to have the *ability* to perform > cross-module implementation-dependent optimizations (at the inevitable > cost of locally damaging separate compilation) *if* you wish? There are two scenarios when I use the compiler: Scenario 1 (most frequent): when I want to incrementally remove typing errors during development. Various optimizations do not matter here. What matters is a short time to rebuild everything (that has to be rebuilt). Scenario 2 (rare one): to produce the final product where quality of various optimizations matter more than the amount of required compilation time If dropping dependencies of *.cmx files on other *.cmx files (rather than on *.cmi files) requires manual intervention or careful thinking, then ocamlopt, with this behavior, is not ideal tool for Scenario 1 (while still being perfectly suitable for Scenario 2). ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] a question about "ocamlopt" and "ocamldep" 2012-03-14 12:38 ` Matej Košík @ 2012-03-14 13:19 ` Virgile Prevosto 2012-03-14 17:12 ` Gabriel Scherer 0 siblings, 1 reply; 8+ messages in thread From: Virgile Prevosto @ 2012-03-14 13:19 UTC (permalink / raw) To: caml-list 2012/3/14 Matej Košík <5764c029b688c1c0d24a2e97cd764f@gmail.com>: > There are two scenarios when I use the compiler: > > Scenario 1 (most frequent): when I want to incrementally remove typing > errors during development. Various optimizations do not matter here. > What matters is a short time to rebuild everything (that has to be rebuilt). > > Scenario 2 (rare one): to produce the final product > where quality of various optimizations matter more than > the amount of required compilation time > > If dropping dependencies of *.cmx files on other *.cmx files (rather > than on *.cmi files) requires manual intervention or careful thinking, > then ocamlopt, with this behavior, is not ideal tool for Scenario 1 > (while still being perfectly suitable for Scenario 2). Scenario 1 is exactly what bytecode compilation is for, and it is indeed fast and without dependencies on .cmo. -- E tutto per oggi, a la prossima volta Virgile ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Caml-list] a question about "ocamlopt" and "ocamldep" 2012-03-14 13:19 ` Virgile Prevosto @ 2012-03-14 17:12 ` Gabriel Scherer 0 siblings, 0 replies; 8+ messages in thread From: Gabriel Scherer @ 2012-03-14 17:12 UTC (permalink / raw) To: Virgile Prevosto; +Cc: caml-list Indeed, ocamlc is faster than ocamlopt (because there is less analysis in the backend, and simply less work to do to bridge the expressivity gap) and suited for fast compile-and-edit cycles. There are occasional cases when I've seen a use for ocamlopt, when I needed to integrate a "run automated tests" in my feedback loop, and some tests where noticeably faster when natively compiled. I must admit I have never give a thought to the .cmx handling, as that has never appeared to be an issue in the past. I'm not sure how hard it is to integrate a "no .cmx dependency" pass in the Makefile; intuitively, just doing "mv *.cmx cmx_save_dir/" before each compilation command ought to be enough. It might be interesting to add a "don't depend on .cmx" option to the compiler, so that it's only a flag to add somewhere in your build system. Given the existence of ocamlc, there is few incentive for this, and apparently the OCaml maintainers have not considered it high-priority. See the following bug report (and do not hesitate to comment there if you are sure that, indeed, you have a real need for this feature): http://caml.inria.fr/mantis/view.php?id=4389 The following bug report is also related to .cmx/.cmxa and implementation dependency (short story: if as a library provider you want your users to be able to cross-optimize with the library code, providing .cmxa is not enough, you also have to provide the separate .cmx): http://caml.inria.fr/mantis/view.php?id=4772 On Wed, Mar 14, 2012 at 2:19 PM, Virgile Prevosto <virgile.prevosto@m4x.org> wrote: > 2012/3/14 Matej Košík <5764c029b688c1c0d24a2e97cd764f@gmail.com>: > >> There are two scenarios when I use the compiler: >> >> Scenario 1 (most frequent): when I want to incrementally remove typing >> errors during development. Various optimizations do not matter here. >> What matters is a short time to rebuild everything (that has to be rebuilt). >> >> Scenario 2 (rare one): to produce the final product >> where quality of various optimizations matter more than >> the amount of required compilation time >> >> If dropping dependencies of *.cmx files on other *.cmx files (rather >> than on *.cmi files) requires manual intervention or careful thinking, >> then ocamlopt, with this behavior, is not ideal tool for Scenario 1 >> (while still being perfectly suitable for Scenario 2). > > Scenario 1 is exactly what bytecode compilation is for, and it is > indeed fast and without dependencies on .cmo. > > > > > > -- > E tutto per oggi, a la prossima volta > Virgile > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/wws/info/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-03-14 17:12 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-03-13 18:19 [Caml-list] a question about "ocamlopt" and "ocamldep" Matej Košík 2012-03-13 18:34 ` Matthias Puech 2012-03-13 19:26 ` Gabriel Scherer 2012-03-14 10:31 Matej Košík 2012-03-14 11:23 ` Gabriel Scherer 2012-03-14 12:38 ` Matej Košík 2012-03-14 13:19 ` Virgile Prevosto 2012-03-14 17:12 ` Gabriel Scherer
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox