The problem is that executable `ppx_forest` and library `forest_ppx` both refer to the same compilation

unit `ppx_forest`.  So the solution is easy, just remove the `Ppx_forest` module from the library (indeed, it is not part of the library, it is the executable implementation).

Here is the PR with the fix: https://github.com/padsproj/oforest/pull/9.

# Details

Just in case if you're interested why the resulting error was so confusing :)

The error message came from the following compilation command:

    ocamlfind ocamlopt -linkpkg -g -linkpkg -thread -package threads -package str -package re.str -package re.glob -package re -package ppx_tools.metaquot -package ppx_let -package ppx_deriving.show -package pads.ppx -package pads -package core -package compiler-libs.common ppx/forest_ppx.cmxa parsing/forest_parser.cmxa lib/forest.cmxa ppx/ppx_forest.cmx -o ppx/ppx_forest.native

It is obvious that the command is incorrect (pun intended). Let's, actually, remove all the packages so that we can see the libraries:

    ocamlfind ocamlopt -linkpkg ppx/forest_ppx.cmxa parsing/forest_parser.cmxa lib/forest.cmxa ppx/ppx_forest.cmx -o ppx/ppx_forest.native

In fact, the problem is in library ordering. Modules and libraries must be sorted in topological order, so the first should be `lib/forest.cmxa`, then `parsing/forest_parser.cmxa` and finally `forest_ppx.cmxa`. 

And, yes, this is how this command looks after the fix:

    ocamlfind ocamlopt -linkpkg lib/forest.cmxa parsing/forest_parser.cmxa ppx/forest_ppx.cmxa ppx/ppx_forest.cmx -o ppx/ppx_forest.native

So how the toposort can be broken? My assumption, that this was because you introduced a loop into a dependency graph, when you added ppx_forest to both: the library and the executable. 

If my assumption is true, then probably we should add to ocamlbuild toposort routine an easy check that will detect loops and output  a proper diagnostic message if a dependency graph contains cycles.

Regards,

Ivan Gotovchits

On Fri, Oct 28, 2016 at 2:14 AM, Jonathan DiLorenzo <dilorenzo@cs.cornell.edu> wrote:

Hmm, sadly this seems to still produce the same error after cleaning up all the oasis generated files and everything:

File "_none_", line 1:

Error: No implementations provided for the following modules:

         Forest_parser_helper referenced from ppx/forest_ppx.cmxa(Skins)

Command exited with code 2.

Maybe notable (or maybe obvious) that if I switch oasis to use ocamlc instead of ocamlopt, I instead get:

File "_none_", line 1:

Error: Error while linking ppx/forest_ppx.cma(Skins):

Reference to undefined global `Forest_parser_helper'

Command exited with code 2.

New _oasis file (not with best -> byte change):

Library forest_parser

  Path:           parsing

  BuildTools:     ocamlbuild, menhir, ocamllex

  Findlibparent:  forest

  Findlibname:    forest_parser

  BuildDepends:   forest, ppx_deriving.show, compiler-libs.common 

  CompiledObject: best

  Modules:  Forest_parser_helper, Forest_types

  InternalModules: Forest_lexer, Forest_parser

Library forest_ppx

  Path:           ppx

  BuildTools:     ocamlbuild

  Findlibparent:  forest

  Findlibname:    forest_ppx

  BuildDepends:   re, re.str, forest.forest_parser, forest, ppx_tools.metaquot

  CompiledObject: best

  Modules:        Ppx_forest

  InternalModules: Ppx_forest_lib, Utility, Skins

  XMETAEnable:    true

  XMETADescription: Syntax extension library for OCaml Forest

  XMETARequires:   str re core threads ppx_tools.metaquot

  XMETAExtraLines:  ppx = "ppx_forest"

Executable ppx_forest

  Path:           ppx

  MainIs:         ppx_forest.ml

  BuildDepends:   forest, forest.forest_parser, forest.forest_ppx, pads.ppx, ppx_tools.metaquot

  CompiledObject: best

On Thu, Oct 27, 2016 at 2:24 PM, Ivan Gotovchits <ivg@ieee.org> wrote:

The problem is  in a name clashing between `$opam-switch/lib/ocaml/compiler-libs/parser` and the `parser` library that is compiled from your internal library named `parser.cmxa`.

The compiler-lib is added to the search path by the `ppx_tools` library, and when you're trying to link your final executable, it takes the wrong archive, that definitely doesn't have 

`Forest_parser_helper` module.

The reason, why it is still able to see that there is no `bad_func` in `Forest_parser_helper` is because the interfaces are not contained in the `cmxa` file, but are looked up directly in 

the `cmi`. Compiler is looking for a file named `forest_parser_helper.cmi` and can see that there are not `bad_func` there. 

The solution is to rename your library, e.g., use `Library forest_parser` instead of `Library parser`

On Thu, Oct 27, 2016 at 1:44 PM, Jonathan DiLorenzo <dilorenzo@cs.cornell.edu> wrote:

Hey,

Ok, now the next step would be the following. Carefully check that all modules, that you're using in libraries are included in `Modules` or `InternalModules`. It is the case

that sometimes when you forgot to include a module, oasis (actually `ocamlbuild`), may produce a strange, and on a first glance, irrelevant error message. 

I checked and sadly the modules I'm trying to access are all in `Modules`. Perhaps interestingly, it can still clearly detect statically if the function exists or not because if I try to use a function that doesn't exist it instead gives me this error (which is what I would normally expect):

File "ppx/skins.ml", line 39, characters 8-37:

Error: Unbound value Forest_parser_helper.bad_func

Command exited with code 2.

 

If it doesn't help then the `_build/_log` file might help us to debug the issue. 

Attached. Thanks so much for helping with this again. I'm at a total loss.

 

On Thu, Oct 27, 2016 at 10:59 AM, Jonathan DiLorenzo <dilorenzo@cs.cornell.edu> wrote:

Hey Ivan,

Thanks for your response.

The first one is that you didn't run `oasis setup` after you made the changes, so I would suggest cleaning current state

and starting from scratch. (I usually just do `git clean -idx` just to be sure that I got rid of any generated files, like setup.data, et alas, but be careful, 

don't delete something that you need). 

I tried cleaning up all the generated files. Same error unfortunately

 

Another idea is that you're using oasis 0.4.7 with ocaml-4.03.0+flambda, they are currently incompatible, and can produce weird bugs. If that so, then consider switching 

either a compiler or oasis to different versions. 

I am using oasis 0.4.7 and ocaml-4.03.0, but I don't appear to be using flambda. I tried passing in -config and it said that flambda was false at least, but I admit, I'm not especially familiar with it, so if there's some other way I should be checking for this please let me know. Hopefully, they're compatible sans flambda?

On Thu, Oct 27, 2016 at 1:01 AM, Jonathan DiLorenzo <dilorenzo@cs.cornell.edu> wrote:

Hey all,

I'm trying to build my system using Oasis and OCamlbuild. I have 3 different libraries and for some reason I seem to only be able to refer from one to the other in seemingly random files. For example, I want my 'ppx' library to be able to use functions from my 'parser' library. One file (ppx_forest.ml) can use functions from it, while another in the same library (skins.ml) cannot, giving me this error:

File "_none_", line 1:

Error: No implementations provided for the following modules:

         Forest_parser_helper referenced from ppx/ppx.cmxa(Skins)

Command exited with code 2.

They can all refer to types I've defined therein though.

I'm not fantastic at Oasis so maybe I'm just missing something basic? Any ideas? Any more information I can gather to give more context otherwise?

Thanks in advance,

Jonathan

Excerpt of my Oasis file that may possibly be relevant:

Library forest 

  Path:          lib

  BuildTools:    ocamlbuild

  BuildDepends:   pads, str, re, re.glob, core, threads, ppx_let

  CompiledObject: best

  Modules:       Forest, PadsInterface

Library parser

  Path:           parsing

  BuildTools:     ocamlbuild, menhir, ocamllex

  Findlibparent:  forest

  Findlibname:    parser

  BuildDepends:   forest, ppx_deriving.show, compiler-libs.common 

  CompiledObject: best

  Modules:  Forest_parser_helper

  InternalModules: Forest_lexer, Forest_parser, Forest_types

Library ppx

  Path:           ppx

  BuildTools:     ocamlbuild

  Findlibparent:  forest

  Findlibname:    ppx

  BuildDepends:   re, re.str, forest.parser, forest, ppx_tools.metaquot

  CompiledObject: best

  Modules:        Ppx_forest

  InternalModules: Ppx_forest_lib, Utility, Skins

  XMETAEnable:    true

  XMETARequires:   str re core threads ppx_tools.metaquot

  XMETAExtraLines:  ppx = "ppx_forest"

Executable ppx_forest

  Path:           ppx

  MainIs:         ppx_forest.ml

  BuildDepends:   forest, forest.parser, forest.ppx, pads.ppx, ppx_tools.metaquot

  CompiledObject: best