[Caml-list] Dynlink problems

[Caml-list] Dynlink problems

[Richard Jones]

I'm building a plug-in module as a .cmo file which requires a mixed
OCaml/C library which I build with ocamlmklib (ie. I have foo.cma and
dllfoo.so).

The plug-in module needs to get loaded using Dynlink.

I can't work out how to compile the plug-in so that it references
the functions in dllfoo.so. On loading, it consistently complains
about:

The external function `...' is not available

where ... is a function defined in the foo library.

I've even tried giving -dllib on the command line when compiling the
.cmo file, but this makes no obvious difference.

What am I doing wrong?!

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - all your business data are belong to you.
"One serious obstacle to the adoption of good programming languages is
the notion that everything has to be sacrificed for speed. In computer
languages as in life, speed kills." -- Mike Vanier

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Dynlink problems

[Jacques Garrigue]

From: Richard Jones <rich@annexia.org>

> I can't work out how to compile the plug-in so that it references
> the functions in dllfoo.so. On loading, it consistently complains
> about:
> 
> The external function `...' is not available
> 
> where ... is a function defined in the foo library.
> 
> I've even tried giving -dllib on the command line when compiling the
> .cmo file, but this makes no obvious difference.
> 
> What am I doing wrong?!

"-dllib dllfoo.so" must be used at link time, either for an executable
or a library. It is ignored when compiling a .cmo.

Jacques Garrigue

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Dynlink problems

[Nicolas Cannasse]

> I'm building a plug-in module as a .cmo file which requires a mixed
> OCaml/C library which I build with ocamlmklib (ie. I have foo.cma and
> dllfoo.so).
>
> The plug-in module needs to get loaded using Dynlink.
>
> I can't work out how to compile the plug-in so that it references
> the functions in dllfoo.so. On loading, it consistently complains
> about:
>
> The external function `...' is not available
>
> where ... is a function defined in the foo library.
>
> I've even tried giving -dllib on the command line when compiling the
> .cmo file, but this makes no obvious difference.
>
> What am I doing wrong?!

This should work :
You need to create a CMA with -dllib containing only the CMO you want to
Dynlink.

Nicolas Cannasse

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Dynlink problems

[Richard Jones]

I still can't work out the linking problems.

http://www.annexia.org/tmp/mod_caml-0.02.tar.gz

The situation is this:

* /usr/sbin/apache is the main program, and exports symbols like
  ap_pstrdup to modules.

* mod_caml.so is my module. This is written in C & OCaml (caml_init.ml)
  and requires the OCaml Apache library to run.

* The Apache library is also written in a mix of C & OCaml, and must be
  loaded dynamically because it contains shared data structures in the C
  part.

* Plug-in OCaml programs (eg: eg_print_trans.ml) need to be dynamically
  linked with OCaml Apache lib, and access external symbols in
  /usr/sbin/apache (because these are needed by Apache).

So far I get strange errors like:

Syntax error on line 84 of /home/rich/d/merjis/test/mod_caml/test/httpd.conf:
error while linking /home/rich/d/merjis/test/mod_caml/eg_print_trans.cma.
Reference to undefined global `Callback'

I can fix that one by added stdlib.cma to the list of libraries linked
into eg_print_trans.cma, but that seems the wrong thing to do. Even
when I do this, I get:

Syntax error on line 84 of /home/rich/d/merjis/test/mod_caml/test/httpd.conf:
error while linking /home/rich/d/merjis/test/mod_caml/eg_print_trans.cma.
The external function `caml_input' is not available

.. which I haven't been able to fix because -lcamlrun is only available
as a static library (why??).

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - all your business data are belong to you.
MAKE+ is a sane replacement for GNU autoconf/automake. One script compiles,
RPMs, pkgs etc. Linux, BSD, Solaris. http://www.annexia.org/freeware/makeplus/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Dynlink problems

[Jacques Garrigue]

From: Richard Jones <rich@annexia.org>

> So far I get strange errors like:
> 
> Syntax error on line 84 of /home/rich/d/merjis/test/mod_caml/test/httpd.conf:
> error while linking /home/rich/d/merjis/test/mod_caml/eg_print_trans.cma.
> Reference to undefined global `Callback'
> 
> I can fix that one by added stdlib.cma to the list of libraries linked
> into eg_print_trans.cma, but that seems the wrong thing to do.

Indeed, this is not the best way to do that. What you should is export
the corresponding units to dynlink modules. From the documentation:

val add_interfaces : string list -> string list -> unit
(** [add_interfaces units path] grants dynamically-linked object
   files access to the compilation  units named in list [units].
   The interfaces ([.cmi] files) for these units are searched in
   [path] (a list of directory names). Initially, dynamically-linked
   object files do not have access to any of the compilation
   units composing the running program, not even the standard library.
   [add_interfaces] or {!Dynlink.add_available_units} (see below) must be
   called to grant access to some of the units. *)

Actually [add_available_units] is probably better.

> Even when I do this, I get:
> 
> Syntax error on line 84 of /home/rich/d/merjis/test/mod_caml/test/httpd.conf:
> error while linking /home/rich/d/merjis/test/mod_caml/eg_print_trans.cma.
> The external function `caml_input' is not available
> 
> .. which I haven't been able to fix because -lcamlrun is only available
> as a static library (why??).

If you dynamically link modules with external definitions, you must
also call [allow_unsafe_modules]!

Jacques Garrigue

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Dynlink problems

[Richard Jones]

The problem seems to be that Dynlink simply doesn't work when called
from code in a shared library (.so file).

The first problem is that Dynlink.init () causes a nasty crash - it
tries to open ("") and fails. This seems to be indicative of something
Very Bad happening in Dynlink library.

If I comment out the call to Dynlink.init (), then things just don't
work either.

Here is the smallest example I could make to demonstrate the failure:

http://www.annexia.org/tmp/dynlink-failure.tar.gz

*Please!* someone download this & tell me how to fix it. I spent much
of yesterday night trying to workaround the problems.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - all your business data are belong to you.
NET::FTPSERVER is a full-featured, secure, configurable, database-backed
FTP server written in Perl: http://www.annexia.org/freeware/netftpserver/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Dynlink problems

[Richard Jones]

Further investigations ... I'm calling caml_startup which appears to
call this function:

CAMLexport void caml_startup_code(code_t code, asize_t code_size,
                                  char *data, char **argv)
{
  // ...
  sys_init("", argv);
  // ...
}

The first argument to sys_init causes Sys.executable_name to be set to
"", which means that Dynlink.init fails here:

(* Initialize the linker for toplevel use *)

let init_toplevel () =
  (* Read back the known global symbols and the known primitives
     from the executable file *)
  let ic = open_in_bin Sys.executable_name in        <-- failure happens here

This looks like a bug in the case where we need to use Dynlink from a
shared library (.so file).

Note that the code expects to find a bytecode executable, not a native
binary or .so file, but of course I'm trying to run all of this from a
shared library embedded in a C program. There's simply no bytecode
executable around to look at.

What should I do?

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - all your business data are belong to you.
MAKE+ is a sane replacement for GNU autoconf/automake. One script compiles,
RPMs, pkgs etc. Linux, BSD, Solaris. http://www.annexia.org/freeware/makeplus/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Dynlink problems

[Richard Jones]

The following patch to OCaml fixes this bug in Dynlink. It allows you
to specify an alternate executable name, and also improves debugging
by printing a useful message if the system tries to use
Sys.executable_name when it is empty.

Rich.

----------------------------------------------------------------------
diff -ur ocaml-3.06.orig/bytecomp/symtable.ml ocaml-3.06/bytecomp/symtable.ml
--- ocaml-3.06.orig/bytecomp/symtable.ml	2002-02-11 14:08:43.000000000 +0000
+++ ocaml-3.06/bytecomp/symtable.ml	2003-07-31 18:16:48.000000000 +0100
@@ -243,10 +243,10 @@
 
 (* Initialize the linker for toplevel use *)
 
-let init_toplevel () =
+let init_toplevel_from_executable filename =
   (* Read back the known global symbols and the known primitives
      from the executable file *)
-  let ic = open_in_bin Sys.executable_name in
+  let ic = open_in_bin filename in
   begin try
     Bytesections.read_toc ic;
     ignore(Bytesections.seek_section ic "SYMB");
@@ -267,6 +267,11 @@
   (* Initialize the Dll machinery for toplevel use *)
   Dll.init_toplevel dllpath
 
+let init_toplevel () =
+  if Sys.executable_name = "" then
+    fatal_error "Sys.executable_name is \"\"";
+  init_toplevel_from_executable Sys.executable_name
+
 (* Find the value of a global identifier *)
 
 let get_global_position id = slot_for_getglobal id
diff -ur ocaml-3.06.orig/bytecomp/symtable.mli ocaml-3.06/bytecomp/symtable.mli
--- ocaml-3.06.orig/bytecomp/symtable.mli	2000-03-06 22:11:11.000000000 +0000
+++ ocaml-3.06/bytecomp/symtable.mli	2003-07-31 18:15:22.000000000 +0100
@@ -29,6 +29,7 @@
 (* Functions for the toplevel *)
 
 val init_toplevel: unit -> unit
+val init_toplevel_from_executable: string -> unit
 val update_global_table: unit -> unit
 val get_global_value: Ident.t -> Obj.t
 val assign_global_value: Ident.t -> Obj.t -> unit
diff -ur ocaml-3.06.orig/otherlibs/dynlink/dynlink.ml ocaml-3.06/otherlibs/dynlink/dynlink.ml
--- ocaml-3.06.orig/otherlibs/dynlink/dynlink.ml	2002-04-24 08:57:17.000000000 +0100
+++ ocaml-3.06/otherlibs/dynlink/dynlink.ml	2003-07-31 18:17:18.000000000 +0100
@@ -39,6 +39,9 @@
 let init () =
   Symtable.init_toplevel()  
 
+let init_from_executable =
+  Symtable.init_toplevel_from_executable
+
 (* Check that the object file being loaded has been compiled against
    the same interfaces as the program itself. In addition, check that
    only authorized compilation units are referenced. *)
diff -ur ocaml-3.06.orig/otherlibs/dynlink/dynlink.mli ocaml-3.06/otherlibs/dynlink/dynlink.mli
--- ocaml-3.06.orig/otherlibs/dynlink/dynlink.mli	2001-12-28 23:21:49.000000000 +0000
+++ ocaml-3.06/otherlibs/dynlink/dynlink.mli	2003-07-31 18:18:59.000000000 +0100
@@ -18,6 +18,12 @@
 val init : unit -> unit
 (** Initialize the library. Must be called before [loadfile]. *)
 
+val init_from_executable : string -> unit
+(** Initialize the library. Must be called before [loadfile].
+    This version takes the bytecode executable filename as an argument,
+    rather than using Sys.executable_name which will not be defined in
+    all cases (eg. if the main program is a C program or shared library). *)
+
 val loadfile : string -> unit
 (** Load the given bytecode object file and link it.
    All toplevel expressions in the loaded compilation unit
----------------------------------------------------------------------

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - all your business data are belong to you.
NET::FTPSERVER is a full-featured, secure, configurable, database-backed
FTP server written in Perl: http://www.annexia.org/freeware/netftpserver/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners