From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 3C2A6820A1 for ; Tue, 10 Sep 2013 12:21:45 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of info@gerd-stolpmann.de) identity=pra; client-ip=212.227.126.186; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="info@gerd-stolpmann.de"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of info@gerd-stolpmann.de) identity=mailfrom; client-ip=212.227.126.186; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="info@gerd-stolpmann.de"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of postmaster@moutng.kundenserver.de designates 212.227.126.186 as permitted sender) identity=helo; client-ip=212.227.126.186; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="postmaster@moutng.kundenserver.de"; x-conformance=sidf_compatible; x-record-type="v=spf1" X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgIBAPHxLlLU4366k2dsb2JhbABbgz+De78HgSgWDgEBAQEHCwsJFAMlgiYBBSMyJBALQgICVxkJEodrCK1SkhqJNYYAJgeCaYE0A454ii2FAI5e X-IPAS-Result: AgIBAPHxLlLU4366k2dsb2JhbABbgz+De78HgSgWDgEBAQEHCwsJFAMlgiYBBSMyJBALQgICVxkJEodrCK1SkhqJNYYAJgeCaYE0A454ii2FAI5e X-IronPort-AV: E=Sophos;i="4.90,877,1371074400"; d="asc'?scan'208";a="32246920" Received: from moutng.kundenserver.de ([212.227.126.186]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 10 Sep 2013 12:21:43 +0200 Received: from office1.lan.sumadev.de (dslb-088-069-140-232.pools.arcor-ip.net [88.69.140.232]) by mrelayeu.kundenserver.de (node=mreu1) with ESMTP (Nemesis) id 0MErYk-1VC8zt0hdS-00GH0N; Tue, 10 Sep 2013 12:21:43 +0200 Received: from [192.168.0.110] (ip-109-90-191-98.unitymediagroup.de [109.90.191.98]) by office1.lan.sumadev.de (Postfix) with ESMTPSA id BB2A1C00CF; Tue, 10 Sep 2013 12:21:42 +0200 (CEST) Message-ID: <1378808502.10274.136.camel@zotac> From: Gerd Stolpmann To: oleg@okmij.org Cc: alain@frisch.fr, Xavier.Leroy@inria.fr, caml-list@inria.fr Date: Tue, 10 Sep 2013 12:21:42 +0200 In-Reply-To: <20130910020157.41613.qmail@www1.g3.pair.com> References: <20130910020157.41613.qmail@www1.g3.pair.com> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-ARy8D/rkgTZ+oj0vrDTl" X-Mailer: Evolution 3.2.3-0ubuntu6 Mime-Version: 1.0 X-Provags-ID: V02:K0:Cqqqdan+yc6sMYOnIT29+Z3bJt81iK3yQPvsJtfpcAf uB3lhEuvb8ZAqDi/Lx3W3cZwPx7/pM1zm7ishPLCFs3L+ybAvx XJs3zdmyiEbpeL+F/DoUUSs8msqDGhMqPtN49a4OseYTpp0Jbi mXEpeHufidv9NSsCLDPHH2T7OTJWqGKZiPDuTHrYHXWhmzZuRu 78b3oyDY4z5+JeRcupgfOQJkTomcfCsDjOBHEf83q0I5n/zNfw HwbsmRPJaHiNlXeMl3CqqEmIiUE8mMS21DDsi4NSxW8aiCev6r 0zoOFMt4Ns0DMoj6RxPOP07hpf/SzS42Mi8b2Biij/JBf6yOCn lTvG2mPoDs6oVttrsEFAt+j/CbtTe9zgo2hLgIAs+ Subject: Re: [Caml-list] Accelerating compilation --=-ARy8D/rkgTZ+oj0vrDTl Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Good point. However, I think without RTLD_NOW there is not enough protection against errors. The better solution would be if ocamlc.opt spawned a child process that is a real byterun executable, and that does the loading check for it. For simplicity, this executable should be the just created executable, and an environment variable requests to do the load check only (no real execution beyond that). NB. The "ldd" utility on Linux uses exactly the same trick. Gerd Am Dienstag, den 10.09.2013, 02:01 +0000 schrieb oleg@okmij.org: > > Or are there compelling arguments in favor of using the bytecode versio= n=20 > > of ocamlc/ocamlopt? >=20 > I have come across one case of the meaningful difference between > ocamlc and ocamlc.opt: a particular project can only be linked on > MacOS using ocamlc; ocamlc.opt fails to link. Admittedly, the real > problem is in what I think a hard-to-justify design decision in > OCaml linker. >=20 > The problem occurs when linking byte-compiled executables that use the > delimcc library. I will only talk about byte-compiled version of > delimcc in the message (there are no problems for the native version). >=20 > The following message > https://sympa.inria.fr/sympa/arc/caml-list/2013-04/msg00072.html > gives more detail. >=20 > Similar to the nums library, delimcc includes some C code, which is > arranged in a shared library libdelimcc.so. That shared library refers > to a particular symbol caml_realloc_stack, which is provided by the > byte-code run-time system (that is, ocamlrun). When we invoke the > byte-code executable that uses delimcc, ocamlrun loads libdelimcc.so, > the library looks for the symbol caml_realloc_stack. Since ocamlrun is > the running executable and its provides the symbol, the resolution > succeeds and everyone is happy. >=20 > The problem occurs when linking the executable that uses delimcc. When > OCaml linker processes delimcc.cma, it notices that a shared library > is required. The OCaml linker then loads the library *and forces the > resolution of all undefined references there*. Therefore, the symbol > caml_realloc_stack required by libdelimcc.so has to be found. When the > linking is done by ocamlc, then the running executable is ocamlrun, > which provides the symbol. However, if the linking is done by > ocamlc.opt, it has a different run-time that does not provide the > symbol. Linking fails. Therefore, although ocamlc.opt can be used to > compile projects with delimcc, the final linking step must be done > with ocamlc rather than ocamlc.opt >=20 > I managed to get around the problem using a weak reference. Alas, the > subterfuge does not seem to work on Mac OS. >=20 > The real problem in my view is the strange decision to load shared > libraries at link time and force the resolution of their undefined > references. This decision certainly makes linking slower, without > providing much benefit, it seems. After all, if the resolution > succeeded at link time, it may still fail at run time since the > linked executable can be run in a different location or even a > different computer. >=20 > So, the real problem to me is ocamlc using RTLD_NOW flag when loading > shared library. Removing the flag would make linking faster, and less > painful. >=20 > A few technical details: dlopen with the problematic flag occurs in > the function caml_dlopen in the file byterun/unix.c >=20 > void * caml_dlopen(char * libname, int for_execution, int global) > { > return dlopen(libname, RTLD_NOW | (global ? RTLD_GLOBAL : RTLD_LOCAL) |= RTLD_NODELETE); > /* Could use RTLD_LAZY if for_execution =3D=3D 0, but needs testing */ > } >=20 > That function caml_dlopen is used within caml_dynlink_open_lib, > which, under alias dll_open, is called in Dll.open_dll. The latter > function is invoked in Bytelink.link_bytecode. >=20 >=20 >=20 >=20 --=20 ------------------------------------------------------------ Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de My OCaml site: http://www.camlcity.org Contact details: http://www.camlcity.org/contact.html Company homepage: http://www.gerd-stolpmann.de ------------------------------------------------------------ --=-ARy8D/rkgTZ+oj0vrDTl Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAABAgAGBQJSLvK2AAoJEAaM4b9ZLB5TIyYH/0krAkBNXuN7PG6XQ0y5aXjc lAm96gtgL0KAQZfN2q1M0AFkIo81ecfNwTa3BB17f/h26eiJHYuvnpKp4mgzIh2D YQVOKjtoZpXbI8ntm7n0MknNB91F+Scpk2tvmD7MfMBl97viLAVI8KXFCQDC4pgE xz/YPvDMEQILevrGHQVObU/Y2v2yHa98OrJITgh2/U7J+3CFby3phSMKFdAN0Qzh qujCs/HYoQoegO2kMVieW0nC+8fJlJkQxLO4pDldba6/8RdOiZxqPELcxLAzA8cX RVDaGYB8zKr1T54c0mhBVeS+HZKrmDZt4iDwjaDTuH3Ioq3TnigWKDHJAJ4XPYo= =cZPV -----END PGP SIGNATURE----- --=-ARy8D/rkgTZ+oj0vrDTl--