Re: [Caml-list] ocaml, llvm and generating code at runtime

[Basile STARYNKEVITCH <basile@starynkevitch.net>]

Joel Reymont wrote:
> Does anybody have example code that shows how to generate OCaml bindings at runtime with LLVM?
> 
> My goal is to compile an AST into code that uses OCaml functions within the same binary that's doing the compiling.
> 
> I don't think it can be done with OCaml since it requires a standalone assembler, linker, etc. 

You could generate some (Ocaml) source code, compile it (using ocamlopt -shared), and then dynamically load the just 
generated shared module using Dynlink.loadfile. IIRC, the "experimental" ocamlnat binary did that.

 > Correct me if I'm wrong, though. Mine is a web-based compiler with potentially many concurrent sessions. Running gas, 
ld, etc. seems a much heavier and less scalable approach that generating code at runtime.

Indeed, forking an ocamlopt & then loading the shared module is more heavy, and you have a slight latency issue: if the 
generated code is not big (i.e. less than a thousand lines of Ocaml code), it could take a few seconds (or tenths of 
seconds).

LLVM is rumored to be a bit faster, but is also rumored to be slow as a pure JIT (just in time) code generated (w.r.t. 
to other non Ocaml implementations - eg SBCL or CLISP common lisp). Polyml http://polyml.org/ is also supposed to be a 
JIT-ed SML implementation (it is SML not Ocaml). A few years ago, metaocaml existed, but seems dead today.

However, inside a web server, you also have another issue: garbage collection (or simply disposal) of generated code. 
And this happens even with LLVM. You probably should dispose explicitly & manually of session data and generated code.

Ten years ago, SML/NJ was supposed to also garbage-collect code. Ocaml don't do that, and today's Ocaml Dynlink module 
don't have any unloading code (it would be unsafe).

A possible work-around could be to restart your web compiling server once in a while (e.g. twice a day).

Happy new year to everyone!

Regards.

-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mines, sont seulement les miennes} ***