[Caml-list] Static linking with Mingw and dune

[Caml-list] Static linking with Mingw and dune

[Andreas Rossberg]

I’m banging my head against a very basic problem, namely creating a simple standalone executable on Windows with OCaml 5.2. So here are my, possibly stupid, questions:

- With OCaml 5 on Mingw64, how can I statically link libwinpthreads into an executable (using dune)?

- Some answers I found from a couple of years ago suggested using `-static`, but dune is (nowadays?) running through flexdll for linking, which doesn’t understand that. Is there a substitute?

- Some other answers/discussions mentioned the new `-l:` option being preferable, but I couldn’t find a working example, and all my feeble attempts to use it with dune were unsuccessful.

- What exactly is the difference between dune's `link_flags` and `ocamlopt_flags`, and why do some online answers use the latter for setting linking-related options?

Any help is appreciated.

Thanks,
/Andreas

Re: [Caml-list] Static linking with Mingw and dune

[Kate Deplaix]

[-- Attachment #1: Type: text/plain, Size: 1681 bytes --]

I can't answer to all your questions at this moment but this commit might be of interest for you: https://github.com/ocaml/opam/commit/f0157ed232ff1100aa049c922b9bdc857ac21998 (see the CLINKING_windows definition which static links everything but the windows APIs)

This works with 4.14 but i haven't tried with 5.3 yet so your milage may vary.

Regarding OCaml 5.3/mingw64 you shouldn't need to have to do anything with libwinpthreads because it's only used for the msvc port, not the mingw one.

Cheers,
Kate
________________________________
From: caml-list-request@inria.fr <caml-list-request@inria.fr> on behalf of Andreas Rossberg <rossberg@mpi-sws.org>
Sent: 01 December 2024 17:02
To: caml-list@inria.fr <caml-list@inria.fr>
Subject: [Caml-list] Static linking with Mingw and dune

I’m banging my head against a very basic problem, namely creating a simple standalone executable on Windows with OCaml 5.2. So here are my, possibly stupid, questions:

- With OCaml 5 on Mingw64, how can I statically link libwinpthreads into an executable (using dune)?

- Some answers I found from a couple of years ago suggested using `-static`, but dune is (nowadays?) running through flexdll for linking, which doesn’t understand that. Is there a substitute?

- Some other answers/discussions mentioned the new `-l:` option being preferable, but I couldn’t find a working example, and all my feeble attempts to use it with dune were unsuccessful.

- What exactly is the difference between dune's `link_flags` and `ocamlopt_flags`, and why do some online answers use the latter for setting linking-related options?

Any help is appreciated.

Thanks,
/Andreas


[-- Attachment #2: Type: text/html, Size: 2714 bytes --]

Re: [Caml-list] Static linking with Mingw and dune

[Antonin Décimo]

> Regarding OCaml 5.3/mingw64 you shouldn't need to have to do anything with libwinpthreads because it's only used for the msvc port, not the mingw one.

That is not quite exact, winpthreads is used with mingw-w64, but found
in the system installation, and linked to. It is vendored for the MSVC
port and we *statically* link into the runtime only the parts we're
interested in.

To statically link with winpthreads, I suggest you do all of it manually:
- clone winpthreads sources at https://github.com/mingw-w64/mingw-w64,
go to mingw-w64-libraries/winpthreads, and build a static version of
the library with your preferred toolchain;

- hack the OCaml compiler / Makefile to *not* use `-lpthreads`:
  https://github.com/ocaml/ocaml/blob/5a5eb481c7a9d0f039e3169aa8ed19c9b926e982/configure.ac#L2372-L2373
  Maybe it's just sufficient to set PTHREAD_LIBS when invoking
configure; however seeing this line makes me think that winpthreads is
already statically linked with the runtime.

     PTHREAD_LIBS="-l:libpthread.a $link_gcc_eh"]

  Haven't dug further.

-- Antonin

Re: [Caml-list] Static linking with Mingw and dune

[Andreas Rossberg]

Good to know, thank you for the suggestions!

However, I’m afraid that hacking the compiler and tool chain for this purpose is not an option — that would require everybody else who wants to build the project to do the same, which is infeasible.

It’s a bit disappointing that there seems to be no easy solution for this problem. I would expect it to be a common pain point for folks using OCaml 5 to develop and publish application binaries for Windows. Or am I just alien?

Cheers,
/Andreas


> On 2. Dec 2024, at 09:25, Antonin Décimo <antonin.decimo@gmail.com> wrote:
> 
>> Regarding OCaml 5.3/mingw64 you shouldn't need to have to do anything with libwinpthreads because it's only used for the msvc port, not the mingw one.
> 
> That is not quite exact, winpthreads is used with mingw-w64, but found
> in the system installation, and linked to. It is vendored for the MSVC
> port and we *statically* link into the runtime only the parts we're
> interested in.
> 
> To statically link with winpthreads, I suggest you do all of it manually:
> - clone winpthreads sources at https://github.com/mingw-w64/mingw-w64,
> go to mingw-w64-libraries/winpthreads, and build a static version of
> the library with your preferred toolchain;
> 
> - hack the OCaml compiler / Makefile to *not* use `-lpthreads`:
>  https://github.com/ocaml/ocaml/blob/5a5eb481c7a9d0f039e3169aa8ed19c9b926e982/configure.ac#L2372-L2373
>  Maybe it's just sufficient to set PTHREAD_LIBS when invoking
> configure; however seeing this line makes me think that winpthreads is
> already statically linked with the runtime.
> 
>     PTHREAD_LIBS="-l:libpthread.a $link_gcc_eh"]
> 
>  Haven't dug further.
> 
> -- Antonin

Re: [Caml-list] Static linking with Mingw and dune

[Xavier Leroy]

[-- Attachment #1: Type: text/plain, Size: 2549 bytes --]

On Thu, Dec 12, 2024 at 8:46 AM Andreas Rossberg <rossberg@mpi-sws.org>
wrote:

> Good to know, thank you for the suggestions!
>
> However, I’m afraid that hacking the compiler and tool chain for this
> purpose is not an option — that would require everybody else who wants to
> build the project to do the same, which is infeasible.
>
> It’s a bit disappointing that there seems to be no easy solution for this
> problem. I would expect it to be a common pain point for folks using OCaml
> 5 to develop and publish application binaries for Windows. Or am I just
> alien?
>

OCaml's dependency on winpthreads will probably go away in the near future,
see https://github.com/ocaml/ocaml/pull/13416 .

But, more generally, static linking of C libraries is getting more and more
difficult -- even from C, not just OCaml.  For example, under Linux, it's
essentially impossible to statically link with the glibc C standard
library; a musl-based distribution must be used if static linking is
desired.  It looks as if the consensus was that static linking of libraries
is a bad idea and must be prevented.  I don't think it is, but that's
something way beyond OCaml's control.

Best,

- Xavier Leroy




>
> Cheers,
> /Andreas
>
>
> > On 2. Dec 2024, at 09:25, Antonin Décimo <antonin.decimo@gmail.com>
> wrote:
> >
> >> Regarding OCaml 5.3/mingw64 you shouldn't need to have to do anything
> with libwinpthreads because it's only used for the msvc port, not the mingw
> one.
> >
> > That is not quite exact, winpthreads is used with mingw-w64, but found
> > in the system installation, and linked to. It is vendored for the MSVC
> > port and we *statically* link into the runtime only the parts we're
> > interested in.
> >
> > To statically link with winpthreads, I suggest you do all of it manually:
> > - clone winpthreads sources at https://github.com/mingw-w64/mingw-w64,
> > go to mingw-w64-libraries/winpthreads, and build a static version of
> > the library with your preferred toolchain;
> >
> > - hack the OCaml compiler / Makefile to *not* use `-lpthreads`:
> >
> https://github.com/ocaml/ocaml/blob/5a5eb481c7a9d0f039e3169aa8ed19c9b926e982/configure.ac#L2372-L2373
> >  Maybe it's just sufficient to set PTHREAD_LIBS when invoking
> > configure; however seeing this line makes me think that winpthreads is
> > already statically linked with the runtime.
> >
> >     PTHREAD_LIBS="-l:libpthread.a $link_gcc_eh"]
> >
> >  Haven't dug further.
> >
> > -- Antonin
>
>

[-- Attachment #2: Type: text/html, Size: 4245 bytes --]

[Caml-list] Static linking with Mingw and dune

[Andreas Rossberg]

[-- Attachment #1: Type: text/plain, Size: 3119 bytes --]

[Oops, forgot to check Reply-all.]


> Begin forwarded message:
> 
>> On 12. Dec 2024, at 18:31, Xavier Leroy <xavier.leroy@college-de-france.fr> wrote:
>> 
>> On Thu, Dec 12, 2024 at 8:46 AM Andreas Rossberg <rossberg@mpi-sws.org <mailto:rossberg@mpi-sws.org>> wrote:
>>> Good to know, thank you for the suggestions!
>>> 
>>> However, I’m afraid that hacking the compiler and tool chain for this purpose is not an option — that would require everybody else who wants to build the project to do the same, which is infeasible.
>>> 
>>> It’s a bit disappointing that there seems to be no easy solution for this problem. I would expect it to be a common pain point for folks using OCaml 5 to develop and publish application binaries for Windows. Or am I just alien?
>> 
>> OCaml's dependency on winpthreads will probably go away in the near future, see https://github.com/ocaml/ocaml/pull/13416 .
> 
> Ah, that’s good to know!
> 
>> But, more generally, static linking of C libraries is getting more and more difficult -- even from C, not just OCaml.  For example, under Linux, it's essentially impossible to statically link with the glibc C standard library; a musl-based distribution must be used if static linking is desired.  It looks as if the consensus was that static linking of libraries is a bad idea and must be prevented.  I don't think it is, but that's something way beyond OCaml's control.
> 
> Yeah, acknowledged. It’s sad IMO, as both static and dynamic linking have valid use cases, and ideally should be interchangeable mostly transparently. But maybe I’m having my ML modules hat on.
> 
> Cheers,
> /Andreas
> 
> 
>> Best,
>> 
>> - Xavier Leroy
>> 
>> 
>>  
>>> 
>>> Cheers,
>>> /Andreas
>>> 
>>> 
>>> > On 2. Dec 2024, at 09:25, Antonin Décimo <antonin.decimo@gmail.com <mailto:antonin.decimo@gmail.com>> wrote:
>>> > 
>>> >> Regarding OCaml 5.3/mingw64 you shouldn't need to have to do anything with libwinpthreads because it's only used for the msvc port, not the mingw one.
>>> > 
>>> > That is not quite exact, winpthreads is used with mingw-w64, but found
>>> > in the system installation, and linked to. It is vendored for the MSVC
>>> > port and we *statically* link into the runtime only the parts we're
>>> > interested in.
>>> > 
>>> > To statically link with winpthreads, I suggest you do all of it manually:
>>> > - clone winpthreads sources at https://github.com/mingw-w64/mingw-w64,
>>> > go to mingw-w64-libraries/winpthreads, and build a static version of
>>> > the library with your preferred toolchain;
>>> > 
>>> > - hack the OCaml compiler / Makefile to *not* use `-lpthreads`:
>>> >  https://github.com/ocaml/ocaml/blob/5a5eb481c7a9d0f039e3169aa8ed19c9b926e982/configure.ac#L2372-L2373
>>> >  Maybe it's just sufficient to set PTHREAD_LIBS when invoking
>>> > configure; however seeing this line makes me think that winpthreads is
>>> > already statically linked with the runtime.
>>> > 
>>> >     PTHREAD_LIBS="-l:libpthread.a $link_gcc_eh"]
>>> > 
>>> >  Haven't dug further.
>>> > 
>>> > -- Antonin


[-- Attachment #2: Type: text/html, Size: 8054 bytes --]