[Caml-list] Build-/Installation-Tools - not enogh of them?

[Caml-list] Build-/Installation-Tools - not enogh of them?

[Oliver Bandel]

Hello,

a while ago it looked like there were not enough build- and installation-tools
for OCaml. I remember some discussions about that.

Now it seems to me that there are a lot of them.
So, developers can pick the one they know about.

For all these tools there might be good reasons to use them, and those
developers who looked at these tools and choose them for their projects, will
know them well enough.

The situation differs, if one wants to package the written software,
and one needs to know many of those tools, just to compile the stuff.
So, when one just wants to compile and install some software,
just for that, it would take much effort to learn the different build-tools.

So, packaging has become more complicated, even though for the developers
these tools may save time.

It would be nice if people who used one of the many new building tools
could provide a Makefile that allows just to type
"make" and "make install", instead of expecting everyone who wants to compile
the software to first learn just-another-build-tool.

Also it would be good, to mention early, which installation tools (make-dependencies)
are in use, and too mention needed packages (opam or others) to just build the stuff.

Thanks and regards,
  Oliver Bandel

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Yawar Amin]

[-- Attachment #1: Type: text/plain, Size: 2500 bytes --]

If anyone would like to chime in and say that OCaml build and packaging
system is not that complicated, I would recommend first looking at
https://github.com/rizo/awesome-ocaml#package-management . IMHO we need to
seriously look at consolidating efforts around OPAM for package management,
packaging, building, testing and running. All the serious language-specific
package managers do it, it's a proven strategy and it simplifies life for
the developer.

This could be a typical workflow:

cd some-ocaml-proj
opam install # Switches compiler if necessary and installs and locally
caches package dependencies
opam build
opam run # Automatically builds if necessary
opam test # Ditto
opam package # Ditto; --upload option can immediately upload to opam
opam doc # Builds documentation with ocamldoc or whatever
opam login -u user -p password

Regards,

Yawar

On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel <oliver@first.in-berlin.de>
wrote:

> Hello,
>
> a while ago it looked like there were not enough build- and
> installation-tools
> for OCaml. I remember some discussions about that.
>
> Now it seems to me that there are a lot of them.
> So, developers can pick the one they know about.
>
> For all these tools there might be good reasons to use them, and those
> developers who looked at these tools and choose them for their projects,
> will
> know them well enough.
>
> The situation differs, if one wants to package the written software,
> and one needs to know many of those tools, just to compile the stuff.
> So, when one just wants to compile and install some software,
> just for that, it would take much effort to learn the different
> build-tools.
>
> So, packaging has become more complicated, even though for the developers
> these tools may save time.
>
> It would be nice if people who used one of the many new building tools
> could provide a Makefile that allows just to type
> "make" and "make install", instead of expecting everyone who wants to
> compile
> the software to first learn just-another-build-tool.
>
> Also it would be good, to mention early, which installation tools
> (make-dependencies)
> are in use, and too mention needed packages (opam or others) to just build
> the stuff.
>
> Thanks and regards,
>   Oliver Bandel
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 3480 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 3405 bytes --]



On Mon, 26 Nov 2018, Yawar Amin wrote:

> If anyone would like to chime in and say that OCaml build and packaging
> system is not that complicated, I would recommend first looking
> at https://github.com/rizo/awesome-ocaml#package-management . IMHO we need
> to seriously look at consolidating efforts around OPAM for package
> management, packaging, building, testing and running. All the serious
> language-specific package managers do it, it's a proven strategy and it
> simplifies life for the developer.

I find it odd that simplifying the life of the developer is the highest
priority.  Doesn't one want to simplify the life of the user?  Ideally the
user who has never touched OCaml before in his life?

As a simple example, the web page for installing OCaml says that the
recommended way to install ocaml is to install opam.  There is a link to a
page explaining how to install opam.  How should anyone even have
confidence that they will end up with OCaml after following those
instructions?  Even step 1 of the installation process leads the user to
confusion.

julia

>
> This could be a typical workflow:
>
> cd some-ocaml-proj
> opam install # Switches compiler if necessary and installs and locally
> caches package dependencies
> opam build
> opam run # Automatically builds if necessary
> opam test # Ditto
> opam package # Ditto; --upload option can immediately upload to opam
> opam doc # Builds documentation with ocamldoc or whatever
> opam login -u user -p password
>
> Regards,
>
> Yawar
>
> On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel <oliver@first.in-berlin.de>
> wrote:
>       Hello,
>
>       a while ago it looked like there were not enough build- and
>       installation-tools
>       for OCaml. I remember some discussions about that.
>
>       Now it seems to me that there are a lot of them.
>       So, developers can pick the one they know about.
>
>       For all these tools there might be good reasons to use them, and
>       those
>       developers who looked at these tools and choose them for their
>       projects, will
>       know them well enough.
>
>       The situation differs, if one wants to package the written
>       software,
>       and one needs to know many of those tools, just to compile the
>       stuff.
>       So, when one just wants to compile and install some software,
>       just for that, it would take much effort to learn the different
>       build-tools.
>
>       So, packaging has become more complicated, even though for the
>       developers
>       these tools may save time.
>
>       It would be nice if people who used one of the many new building
>       tools
>       could provide a Makefile that allows just to type
>       "make" and "make install", instead of expecting everyone who
>       wants to compile
>       the software to first learn just-another-build-tool.
>
>       Also it would be good, to mention early, which installation
>       tools (make-dependencies)
>       are in use, and too mention needed packages (opam or others) to
>       just build the stuff.
>
>       Thanks and regards,
>         Oliver Bandel
>
>       --
>       Caml-list mailing list.  Subscription management and archives:
>       https://sympa.inria.fr/sympa/arc/caml-list
>       https://inbox.ocaml.org/caml-list
>       Forum: https://discuss.ocaml.org/
>       Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Yawar Amin]

[-- Attachment #1: Type: text/plain, Size: 4461 bytes --]

Hi Julia, I agree that we need to simplify the life of the user. But that's
the developer's job, not the language toolchain. Users should not need to
know or care about OCaml (ideally), they should download and run binary
packages or install them through their operating system package manager.

Since it's not possible for many projects to provide all possible system
binaries to users, the fallback should be for the user to build the package
with clear instructions that they'll need to install opam and then
`opam build` (e.g.).

Regards,

Yawar

On Mon, Nov 26, 2018 at 11:59 AM Julia Lawall <julia.lawall@lip6.fr> wrote:

>
>
> On Mon, 26 Nov 2018, Yawar Amin wrote:
>
> > If anyone would like to chime in and say that OCaml build and packaging
> > system is not that complicated, I would recommend first looking
> > at https://github.com/rizo/awesome-ocaml#package-management . IMHO we
> need
> > to seriously look at consolidating efforts around OPAM for package
> > management, packaging, building, testing and running. All the serious
> > language-specific package managers do it, it's a proven strategy and it
> > simplifies life for the developer.
>
> I find it odd that simplifying the life of the developer is the highest
> priority.  Doesn't one want to simplify the life of the user?  Ideally the
> user who has never touched OCaml before in his life?
>
> As a simple example, the web page for installing OCaml says that the
> recommended way to install ocaml is to install opam.  There is a link to a
> page explaining how to install opam.  How should anyone even have
> confidence that they will end up with OCaml after following those
> instructions?  Even step 1 of the installation process leads the user to
> confusion.
>
> julia
>
> >
> > This could be a typical workflow:
> >
> > cd some-ocaml-proj
> > opam install # Switches compiler if necessary and installs and locally
> > caches package dependencies
> > opam build
> > opam run # Automatically builds if necessary
> > opam test # Ditto
> > opam package # Ditto; --upload option can immediately upload to opam
> > opam doc # Builds documentation with ocamldoc or whatever
> > opam login -u user -p password
> >
> > Regards,
> >
> > Yawar
> >
> > On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel <oliver@first.in-berlin.de
> >
> > wrote:
> >       Hello,
> >
> >       a while ago it looked like there were not enough build- and
> >       installation-tools
> >       for OCaml. I remember some discussions about that.
> >
> >       Now it seems to me that there are a lot of them.
> >       So, developers can pick the one they know about.
> >
> >       For all these tools there might be good reasons to use them, and
> >       those
> >       developers who looked at these tools and choose them for their
> >       projects, will
> >       know them well enough.
> >
> >       The situation differs, if one wants to package the written
> >       software,
> >       and one needs to know many of those tools, just to compile the
> >       stuff.
> >       So, when one just wants to compile and install some software,
> >       just for that, it would take much effort to learn the different
> >       build-tools.
> >
> >       So, packaging has become more complicated, even though for the
> >       developers
> >       these tools may save time.
> >
> >       It would be nice if people who used one of the many new building
> >       tools
> >       could provide a Makefile that allows just to type
> >       "make" and "make install", instead of expecting everyone who
> >       wants to compile
> >       the software to first learn just-another-build-tool.
> >
> >       Also it would be good, to mention early, which installation
> >       tools (make-dependencies)
> >       are in use, and too mention needed packages (opam or others) to
> >       just build the stuff.
> >
> >       Thanks and regards,
> >         Oliver Bandel
> >
> >       --
> >       Caml-list mailing list.  Subscription management and archives:
> >       https://sympa.inria.fr/sympa/arc/caml-list
> >       https://inbox.ocaml.org/caml-list
> >       Forum: https://discuss.ocaml.org/
> >       Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> >
> >
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs

[-- Attachment #2: Type: text/html, Size: 6351 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 5582 bytes --]



On Mon, 26 Nov 2018, Yawar Amin wrote:

> Hi Julia, I agree that we need to simplify the life of the user. But that's
> the developer's job, not the language toolchain. Users should not need to
> know or care about OCaml (ideally), they should download and run binary
> packages or install them through their operating system package manager.
> Since it's not possible for many projects to provide all possible system
> binaries to users, the fallback should be for the user to build the package
> with clear instructions that they'll need to install opam and then
> `opam build` (e.g.).

Like the original poster, I would very much prefer something based on
make.

julia

>
> Regards,
>
> Yawar
>
> On Mon, Nov 26, 2018 at 11:59 AM Julia Lawall <julia.lawall@lip6.fr> wrote:
>
>
>       On Mon, 26 Nov 2018, Yawar Amin wrote:
>
>       > If anyone would like to chime in and say that OCaml build and
>       packaging
>       > system is not that complicated, I would recommend first
>       looking
>       > at https://github.com/rizo/awesome-ocaml#package-management
>       . IMHO we need
>       > to seriously look at consolidating efforts around OPAM for
>       package
>       > management, packaging, building, testing and running. All the
>       serious
>       > language-specific package managers do it, it's a proven
>       strategy and it
>       > simplifies life for the developer.
>
>       I find it odd that simplifying the life of the developer is the
>       highest
>       priority.  Doesn't one want to simplify the life of the user? 
>       Ideally the
>       user who has never touched OCaml before in his life?
>
>       As a simple example, the web page for installing OCaml says that
>       the
>       recommended way to install ocaml is to install opam.  There is a
>       link to a
>       page explaining how to install opam.  How should anyone even
>       have
>       confidence that they will end up with OCaml after following
>       those
>       instructions?  Even step 1 of the installation process leads the
>       user to
>       confusion.
>
>       julia
>
>       >
>       > This could be a typical workflow:
>       >
>       > cd some-ocaml-proj
>       > opam install # Switches compiler if necessary and installs and
>       locally
>       > caches package dependencies
>       > opam build
>       > opam run # Automatically builds if necessary
>       > opam test # Ditto
>       > opam package # Ditto; --upload option can immediately upload
>       to opam
>       > opam doc # Builds documentation with ocamldoc or whatever
>       > opam login -u user -p password
>       >
>       > Regards,
>       >
>       > Yawar
>       >
>       > On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel
>       <oliver@first.in-berlin.de>
>       > wrote:
>       >       Hello,
>       >
>       >       a while ago it looked like there were not enough build-
>       and
>       >       installation-tools
>       >       for OCaml. I remember some discussions about that.
>       >
>       >       Now it seems to me that there are a lot of them.
>       >       So, developers can pick the one they know about.
>       >
>       >       For all these tools there might be good reasons to use
>       them, and
>       >       those
>       >       developers who looked at these tools and choose them for
>       their
>       >       projects, will
>       >       know them well enough.
>       >
>       >       The situation differs, if one wants to package the
>       written
>       >       software,
>       >       and one needs to know many of those tools, just to
>       compile the
>       >       stuff.
>       >       So, when one just wants to compile and install some
>       software,
>       >       just for that, it would take much effort to learn the
>       different
>       >       build-tools.
>       >
>       >       So, packaging has become more complicated, even though
>       for the
>       >       developers
>       >       these tools may save time.
>       >
>       >       It would be nice if people who used one of the many new
>       building
>       >       tools
>       >       could provide a Makefile that allows just to type
>       >       "make" and "make install", instead of expecting everyone
>       who
>       >       wants to compile
>       >       the software to first learn just-another-build-tool.
>       >
>       >       Also it would be good, to mention early, which
>       installation
>       >       tools (make-dependencies)
>       >       are in use, and too mention needed packages (opam or
>       others) to
>       >       just build the stuff.
>       >
>       >       Thanks and regards,
>       >         Oliver Bandel
>       >
>       >       --
>       >       Caml-list mailing list.  Subscription management and
>       archives:
>       >       https://sympa.inria.fr/sympa/arc/caml-list
>       >       https://inbox.ocaml.org/caml-list
>       >       Forum: https://discuss.ocaml.org/
>       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
>       >
>       >
>       >
>
>       --
>       Caml-list mailing list.  Subscription management and archives:
>       https://sympa.inria.fr/sympa/arc/caml-list
>       https://inbox.ocaml.org/caml-list
>       Forum: https://discuss.ocaml.org/
>       Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Yawar Amin]

[-- Attachment #1: Type: text/plain, Size: 6336 bytes --]

But you still need an OCaml compiler, right? Are you not using opam to get
it? If you are, you already have opam. If you're not ... you will have a
tougher time than you need to :-) opam is the recommended installation
method by the OCaml team: http://ocaml.org/docs/install.html

Regards,

Yawar

On Mon, Nov 26, 2018 at 3:34 PM Julia Lawall <julia.lawall@lip6.fr> wrote:

>
>
> On Mon, 26 Nov 2018, Yawar Amin wrote:
>
> > Hi Julia, I agree that we need to simplify the life of the user. But
> that's
> > the developer's job, not the language toolchain. Users should not need to
> > know or care about OCaml (ideally), they should download and run binary
> > packages or install them through their operating system package manager.
> > Since it's not possible for many projects to provide all possible system
> > binaries to users, the fallback should be for the user to build the
> package
> > with clear instructions that they'll need to install opam and then
> > `opam build` (e.g.).
>
> Like the original poster, I would very much prefer something based on
> make.
>
> julia
>
> >
> > Regards,
> >
> > Yawar
> >
> > On Mon, Nov 26, 2018 at 11:59 AM Julia Lawall <julia.lawall@lip6.fr>
> wrote:
> >
> >
> >       On Mon, 26 Nov 2018, Yawar Amin wrote:
> >
> >       > If anyone would like to chime in and say that OCaml build and
> >       packaging
> >       > system is not that complicated, I would recommend first
> >       looking
> >       > at https://github.com/rizo/awesome-ocaml#package-management
> >       . IMHO we need
> >       > to seriously look at consolidating efforts around OPAM for
> >       package
> >       > management, packaging, building, testing and running. All the
> >       serious
> >       > language-specific package managers do it, it's a proven
> >       strategy and it
> >       > simplifies life for the developer.
> >
> >       I find it odd that simplifying the life of the developer is the
> >       highest
> >       priority.  Doesn't one want to simplify the life of the user?
> >       Ideally the
> >       user who has never touched OCaml before in his life?
> >
> >       As a simple example, the web page for installing OCaml says that
> >       the
> >       recommended way to install ocaml is to install opam.  There is a
> >       link to a
> >       page explaining how to install opam.  How should anyone even
> >       have
> >       confidence that they will end up with OCaml after following
> >       those
> >       instructions?  Even step 1 of the installation process leads the
> >       user to
> >       confusion.
> >
> >       julia
> >
> >       >
> >       > This could be a typical workflow:
> >       >
> >       > cd some-ocaml-proj
> >       > opam install # Switches compiler if necessary and installs and
> >       locally
> >       > caches package dependencies
> >       > opam build
> >       > opam run # Automatically builds if necessary
> >       > opam test # Ditto
> >       > opam package # Ditto; --upload option can immediately upload
> >       to opam
> >       > opam doc # Builds documentation with ocamldoc or whatever
> >       > opam login -u user -p password
> >       >
> >       > Regards,
> >       >
> >       > Yawar
> >       >
> >       > On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel
> >       <oliver@first.in-berlin.de>
> >       > wrote:
> >       >       Hello,
> >       >
> >       >       a while ago it looked like there were not enough build-
> >       and
> >       >       installation-tools
> >       >       for OCaml. I remember some discussions about that.
> >       >
> >       >       Now it seems to me that there are a lot of them.
> >       >       So, developers can pick the one they know about.
> >       >
> >       >       For all these tools there might be good reasons to use
> >       them, and
> >       >       those
> >       >       developers who looked at these tools and choose them for
> >       their
> >       >       projects, will
> >       >       know them well enough.
> >       >
> >       >       The situation differs, if one wants to package the
> >       written
> >       >       software,
> >       >       and one needs to know many of those tools, just to
> >       compile the
> >       >       stuff.
> >       >       So, when one just wants to compile and install some
> >       software,
> >       >       just for that, it would take much effort to learn the
> >       different
> >       >       build-tools.
> >       >
> >       >       So, packaging has become more complicated, even though
> >       for the
> >       >       developers
> >       >       these tools may save time.
> >       >
> >       >       It would be nice if people who used one of the many new
> >       building
> >       >       tools
> >       >       could provide a Makefile that allows just to type
> >       >       "make" and "make install", instead of expecting everyone
> >       who
> >       >       wants to compile
> >       >       the software to first learn just-another-build-tool.
> >       >
> >       >       Also it would be good, to mention early, which
> >       installation
> >       >       tools (make-dependencies)
> >       >       are in use, and too mention needed packages (opam or
> >       others) to
> >       >       just build the stuff.
> >       >
> >       >       Thanks and regards,
> >       >         Oliver Bandel
> >       >
> >       >       --
> >       >       Caml-list mailing list.  Subscription management and
> >       archives:
> >       >       https://sympa.inria.fr/sympa/arc/caml-list
> >       >       https://inbox.ocaml.org/caml-list
> >       >       Forum: https://discuss.ocaml.org/
> >       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
> >       >
> >       >
> >       >
> >
> >       --
> >       Caml-list mailing list.  Subscription management and archives:
> >       https://sympa.inria.fr/sympa/arc/caml-list
> >       https://inbox.ocaml.org/caml-list
> >       Forum: https://discuss.ocaml.org/
> >       Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> >
> >
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs

[-- Attachment #2: Type: text/html, Size: 9804 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 8850 bytes --]



On Mon, 26 Nov 2018, Yawar Amin wrote:

> But you still need an OCaml compiler, right? Are you not using opam to get
> it? If you are, you already have opam. If you're not ... you will have a
> tougher time than you need to :-) opam is the recommended installation
> method by the OCaml team: http://ocaml.org/docs/install.html

Personally, I was in the end forced to install opam.  But I don't want to
force my users to install it (to figure out how to get it to work, I had
to contact a member of Gallium - and my users don't have that
opportunity).  I would prefer that they can just use the ocaml that comes
with their system package manager.

Since opam is the recommended method of installation, couldn't there at
least be provided understandable instructions?  That really seems like a
tougher time than what is necessary.

julia

>
> Regards,
>
> Yawar
>
> On Mon, Nov 26, 2018 at 3:34 PM Julia Lawall <julia.lawall@lip6.fr> wrote:
>
>
>       On Mon, 26 Nov 2018, Yawar Amin wrote:
>
>       > Hi Julia, I agree that we need to simplify the life of the
>       user. But that's
>       > the developer's job, not the language toolchain. Users should
>       not need to
>       > know or care about OCaml (ideally), they should download and
>       run binary
>       > packages or install them through their operating system
>       package manager.
>       > Since it's not possible for many projects to provide all
>       possible system
>       > binaries to users, the fallback should be for the user to
>       build the package
>       > with clear instructions that they'll need to install opam and
>       then
>       > `opam build` (e.g.).
>
>       Like the original poster, I would very much prefer something
>       based on
>       make.
>
>       julia
>
>       >
>       > Regards,
>       >
>       > Yawar
>       >
>       > On Mon, Nov 26, 2018 at 11:59 AM Julia Lawall
>       <julia.lawall@lip6.fr> wrote:
>       >
>       >
>       >       On Mon, 26 Nov 2018, Yawar Amin wrote:
>       >
>       >       > If anyone would like to chime in and say that OCaml
>       build and
>       >       packaging
>       >       > system is not that complicated, I would recommend
>       first
>       >       looking
>       >       >
>       at https://github.com/rizo/awesome-ocaml#package-management
>       >       . IMHO we need
>       >       > to seriously look at consolidating efforts around OPAM
>       for
>       >       package
>       >       > management, packaging, building, testing and running.
>       All the
>       >       serious
>       >       > language-specific package managers do it, it's a
>       proven
>       >       strategy and it
>       >       > simplifies life for the developer.
>       >
>       >       I find it odd that simplifying the life of the developer
>       is the
>       >       highest
>       >       priority.  Doesn't one want to simplify the life of the
>       user? 
>       >       Ideally the
>       >       user who has never touched OCaml before in his life?
>       >
>       >       As a simple example, the web page for installing OCaml
>       says that
>       >       the
>       >       recommended way to install ocaml is to install opam. 
>       There is a
>       >       link to a
>       >       page explaining how to install opam.  How should anyone
>       even
>       >       have
>       >       confidence that they will end up with OCaml after
>       following
>       >       those
>       >       instructions?  Even step 1 of the installation process
>       leads the
>       >       user to
>       >       confusion.
>       >
>       >       julia
>       >
>       >       >
>       >       > This could be a typical workflow:
>       >       >
>       >       > cd some-ocaml-proj
>       >       > opam install # Switches compiler if necessary and
>       installs and
>       >       locally
>       >       > caches package dependencies
>       >       > opam build
>       >       > opam run # Automatically builds if necessary
>       >       > opam test # Ditto
>       >       > opam package # Ditto; --upload option can immediately
>       upload
>       >       to opam
>       >       > opam doc # Builds documentation with ocamldoc or
>       whatever
>       >       > opam login -u user -p password
>       >       >
>       >       > Regards,
>       >       >
>       >       > Yawar
>       >       >
>       >       > On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel
>       >       <oliver@first.in-berlin.de>
>       >       > wrote:
>       >       >       Hello,
>       >       >
>       >       >       a while ago it looked like there were not enough
>       build-
>       >       and
>       >       >       installation-tools
>       >       >       for OCaml. I remember some discussions about
>       that.
>       >       >
>       >       >       Now it seems to me that there are a lot of them.
>       >       >       So, developers can pick the one they know about.
>       >       >
>       >       >       For all these tools there might be good reasons
>       to use
>       >       them, and
>       >       >       those
>       >       >       developers who looked at these tools and choose
>       them for
>       >       their
>       >       >       projects, will
>       >       >       know them well enough.
>       >       >
>       >       >       The situation differs, if one wants to package
>       the
>       >       written
>       >       >       software,
>       >       >       and one needs to know many of those tools, just
>       to
>       >       compile the
>       >       >       stuff.
>       >       >       So, when one just wants to compile and install
>       some
>       >       software,
>       >       >       just for that, it would take much effort to
>       learn the
>       >       different
>       >       >       build-tools.
>       >       >
>       >       >       So, packaging has become more complicated, even
>       though
>       >       for the
>       >       >       developers
>       >       >       these tools may save time.
>       >       >
>       >       >       It would be nice if people who used one of the
>       many new
>       >       building
>       >       >       tools
>       >       >       could provide a Makefile that allows just to
>       type
>       >       >       "make" and "make install", instead of expecting
>       everyone
>       >       who
>       >       >       wants to compile
>       >       >       the software to first learn
>       just-another-build-tool.
>       >       >
>       >       >       Also it would be good, to mention early, which
>       >       installation
>       >       >       tools (make-dependencies)
>       >       >       are in use, and too mention needed packages
>       (opam or
>       >       others) to
>       >       >       just build the stuff.
>       >       >
>       >       >       Thanks and regards,
>       >       >         Oliver Bandel
>       >       >
>       >       >       --
>       >       >       Caml-list mailing list.  Subscription management
>       and
>       >       archives:
>       >       >       https://sympa.inria.fr/sympa/arc/caml-list
>       >       >       https://inbox.ocaml.org/caml-list
>       >       >       Forum: https://discuss.ocaml.org/
>       >       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
>       >       >
>       >       >
>       >       >
>       >
>       >       --
>       >       Caml-list mailing list.  Subscription management and
>       archives:
>       >       https://sympa.inria.fr/sympa/arc/caml-list
>       >       https://inbox.ocaml.org/caml-list
>       >       Forum: https://discuss.ocaml.org/
>       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
>       >
>       >
>       >
>
>       --
>       Caml-list mailing list.  Subscription management and archives:
>       https://sympa.inria.fr/sympa/arc/caml-list
>       https://inbox.ocaml.org/caml-list
>       Forum: https://discuss.ocaml.org/
>       Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Yawar Amin]

[-- Attachment #1: Type: text/plain, Size: 9432 bytes --]

Yes, opam should be easy to install. And in my experience it is; the
process usually goes as described in http://ocaml.org/docs/install.html#OPAM
(install with your system package manager). After that you should be a
couple of commands away from a working compiler.

Realistically, today, system ocaml compiler + make is a corner case in
OCaml development/distribution and shouldn't take precedence over a
standardized developer workflow with opam.

Regards,

Yawar



On Mon, Nov 26, 2018 at 3:54 PM Julia Lawall <julia.lawall@lip6.fr> wrote:

>
>
> On Mon, 26 Nov 2018, Yawar Amin wrote:
>
> > But you still need an OCaml compiler, right? Are you not using opam to
> get
> > it? If you are, you already have opam. If you're not ... you will have a
> > tougher time than you need to :-) opam is the recommended installation
> > method by the OCaml team: http://ocaml.org/docs/install.html
>
> Personally, I was in the end forced to install opam.  But I don't want to
> force my users to install it (to figure out how to get it to work, I had
> to contact a member of Gallium - and my users don't have that
> opportunity).  I would prefer that they can just use the ocaml that comes
> with their system package manager.
>
> Since opam is the recommended method of installation, couldn't there at
> least be provided understandable instructions?  That really seems like a
> tougher time than what is necessary.
>
> julia
>
> >
> > Regards,
> >
> > Yawar
> >
> > On Mon, Nov 26, 2018 at 3:34 PM Julia Lawall <julia.lawall@lip6.fr>
> wrote:
> >
> >
> >       On Mon, 26 Nov 2018, Yawar Amin wrote:
> >
> >       > Hi Julia, I agree that we need to simplify the life of the
> >       user. But that's
> >       > the developer's job, not the language toolchain. Users should
> >       not need to
> >       > know or care about OCaml (ideally), they should download and
> >       run binary
> >       > packages or install them through their operating system
> >       package manager.
> >       > Since it's not possible for many projects to provide all
> >       possible system
> >       > binaries to users, the fallback should be for the user to
> >       build the package
> >       > with clear instructions that they'll need to install opam and
> >       then
> >       > `opam build` (e.g.).
> >
> >       Like the original poster, I would very much prefer something
> >       based on
> >       make.
> >
> >       julia
> >
> >       >
> >       > Regards,
> >       >
> >       > Yawar
> >       >
> >       > On Mon, Nov 26, 2018 at 11:59 AM Julia Lawall
> >       <julia.lawall@lip6.fr> wrote:
> >       >
> >       >
> >       >       On Mon, 26 Nov 2018, Yawar Amin wrote:
> >       >
> >       >       > If anyone would like to chime in and say that OCaml
> >       build and
> >       >       packaging
> >       >       > system is not that complicated, I would recommend
> >       first
> >       >       looking
> >       >       >
> >       at https://github.com/rizo/awesome-ocaml#package-management
> >       >       . IMHO we need
> >       >       > to seriously look at consolidating efforts around OPAM
> >       for
> >       >       package
> >       >       > management, packaging, building, testing and running.
> >       All the
> >       >       serious
> >       >       > language-specific package managers do it, it's a
> >       proven
> >       >       strategy and it
> >       >       > simplifies life for the developer.
> >       >
> >       >       I find it odd that simplifying the life of the developer
> >       is the
> >       >       highest
> >       >       priority.  Doesn't one want to simplify the life of the
> >       user?
> >       >       Ideally the
> >       >       user who has never touched OCaml before in his life?
> >       >
> >       >       As a simple example, the web page for installing OCaml
> >       says that
> >       >       the
> >       >       recommended way to install ocaml is to install opam.
> >       There is a
> >       >       link to a
> >       >       page explaining how to install opam.  How should anyone
> >       even
> >       >       have
> >       >       confidence that they will end up with OCaml after
> >       following
> >       >       those
> >       >       instructions?  Even step 1 of the installation process
> >       leads the
> >       >       user to
> >       >       confusion.
> >       >
> >       >       julia
> >       >
> >       >       >
> >       >       > This could be a typical workflow:
> >       >       >
> >       >       > cd some-ocaml-proj
> >       >       > opam install # Switches compiler if necessary and
> >       installs and
> >       >       locally
> >       >       > caches package dependencies
> >       >       > opam build
> >       >       > opam run # Automatically builds if necessary
> >       >       > opam test # Ditto
> >       >       > opam package # Ditto; --upload option can immediately
> >       upload
> >       >       to opam
> >       >       > opam doc # Builds documentation with ocamldoc or
> >       whatever
> >       >       > opam login -u user -p password
> >       >       >
> >       >       > Regards,
> >       >       >
> >       >       > Yawar
> >       >       >
> >       >       > On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel
> >       >       <oliver@first.in-berlin.de>
> >       >       > wrote:
> >       >       >       Hello,
> >       >       >
> >       >       >       a while ago it looked like there were not enough
> >       build-
> >       >       and
> >       >       >       installation-tools
> >       >       >       for OCaml. I remember some discussions about
> >       that.
> >       >       >
> >       >       >       Now it seems to me that there are a lot of them.
> >       >       >       So, developers can pick the one they know about.
> >       >       >
> >       >       >       For all these tools there might be good reasons
> >       to use
> >       >       them, and
> >       >       >       those
> >       >       >       developers who looked at these tools and choose
> >       them for
> >       >       their
> >       >       >       projects, will
> >       >       >       know them well enough.
> >       >       >
> >       >       >       The situation differs, if one wants to package
> >       the
> >       >       written
> >       >       >       software,
> >       >       >       and one needs to know many of those tools, just
> >       to
> >       >       compile the
> >       >       >       stuff.
> >       >       >       So, when one just wants to compile and install
> >       some
> >       >       software,
> >       >       >       just for that, it would take much effort to
> >       learn the
> >       >       different
> >       >       >       build-tools.
> >       >       >
> >       >       >       So, packaging has become more complicated, even
> >       though
> >       >       for the
> >       >       >       developers
> >       >       >       these tools may save time.
> >       >       >
> >       >       >       It would be nice if people who used one of the
> >       many new
> >       >       building
> >       >       >       tools
> >       >       >       could provide a Makefile that allows just to
> >       type
> >       >       >       "make" and "make install", instead of expecting
> >       everyone
> >       >       who
> >       >       >       wants to compile
> >       >       >       the software to first learn
> >       just-another-build-tool.
> >       >       >
> >       >       >       Also it would be good, to mention early, which
> >       >       installation
> >       >       >       tools (make-dependencies)
> >       >       >       are in use, and too mention needed packages
> >       (opam or
> >       >       others) to
> >       >       >       just build the stuff.
> >       >       >
> >       >       >       Thanks and regards,
> >       >       >         Oliver Bandel
> >       >       >
> >       >       >       --
> >       >       >       Caml-list mailing list.  Subscription management
> >       and
> >       >       archives:
> >       >       >       https://sympa.inria.fr/sympa/arc/caml-list
> >       >       >       https://inbox.ocaml.org/caml-list
> >       >       >       Forum: https://discuss.ocaml.org/
> >       >       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
> >       >       >
> >       >       >
> >       >       >
> >       >
> >       >       --
> >       >       Caml-list mailing list.  Subscription management and
> >       archives:
> >       >       https://sympa.inria.fr/sympa/arc/caml-list
> >       >       https://inbox.ocaml.org/caml-list
> >       >       Forum: https://discuss.ocaml.org/
> >       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
> >       >
> >       >
> >       >
> >
> >       --
> >       Caml-list mailing list.  Subscription management and archives:
> >       https://sympa.inria.fr/sympa/arc/caml-list
> >       https://inbox.ocaml.org/caml-list
> >       Forum: https://discuss.ocaml.org/
> >       Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> >
> >
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs

[-- Attachment #2: Type: text/html, Size: 15183 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Julia Lawall]

[-- Attachment #1: Type: text/plain, Size: 13700 bytes --]



On Mon, 26 Nov 2018, Yawar Amin wrote:

> Yes, opam should be easy to install. And in my experience it is; the process
> usually goes as described in http://ocaml.org/docs/install.html#OPAM
> (install with your system package manager). After that you should be a
> couple of commands away from a working compiler.

I guess opam is reasonably easy to install.  The problem is that as far as
I can see there is nothing to explain how to get from installing opam to
having a working compiler.  Or if there is an explanation, it is not
obvious where it is.

> Realistically, today, system ocaml compiler + make is a corner case in OCaml
> development/distribution and shouldn't take precedence over a standardized
> developer workflow with opam.

The problem is how many programming languages exist today.  Is it really
realistic to expect users to learn how to use a different package manager
for all of them?  Opam is fine for people who use OCaml.  It is a
hindrance for people who don't care at all about OCaml.  And if it really
has to be the only way OCaml software can be used, a lot more effort has
to be put into making it friendly for people who are totally ignorant
about it and have no contact with the OCaml community.  Saying "the way to
install ocaml is use opam" and then giving a link showing how to install
opam with one's package manager is not a solution.

julia

>
> Regards,
>
> Yawar
>
>
>
> On Mon, Nov 26, 2018 at 3:54 PM Julia Lawall <julia.lawall@lip6.fr> wrote:
>
>
>       On Mon, 26 Nov 2018, Yawar Amin wrote:
>
>       > But you still need an OCaml compiler, right? Are you not using
>       opam to get
>       > it? If you are, you already have opam. If you're not ... you
>       will have a
>       > tougher time than you need to :-) opam is the recommended
>       installation
>       > method by the OCaml team: http://ocaml.org/docs/install.html
>
>       Personally, I was in the end forced to install opam.  But I
>       don't want to
>       force my users to install it (to figure out how to get it to
>       work, I had
>       to contact a member of Gallium - and my users don't have that
>       opportunity).  I would prefer that they can just use the ocaml
>       that comes
>       with their system package manager.
>
>       Since opam is the recommended method of installation, couldn't
>       there at
>       least be provided understandable instructions?  That really
>       seems like a
>       tougher time than what is necessary.
>
>       julia
>
>       >
>       > Regards,
>       >
>       > Yawar
>       >
>       > On Mon, Nov 26, 2018 at 3:34 PM Julia Lawall
>       <julia.lawall@lip6.fr> wrote:
>       >
>       >
>       >       On Mon, 26 Nov 2018, Yawar Amin wrote:
>       >
>       >       > Hi Julia, I agree that we need to simplify the life of
>       the
>       >       user. But that's
>       >       > the developer's job, not the language toolchain. Users
>       should
>       >       not need to
>       >       > know or care about OCaml (ideally), they should
>       download and
>       >       run binary
>       >       > packages or install them through their operating
>       system
>       >       package manager.
>       >       > Since it's not possible for many projects to provide
>       all
>       >       possible system
>       >       > binaries to users, the fallback should be for the user
>       to
>       >       build the package
>       >       > with clear instructions that they'll need to install
>       opam and
>       >       then
>       >       > `opam build` (e.g.).
>       >
>       >       Like the original poster, I would very much prefer
>       something
>       >       based on
>       >       make.
>       >
>       >       julia
>       >
>       >       >
>       >       > Regards,
>       >       >
>       >       > Yawar
>       >       >
>       >       > On Mon, Nov 26, 2018 at 11:59 AM Julia Lawall
>       >       <julia.lawall@lip6.fr> wrote:
>       >       >
>       >       >
>       >       >       On Mon, 26 Nov 2018, Yawar Amin wrote:
>       >       >
>       >       >       > If anyone would like to chime in and say that
>       OCaml
>       >       build and
>       >       >       packaging
>       >       >       > system is not that complicated, I would
>       recommend
>       >       first
>       >       >       looking
>       >       >       >
>       >     
>        at https://github.com/rizo/awesome-ocaml#package-management
>       >       >       . IMHO we need
>       >       >       > to seriously look at consolidating efforts
>       around OPAM
>       >       for
>       >       >       package
>       >       >       > management, packaging, building, testing and
>       running.
>       >       All the
>       >       >       serious
>       >       >       > language-specific package managers do it, it's
>       a
>       >       proven
>       >       >       strategy and it
>       >       >       > simplifies life for the developer.
>       >       >
>       >       >       I find it odd that simplifying the life of the
>       developer
>       >       is the
>       >       >       highest
>       >       >       priority.  Doesn't one want to simplify the life
>       of the
>       >       user? 
>       >       >       Ideally the
>       >       >       user who has never touched OCaml before in his
>       life?
>       >       >
>       >       >       As a simple example, the web page for installing
>       OCaml
>       >       says that
>       >       >       the
>       >       >       recommended way to install ocaml is to install
>       opam. 
>       >       There is a
>       >       >       link to a
>       >       >       page explaining how to install opam.  How should
>       anyone
>       >       even
>       >       >       have
>       >       >       confidence that they will end up with OCaml
>       after
>       >       following
>       >       >       those
>       >       >       instructions?  Even step 1 of the installation
>       process
>       >       leads the
>       >       >       user to
>       >       >       confusion.
>       >       >
>       >       >       julia
>       >       >
>       >       >       >
>       >       >       > This could be a typical workflow:
>       >       >       >
>       >       >       > cd some-ocaml-proj
>       >       >       > opam install # Switches compiler if necessary
>       and
>       >       installs and
>       >       >       locally
>       >       >       > caches package dependencies
>       >       >       > opam build
>       >       >       > opam run # Automatically builds if necessary
>       >       >       > opam test # Ditto
>       >       >       > opam package # Ditto; --upload option can
>       immediately
>       >       upload
>       >       >       to opam
>       >       >       > opam doc # Builds documentation with
>       ocamldoc or
>       >       whatever
>       >       >       > opam login -u user -p password
>       >       >       >
>       >       >       > Regards,
>       >       >       >
>       >       >       > Yawar
>       >       >       >
>       >       >       > On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel
>       >       >       <oliver@first.in-berlin.de>
>       >       >       > wrote:
>       >       >       >       Hello,
>       >       >       >
>       >       >       >       a while ago it looked like there were
>       not enough
>       >       build-
>       >       >       and
>       >       >       >       installation-tools
>       >       >       >       for OCaml. I remember some discussions
>       about
>       >       that.
>       >       >       >
>       >       >       >       Now it seems to me that there are a lot
>       of them.
>       >       >       >       So, developers can pick the one they
>       know about.
>       >       >       >
>       >       >       >       For all these tools there might be good
>       reasons
>       >       to use
>       >       >       them, and
>       >       >       >       those
>       >       >       >       developers who looked at these tools and
>       choose
>       >       them for
>       >       >       their
>       >       >       >       projects, will
>       >       >       >       know them well enough.
>       >       >       >
>       >       >       >       The situation differs, if one wants to
>       package
>       >       the
>       >       >       written
>       >       >       >       software,
>       >       >       >       and one needs to know many of those
>       tools, just
>       >       to
>       >       >       compile the
>       >       >       >       stuff.
>       >       >       >       So, when one just wants to compile and
>       install
>       >       some
>       >       >       software,
>       >       >       >       just for that, it would take much effort
>       to
>       >       learn the
>       >       >       different
>       >       >       >       build-tools.
>       >       >       >
>       >       >       >       So, packaging has become more
>       complicated, even
>       >       though
>       >       >       for the
>       >       >       >       developers
>       >       >       >       these tools may save time.
>       >       >       >
>       >       >       >       It would be nice if people who used one
>       of the
>       >       many new
>       >       >       building
>       >       >       >       tools
>       >       >       >       could provide a Makefile that allows
>       just to
>       >       type
>       >       >       >       "make" and "make install", instead of
>       expecting
>       >       everyone
>       >       >       who
>       >       >       >       wants to compile
>       >       >       >       the software to first learn
>       >       just-another-build-tool.
>       >       >       >
>       >       >       >       Also it would be good, to mention early,
>       which
>       >       >       installation
>       >       >       >       tools (make-dependencies)
>       >       >       >       are in use, and too mention needed
>       packages
>       >       (opam or
>       >       >       others) to
>       >       >       >       just build the stuff.
>       >       >       >
>       >       >       >       Thanks and regards,
>       >       >       >         Oliver Bandel
>       >       >       >
>       >       >       >       --
>       >       >       >       Caml-list mailing list.  Subscription
>       management
>       >       and
>       >       >       archives:
>       >       >       >     
>        https://sympa.inria.fr/sympa/arc/caml-list
>       >       >       >       https://inbox.ocaml.org/caml-list
>       >       >       >       Forum: https://discuss.ocaml.org/
>       >       >       >       Bug reports:
>       http://caml.inria.fr/bin/caml-bugs
>       >       >       >
>       >       >       >
>       >       >       >
>       >       >
>       >       >       --
>       >       >       Caml-list mailing list.  Subscription management
>       and
>       >       archives:
>       >       >       https://sympa.inria.fr/sympa/arc/caml-list
>       >       >       https://inbox.ocaml.org/caml-list
>       >       >       Forum: https://discuss.ocaml.org/
>       >       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
>       >       >
>       >       >
>       >       >
>       >
>       >       --
>       >       Caml-list mailing list.  Subscription management and
>       archives:
>       >       https://sympa.inria.fr/sympa/arc/caml-list
>       >       https://inbox.ocaml.org/caml-list
>       >       Forum: https://discuss.ocaml.org/
>       >       Bug reports: http://caml.inria.fr/bin/caml-bugs
>       >
>       >
>       >
>
>       --
>       Caml-list mailing list.  Subscription management and archives:
>       https://sympa.inria.fr/sympa/arc/caml-list
>       https://inbox.ocaml.org/caml-list
>       Forum: https://discuss.ocaml.org/
>       Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[SP]

On 26/11/2018 21:29, Julia Lawall wrote:
> The problem is how many programming languages exist today.  Is it really
> realistic to expect users to learn how to use a different package manager
> for all of them?
Users of software written in OCaml don't need to install any compilers, OPAM, etc, since OCaml can be compiled into a static binary, which will depend on libraries found in most Linux environments.

Now if you are talking about developers needing to get tool-chains and libraries.. there is no clear answer. From different perspectives, the ideal answer is different.

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Malcolm Matalka]

SP <sp@orbitalfox.eu> writes:

> On 26/11/2018 21:29, Julia Lawall wrote:
>> The problem is how many programming languages exist today.  Is it really
>> realistic to expect users to learn how to use a different package manager
>> for all of them?
> Users of software written in OCaml don't need to install any compilers, OPAM, etc, since OCaml can be compiled into a static binary, which will depend on libraries found in most Linux environments.
>
> Now if you are talking about developers needing to get tool-chains and libraries.. there is no clear answer. From different perspectives, the ideal answer is different.

Is there a reasonable workflow for how to turn opam packages into
packages for existing OS's though?  Currently it does seem like users
need to know about Ocaml to use things written in Ocaml, if only because
most of the focus has been on opam.  Other OSs tend to have old
packages.

I just looked at opam-bundle and while the idea is great it is clearly
not much part of opam (it doesn't make use of my pins, or my local
repos, for example).  But tooling like that would be great to help get
packages out of opam and into OS's faster.

/Malcolm

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[SP]

On 27/11/2018 05:24, Malcolm Matalka wrote:
> Is there a reasonable workflow for how to turn opam packages into
> packages for existing OS's though?  Currently it does seem like users
> need to know about Ocaml to use things written in Ocaml, if only because
> most of the focus has been on opam.  Other OSs tend to have old
> packages.

For libraries, of course one needs to know about OCaml and the best path generally is to rely on OPAM rather than the OS's package manager. I used to think otherwise, but the reality is that there aren't that many OCamlers to have packagers for the repositories of even the most popular distros. This probably applies to other languages too, which leads to the point that it's best to use a language's package-manager for development purposes; higher odds of staying in a stable, reproducible and up to date environment.

For binaries and runtime dependencies, some packages provide Makefile files which wrap the Dune commands. OPAM is again required but a lot can be scripted or documented in a few lines of how to build the binary or dynamic library for distribution. For example: https://wiki.archlinux.org/index.php/OCaml_package_guidelines

Also consider https://appimage.org/ for cross-OS builds, so long they don't need to be installed in the system.

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Julia Lawall]

On Mon, 26 Nov 2018, SP wrote:

> On 26/11/2018 21:29, Julia Lawall wrote:
> > The problem is how many programming languages exist today.  Is it really
> > realistic to expect users to learn how to use a different package manager
> > for all of them?
> Users of software written in OCaml don't need to install any compilers, OPAM, etc, since OCaml can be compiled into a static binary, which will depend on libraries found in most Linux environments.
>
> Now if you are talking about developers needing to get tool-chains and libraries.. there is no clear answer. From different perspectives, the ideal answer is different.

Actually, I typically just post fixes to github and ask people to download
and compile the new version.  It's more practical than making a release
and binaries for each small change.

julia

>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[SF Markus Elfring]

> Actually, I typically just post fixes to github and ask people to download
> and compile the new version.  It's more practical than making a release
> and binaries for each small change.

Would you like to improve the support for publishing your software
in popular package formats?

Regards,
Markus

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[SP]

On 27/11/2018 06:11, Julia Lawall wrote:
> Actually, I typically just post fixes to github and ask people to download
> and compile the new version.  It's more practical than making a release
> and binaries for each small change.

Practical for the developer and certain groups (I don't know the specifics of your case). It is not practical in a general context to expect (power) users to install tool-chains and build dependencies to make use of applications.

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[SF Markus Elfring]

> Saying "the way to install ocaml is use opam" and then giving a link
> showing how to install opam with one's package manager is not a solution.

Which solution variants would you prefer instead for the management
of involved dependencies by software package formats?

Regards,
Markus

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Julia Lawall]

On Tue, 27 Nov 2018, SF Markus Elfring wrote:

> > Saying "the way to install ocaml is use opam" and then giving a link
> > showing how to install opam with one's package manager is not a solution.
>
> Which solution variants would you prefer instead for the management
> of involved dependencies by software package formats?

The question is what to do after installing opam.  Even if there is
nothing to do after installing opam, it would be helpful to say that.  If
there is something to do after installing opam to get the ocaml compiler
for example, as opposed to just the runtime system, it would be helpful to
say that too.  The documentation should be designed from the point of view
of the person who has never used ocaml or opam, never wants to use them
again, and has no contact with the ocaml community. Otherwise, people who
want to try some ocaml software, but are not actually forced to use it,
will just give up.

julia

Re: [Caml-list] Build-/Installation-Tools - not enough of them?

[SF Markus Elfring]

> The question is what to do after installing opam.

This is just another software package management system, isn't it?

It should generally ensure that required dependencies will be fulfilled
for the use of the provided contents.


> Otherwise, people who want to try some ocaml software,
> but are not actually forced to use it, will just give up.

I am not so concerned in this direction.

But there are the usual development challenges to consider around
the varying popularity of involved build tools and software products.
I am curious on how the situation will evolve further also for the management
of application binary interfaces.
Would you like to support collateral evolution any more here?

Regards,
Markus

Re: [Caml-list] Build-/Installation-Tools - not enough of them?

[Julia Lawall]

On Tue, 27 Nov 2018, SF Markus Elfring wrote:

> > The question is what to do after installing opam.
>
> This is just another software package management system, isn't it?

Sure, but each one may have its own particular commands.  One can't
spontaneously know what they are.  The commands required for installing
ocaml should be documented on the page for installing ocaml.  Installing
opam is recommended as one of them, but one has no idea what if anything
is needed after that.

julia

Re: [Caml-list] Build-/Installation-Tools - not enough of them?

[Jean-Francois Monin]

On Tue, Nov 27, 2018 at 11:34:41AM +0100, Julia Lawall wrote:
> 
> 
> On Tue, 27 Nov 2018, SF Markus Elfring wrote:
> 
> > > The question is what to do after installing opam.
> >
> > This is just another software package management system, isn't it?
> 
> Sure, but each one may have its own particular commands.  One can't
> spontaneously know what they are.  The commands required for installing
> ocaml should be documented on the page for installing ocaml.  Installing
> opam is recommended as one of them, but one has no idea what if anything
> is needed after that.
> 
> julia

I fully second Julia.
In order to teach OCaml, I need simple and clear instructions
for students.
The current situation is
  Search engine ->
   https://ocaml.org/docs/install.html ->
    https://opam.ocaml.org/doc/Install.html ->
     nightmare.

Too bad.

Jean-François

-- 
Jean-Francois Monin
Verimag
Resp. dpt Informatique Polytech Grenoble, 
Universite Grenoble Alpes

Re: [Caml-list] Build-/Installation-Tools - not enough of them?

[Kakadu]

> I fully second Julia.
> In order to teach OCaml, I need simple and clear instructions
> for students.
> The current situation is
>   Search engine ->
>    https://ocaml.org/docs/install.html ->
>     https://opam.ocaml.org/doc/Install.html ->
>      nightmare.
>

+1. Installing Opam 2.0 on Ubuntu is even more complicated (The only
trick I know is adding avsm's PPA which is not easy to google)

Happy hacking,
Kakadu

Re: [Caml-list] Build-/Installation-Tools - not enough of them?

[Malcolm Matalka]

Kakadu <kakadu.hafanana@gmail.com> writes:

>> I fully second Julia.
>> In order to teach OCaml, I need simple and clear instructions
>> for students.
>> The current situation is
>>   Search engine ->
>>    https://ocaml.org/docs/install.html ->
>>     https://opam.ocaml.org/doc/Install.html ->
>>      nightmare.
>>
>
> +1. Installing Opam 2.0 on Ubuntu is even more complicated (The only
> trick I know is adding avsm's PPA which is not easy to google)
>
> Happy hacking,
> Kakadu

I know at one point there was talk of opam2{rpm,pkg,deb} style tools.
Does anyone know the status of those?  I, personally, would benefit from
an opam2freebsdports.  ocaml-bundle came up earlier in the thread but as
I said it feels stapled onto opam and doesn't use the opam env I've
already setup (specifically pins + repos).

Maybe opam itself would benefit from a tool that could take a package
and produce the package information in a format easily consumable by
other languages so others can more easily write tooling around it?
Heck, that information would make it easier to writes something in
ocaml.  I know this is one challenge I have, I *could* write some tools
to make life easier but I don't have easy access to all the information
in opam packages and poking around the opam libraries to build such a
tool is too much effort for its priority for me.

Re: [Caml-list] Build-/Installation-Tools - not enough of them?

[Francois Berenger]

On 27/11/2018 22:18, Malcolm Matalka wrote:
> Kakadu <kakadu.hafanana@gmail.com> writes:
> 
>>> I fully second Julia.
>>> In order to teach OCaml, I need simple and clear instructions
>>> for students.
>>> The current situation is
>>>   Search engine ->
>>>    https://ocaml.org/docs/install.html ->
>>>     https://opam.ocaml.org/doc/Install.html ->
>>>      nightmare.
>>> 
>> 
>> +1. Installing Opam 2.0 on Ubuntu is even more complicated (The only
>> trick I know is adding avsm's PPA which is not easy to google)
>> 
>> Happy hacking,
>> Kakadu
> 
> I know at one point there was talk of opam2{rpm,pkg,deb} style tools.
> Does anyone know the status of those?  I, personally, would benefit 
> from
> an opam2freebsdports.  ocaml-bundle came up earlier in the thread but 
> as
> I said it feels stapled onto opam and doesn't use the opam env I've
> already setup (specifically pins + repos).

If you can build from sources, there is fpm that can create a deb or 
rpm:

https://github.com/jordansissel/fpm

That's not a silver bullet, and it's not even specialized at ocaml 
software/opam pacakges, but sometimes it helps.

> Maybe opam itself would benefit from a tool that could take a package
> and produce the package information in a format easily consumable by
> other languages so others can more easily write tooling around it?
> Heck, that information would make it easier to writes something in
> ocaml.  I know this is one challenge I have, I *could* write some tools
> to make life easier but I don't have easy access to all the information
> in opam packages and poking around the opam libraries to build such a
> tool is too much effort for its priority for me.

Re: [Caml-list] Build-/Installation-Tools - not enough of them?

[Ian Zimmerman]

On 2018-11-27 14:00, Kakadu wrote:

> +1. Installing Opam 2.0 on Ubuntu is even more complicated (The only
> trick I know is adding avsm's PPA which is not easy to google)

FWIW, Fedora 28 (which is now 1 version behind "stable") packages opam2
natively.  This has been the easiest setting to manage Ocaml based
software ever.

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Jean-Marc Alliot]

Le 27/11/2018 à 11:08, Julia Lawall a écrit :
>
> On Tue, 27 Nov 2018, SF Markus Elfring wrote:
>
>>> Saying "the way to install ocaml is use opam" and then giving a link
>>> showing how to install opam with one's package manager is not a solution.
>> Which solution variants would you prefer instead for the management
>> of involved dependencies by software package formats?
> The question is what to do after installing opam.  Even if there is
> nothing to do after installing opam, it would be helpful to say that.  If
> there is something to do after installing opam to get the ocaml compiler
> for example, as opposed to just the runtime system, it would be helpful to
> say that too.  The documentation should be designed from the point of view
> of the person who has never used ocaml or opam, never wants to use them
> again, and has no contact with the ocaml community. Otherwise, people who
> want to try some ocaml software, but are not actually forced to use it,
> will just give up.
>
> julia
>
+1

Even if I have been using ocaml/caml-light for almost 30 years now, it 
took me a while to switch to opam (I only did that something like 2 
years ago), because I found it easier to use the linux distribution 
package manager, and didn't like much to have to learn a new tool. It is 
also possible that it took me so long because I was used to compile 
myself from source the packages that I needed when they were not in the 
distro.

opam is a really fine tool, but the documentation is "scarse", and 
sometimes looks like it is meant for people knowing opam already. It's 
no big deal as soon as you really need to use it (in the end, compiling 
packages from source had become somewhat nightmarish, so speending some 
time to learn opam was a reasonable decision), but its documentation 
could easily deter someone who is just passing by.

It's not a criticism in any way to the people who have been working on 
opam, who I thank for they excellent work, just the expression of a 
feeling from a happy user of ocaml who would like to see more people use it.

Jean-Marc

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Richard W.M. Jones]

On Mon, Nov 26, 2018 at 04:19:34PM -0500, Yawar Amin wrote:
> Realistically, today, system ocaml compiler + make is a corner case in
> OCaml development/distribution and shouldn't take precedence over a
> standardized developer workflow with opam.

Actual figures, otherwise I'll have to assume you're talking nonsense.
The system OCaml compiler is by far the way we prefer users to consume
OCaml and programs built by OCaml on Fedora, probably on Debian too.

Rich.

-- 
Richard Jones

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Yawar Amin]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

I think there has been a communication gap here, so let me expand: opam is
the _intended_ main tool for development of OCaml programs and the related
distribution of the OCaml compiler. You are talking about end users
consuming OCaml programs, I am talking about developers making them, two
different use cases.

Yawar

On Thu, Dec 6, 2018 at 7:21 AM Richard W.M. Jones <rich@annexia.org> wrote:

> On Mon, Nov 26, 2018 at 04:19:34PM -0500, Yawar Amin wrote:
> > Realistically, today, system ocaml compiler + make is a corner case in
> > OCaml development/distribution and shouldn't take precedence over a
> > standardized developer workflow with opam.
>
> Actual figures, otherwise I'll have to assume you're talking nonsense.
> The system OCaml compiler is by far the way we prefer users to consume
> OCaml and programs built by OCaml on Fedora, probably on Debian too.
>
> Rich.
>
> --
> Richard Jones
>

[-- Attachment #2: Type: text/html, Size: 1242 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Richard W.M. Jones]

On Thu, Dec 06, 2018 at 11:10:45AM -0500, Yawar Amin wrote:
> I think there has been a communication gap here, so let me expand: opam is
> the _intended_ main tool for development of OCaml programs and the related
> distribution of the OCaml compiler. You are talking about end users
> consuming OCaml programs, I am talking about developers making them, two
> different use cases.

There's not much different.  Who here routinely compiles their own GCC?

The problem here is you've gone and invented a second package manager.

Linux distros already have a package manager, used by users and
developers alike.  The package manager solves a whole lot of stuff
like maintaining the whole system, dependencies, releases, security updates,
mirroring and distribution, etc.  Having a second one (and let's face
it if you're using things like npm, cargo, etc., a third, fourth)
complicates everything.

However there is likely a good answer here: Tooling to turn opam
packages into RPMs, Debian packages, etc.

Rich.

-- 
Richard Jones

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Daniel Bünzli]

On 6 December 2018 at 21:18:23, Richard W.M. Jones (rich@annexia.org) wrote:

> The problem here is you've gone and invented a second package manager. 

The problem here is rather that the fragmentation around system package managers renders the development of a consistent and cross-platform package set like is being done in the opam-repository largely undoable without an indirection -- the latter materializing as these nth (sadly usually language specific) package managers.

> Linux distros already have a package manager, used by users and
> developers alike. The package manager solves a whole lot of stuff
> like maintaining the whole system, dependencies, releases, security updates,
> mirroring and distribution, etc. Having a second one (and let's face
> it if you're using things like npm, cargo, etc., a third, fourth)
> complicates everything.

It surely does. 

I suspect that if linux distributions had agreed on a single package manager (or even metadata format and package **names**), if this one had added a few developer friendly features (e.g. universes to easily distinguish what you want to compile a project from what you want for your system) and if it had been ported to other platforms we wouldn't need all these language specific package managers. But that's a lot of unfufilled ifs...

The fact that opam v2 became language agnostic largely supports this point. In some of my projects that's the actual package manager I'm using as my integration layer for C packages.

> However there is likely a good answer here: Tooling to turn opam 
> packages into RPMs, Debian packages, etc.

Maybe we should do this the other way round ;-)

Best, 

Daniel

[Caml-list] What happened to the 'ancient' library for OCaml?

[Francois Berenger]

Dear all,

Synopsis of ancient:
"Allows to use in-memory data structures which are larger than available 
memory and so are
kept in swap"

The ancient library in opam is constrained to ocaml < 4.03.0.

Was this library abandoned?

Was it superseded by something else?
I think I saw this somewhere but I cannot find it back.

Also, some questions regarding performance:
- was moving a datastructure to ancient faster than marshalling?
- was reading a datastructure from ancient faster than unmarshalling?

Thanks a lot,
F.

Re: [Caml-list] What happened to the 'ancient' library for OCaml?

[Richard W.M. Jones]

On Fri, Dec 07, 2018 at 04:44:46PM +0900, Francois Berenger wrote:
> Dear all,
> 
> Synopsis of ancient:
> "Allows to use in-memory data structures which are larger than available
> memory and so are
> kept in swap"
> 
> The ancient library in opam is constrained to ocaml < 4.03.0.

It should work on more recent versions of OCaml as far as I know.
However ...

> Was this library abandoned?

... yes it's not being developed any longer.  It needs some attention.

> Was it superseded by something else?
> I think I saw this somewhere but I cannot find it back.
> 
> Also, some questions regarding performance:
> - was moving a datastructure to ancient faster than marshalling?
> - was reading a datastructure from ancient faster than unmarshalling?

It involves a single copy, so at least in theory yes ancient should be
faster than marshalling.

However that's not the reason to use ancient - it's so you can process
very large datasets (larger than available RAM) using OCaml code,
which is something that can't be done very easily on marshalled data.

Rich.

> Thanks a lot,
> F.
> 
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs

-- 
Richard Jones

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Richard W.M. Jones]

On Fri, Dec 07, 2018 at 08:31:01AM +0100, Daniel Bünzli wrote:
> On 6 December 2018 at 21:18:23, Richard W.M. Jones (rich@annexia.org) wrote:
> 
> > The problem here is you've gone and invented a second package manager. 
> 
> The problem here is rather that the fragmentation around system package managers renders the development of a consistent and cross-platform package set like is being done in the opam-repository largely undoable without an indirection -- the latter materializing as these nth (sadly usually language specific) package managers.
> 
> > Linux distros already have a package manager, used by users and
> > developers alike. The package manager solves a whole lot of stuff
> > like maintaining the whole system, dependencies, releases, security updates,
> > mirroring and distribution, etc. Having a second one (and let's face
> > it if you're using things like npm, cargo, etc., a third, fourth)
> > complicates everything.
> 
> It surely does. 
> 
> I suspect that if linux distributions had agreed on a single package manager (or even metadata format and package **names**), if this one had added a few developer friendly features (e.g. universes to easily distinguish what you want to compile a project from what you want for your system) and if it had been ported to other platforms we wouldn't need all these language specific package managers. But that's a lot of unfufilled ifs...
>
> The fact that opam v2 became language agnostic largely supports this point. In some of my projects that's the actual package manager I'm using as my integration layer for C packages.
> 
> > However there is likely a good answer here: Tooling to turn opam 
> > packages into RPMs, Debian packages, etc.
> 
> Maybe we should do this the other way round ;-)

There are already efforts to replace all Linux distro package
managers, but they don't have much traction and there's no particular
reason to think that opam is in a good position either, especially
since it does about 5% of what a Linux distro package manager can do.

Most people who want per-user packaging are now using containers,
using the existing Linux distro package managers.

Anyway, tools to take opam metadata and convert it to spec files
and debian/rules are welcome.

Rich.

-- 
Richard Jones

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Daniel Bünzli]

On 7 December 2018 at 09:26:51, Richard W.M. Jones (rich@annexia.org) wrote:

> Most people who want per-user packaging are now using containers,
> using the existing Linux distro package managers.

I hope you realize this still doesn't solve the initial problem I mentioned (and is rather painful to work with as a developer). I'm afraid if system packagers would like their systems to be used for development they will need to focus quite a bit more on the developer user experience and interoperability. I'd be glad to see this happening but I'm afraid it's not.

Best,

Daniel

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Stéphane Glondu]

Le 07/12/2018 à 08:31, Daniel Bünzli a écrit :
>> The problem here is you've gone and invented a second package manager. 
> 
> The problem here is rather that the fragmentation around system package managers renders the development of a consistent and cross-platform package set like is being done in the opam-repository largely undoable without an indirection -- the latter materializing as these nth (sadly usually language specific) package managers.
> 
>> Linux distros already have a package manager, used by users and
>> developers alike. The package manager solves a whole lot of stuff
>> like maintaining the whole system, dependencies, releases, security updates,
>> mirroring and distribution, etc. Having a second one (and let's face
>> it if you're using things like npm, cargo, etc., a third, fourth)
>> complicates everything.
> 
> It surely does. 
> 
> I suspect that if linux distributions had agreed on a single package manager (or even metadata format and package **names**), if this one had added a few developer friendly features (e.g. universes to easily distinguish what you want to compile a project from what you want for your system) and if it had been ported to other platforms we wouldn't need all these language specific package managers. But that's a lot of unfufilled ifs...

If everybody had agreed on a single language, we wouldn't need all these
language specific package managers either :-) Replace "language" by
"operating system" or "platform" or ...

Each package manager has its own set of features ("developer
friendly"-ness is just a matter of taste IMHO)... You sound like system
package managers have no features, but the example you give
("distinguish what you want to compile a project from what you want for
your system") is supported by the Debian toolchain (and I guess others).
I haven't been very much impressed by opam (I mean, the package manager
itself) so far, actually there are basic features that are missing (or
not obvious) in opam (such as redistributable binary packages).

> The fact that opam v2 became language agnostic largely supports this point. In some of my projects that's the actual package manager I'm using as my integration layer for C packages.

The mere fact that it is written in OCaml will put many people off, in
particular those who have strong opinions on languages. I don't believe
the existence of a "single package manager" as you hypothesized would
have prevented the proliferation of language specific package managers.

-- 
Stéphane

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Daniel Bünzli]

 On 7 December 2018 at 14:22:06, Stéphane Glondu (steph@glondu.net) wrote:

> If everybody had agreed on a single language, we wouldn't need all these
> language specific package managers either :-) Replace "language" by
> "operating system" or "platform" or ...

This comparison is entirely inadequate. Distribution tarballs are distribution tarballs. The fact that linux distros/bsds/whatever are not even able to agree on a single name and version number for the same tarballs and that we need to lose our time to maintain mappings on how each random distribution names the same fucking software artefact should be a hint that this is *not* about agreeing on a single language and platform but rather a problem in consistent human stupidity.

> Each package manager has its own set of features ("developer
> friendly"-ness is just a matter of taste IMHO)... You sound like system
> package managers have no features, but the example you give
> ("distinguish what you want to compile a project from what you want for
> your system") is supported by the Debian toolchain (and I guess others).

This obsession about features is quite interesting, always this myth about the more features the better... Maybe opam does only the 5% of your truly awesome system package managers but it does so in a relatively lean way, that is adequate for development and allows to define and maintain (at the risk of repeating myself) packages universes accross your pet distribution and/or operating system. 

You are all most welcome to live in your distro and package manager bubbles and idiosyncrasies but don't start claiming they are able to provide what opam and OCaml opam repository currently provide because they simply do not.

> I haven't been very much impressed by opam (I mean, the package manager
> itself) so far, actually there are basic features that are missing (or
> not obvious) in opam (such as redistributable binary packages).

Opam is far from perfect and I have many concerns about its current usability. But the basic feature you mention is missing is absolutely unuseful to solve the problem opam is solving right now (see above in case that is still not clear).

> I don't believe the existence of a "single package manager" as you hypothesized would
> have prevented the proliferation of language specific package managers.

I gave one explanation, I don't claim that this is it, but at least provide a competing one. So far you did not so maybe you should try to reflect a bit more on the problem. I don't think anyone here would be sad to see language specific package managers go and frankly I'm pretty sure both opam developers and opam repository maintainers would be glad to spend their time on more fun projects.

Best,

Daniel


 

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[SP]

Reading this discussion inspired me to write this:

https://notes.orbitalfox.eu/distributing-software.html

I hope it helps. It doesn't cover all the points raised here.
Any constructive remarks are welcome.

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Francois Berenger]

On 07/12/2018 22:22, Stéphane Glondu wrote:
> Le 07/12/2018 à 08:31, Daniel Bünzli a écrit :
>>> The problem here is you've gone and invented a second package 
>>> manager. 
>> 
>> The problem here is rather that the fragmentation around system 
>> package managers renders the development of a consistent and 
>> cross-platform package set like is being done in the opam-repository 
>> largely undoable without an indirection -- the latter materializing as 
>> these nth (sadly usually language specific) package managers.
>> 
>>> Linux distros already have a package manager, used by users and
>>> developers alike. The package manager solves a whole lot of stuff
>>> like maintaining the whole system, dependencies, releases, security 
>>> updates,
>>> mirroring and distribution, etc. Having a second one (and let's face
>>> it if you're using things like npm, cargo, etc., a third, fourth)
>>> complicates everything.
>> 
>> It surely does. 
>> 
>> I suspect that if linux distributions had agreed on a single package 
>> manager (or even metadata format and package **names**), if this one 
>> had added a few developer friendly features (e.g. universes to easily 
>> distinguish what you want to compile a project from what you want for 
>> your system) and if it had been ported to other platforms we wouldn't 
>> need all these language specific package managers. But that's a lot of 
>> unfufilled ifs...
> 
> If everybody had agreed on a single language, we wouldn't need all 
> these
> language specific package managers either :-) Replace "language" by
> "operating system" or "platform" or ...
> 
> Each package manager has its own set of features ("developer
> friendly"-ness is just a matter of taste IMHO)... You sound like system
> package managers have no features, but the example you give
> ("distinguish what you want to compile a project from what you want for
> your system") is supported by the Debian toolchain (and I guess 
> others).
> I haven't been very much impressed by opam (I mean, the package manager
> itself) so far, actually there are basic features that are missing (or
> not obvious) in opam (such as redistributable binary packages).

Hello,

I asked a long time ago for this feature in opam:

"support for binary packages"
https://github.com/ocaml/opam/issues/1159

Note that, as Richard just suggested, if opam was able to synthetize
package descriptions for rpm/deb, we would gain binary packages for 
free.

Regards,
F.

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Malcolm Matalka]

Richard W.M. Jones <rich@annexia.org> writes:

> On Thu, Dec 06, 2018 at 11:10:45AM -0500, Yawar Amin wrote:
>> I think there has been a communication gap here, so let me expand: opam is
>> the _intended_ main tool for development of OCaml programs and the related
>> distribution of the OCaml compiler. You are talking about end users
>> consuming OCaml programs, I am talking about developers making them, two
>> different use cases.
>
> There's not much different.  Who here routinely compiles their own GCC?
>
> The problem here is you've gone and invented a second package manager.
>
> Linux distros already have a package manager, used by users and
> developers alike.  The package manager solves a whole lot of stuff
> like maintaining the whole system, dependencies, releases, security updates,
> mirroring and distribution, etc.  Having a second one (and let's face
> it if you're using things like npm, cargo, etc., a third, fourth)
> complicates everything.
>
> However there is likely a good answer here: Tooling to turn opam
> packages into RPMs, Debian packages, etc.
>
> Rich.
>
> --
> Richard Jones

Apologies, the following is not very constructive, just my own gripe: I
really wish that Opam has based itself on existing package manager
instead of building a new one from scratch.  And I really wish the
universe would standardize on 'nix' since that that seems to solve a lot
of problems people run into (I say this as a FreeBSD user utterly
annoyed with the existence of pkg, despite it being a step up in FreeBSD
world).  That being said, the nix community hasn't done a good job of
making their package manager a good foundation for other package
managers.  The Haskell package manager has gone this route, which I
think is really great.

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Anil Madhavapeddy]

[-- Attachment #1: Type: text/plain, Size: 7201 bytes --]


> On 26 Nov 2018, at 20:54, Julia Lawall <julia.lawall@lip6.fr> wrote:
> 
> Personally, I was in the end forced to install opam.  But I don't want to
> force my users to install it (to figure out how to get it to work, I had
> to contact a member of Gallium - and my users don't have that
> opportunity).  I would prefer that they can just use the ocaml that comes
> with their system package manager.
> 
> Since opam is the recommended method of installation, couldn't there at
> least be provided understandable instructions?  That really seems like a
> tougher time than what is necessary.
> 

Dear Julia, dear all,

Thanks for the helpful reminder that the opam and ocaml documentation
needs a refresh.  We’ve just gone through a fairly major release cycle with
opam2 and dune1, and it’s to be expected that there are some inconsistencies.

To clear up one misconception on this thread — opam is *not* the only
recommended mechanism to get the OCaml compiler.  We go to some
trouble to ensure that OCaml is packaged up natively on the OS package
managers where possible, and there is a list on the ocaml.org installation
page with popular distributions.

The problems begin when the user needs a particular version of OCaml,
which is a common requirement for teaching.  The system package 
managers typically lag in their versions (again, listed on the ocaml.org
installation page).  At this point, opam is indeed the simplest tool to bootstrap
the precise version of the compiler and packages that you require, but
at the expense of requiring an OCaml-specific tool.

Now, I recognise the need for improving the installation instructions,
and particularly so with step-by-step guides to getting things installed.
However, I would implore those with complaints to register the inaccuracies
on the opam issue tracker [1], so that we can at least learn about them
and fix them just as we do with any software bug.

[1] https://github.com/ocaml/opam/issues

It’s particularly frustrating to see these threads erupt with pent up
(and valid) annoyances from users that we could have addressed if someone
had reported it. Even better, please do send in PRs with some suggested
text for the website, and take some load off our overworked maintainers :-)

For whoever said that my Ubuntu opam2 PPA is hard to Google, 
the process of creating it was roughly:

- spend a few hours fighting with the Debian packaging scripts
- get x86_64/i386/arm32/arm64/ppc64le all working in Launchpad
- post on https://discuss.ocaml.org/t/opam-2-0-experimental-ppas/2446 to get no feedback whatsoever
- eventually just push it live after one user gets back with feedback
- wait for complaints to show up that its hard to find

Without feedback, it’s very difficult to know what to prioritise, and
Ubuntu is just one of many, many Linux distributions that we have to
worry about.

Japp Boender also wrote:
> I'd like to chime in here - as the person responsible for doing most of the OCaml packaging for pkgsrc, I've noticed that I can hardly ever just update a package to a new version anymore - I'll generally also need to update the infrastructure as some sort of new build tool will have become involved, with all its attendant quirks, or someone will have decided to change the name of packages, or some other change will have occurred. Things seem to be in a constant state of flux, and thus packaging becomes quite time-consuming.

This is indeed the result of progress on the tooling side.  There’s been
a lot of work in the last few years on deprecating older tools (camlp4,
ocamlbuild) and moving towards modern alternatives such as dune
for build and opam2 for publishing.

Things are settling down now: dune is emerging as a well-maintained
and portable alternative to the myriad of build systems that have been
proposed in the past.  The bootstrapping problems you mention should
also have become a *lot* easier with opam2, since the solver is linked
into the binary and no external aspcud is needed.  If you do still run
into any problems with fakeroots and such, please report them on the
dune or opam issue tracker and we’ll take a look.

Behind the scenes, there are other benefits from a consistent build
system: cross compilation and Windows support are greatly improved
now.

Malcolm Matalka also wrote:
> Is there a reasonable workflow for how to turn opam packages into
> packages for existing OS's though?  Currently it does seem like users
> need to know about Ocaml to use things written in Ocaml, if only because
> most of the focus has been on opam.  Other OSs tend to have old
> packages.

Our goal (with my dune and opam developer hat on) is to provide sufficient
metadata within a dune description of a project to automatically generate
opam *and* upstreamable OS packages (from templates).  This will
look something like “dune @package” and really reduce the burden on
developers and OS maintainers.  I’d like, for example, the native OpenBSD
packages to be able to be replaced by a mechanically generated version.

See this issue for more: https://github.com/ocaml/dune/issues/1498

Julia Lawall also wrote:
> The question is what to do after installing opam.  Even if there is
> nothing to do after installing opam, it would be helpful to say that.  If
> there is something to do after installing opam to get the ocaml compiler
> for example, as opposed to just the runtime system, it would be helpful to
> say that too.  The documentation should be designed from the point of view
> of the person who has never used ocaml or opam, never wants to use them
> again, and has no contact with the ocaml community. Otherwise, people who
> want to try some ocaml software, but are not actually forced to use it,
> will just give up.

I completely agree with this sentiment.  I’m personally motivated to
make sure ocaml/opam meet these standards, as we are moving to
teaching using OCaml in the Cambridge University Computer Lab
from next year, so I’ll definitely be proposing some changes myself :-)

I’d love to hear from other teachers about the sort of environments you
have to work in — we’ve been considering shifting to a combination of
browser-based IDEs (for homogeneity across student laptops) and
other systems.

Finally, this thread might seem like it is full of complaints, but the tooling
is steadily improving and leaving behind silent and satisfied users.
Dune in particular has made as big a difference to our community as
opam did when it was first released five years ago.  At ICFP this year,
the OCaml Workshop was full of industrial users who were thankful for
Dune improving their day-to-day development, as well as the ecosystem
of new tools such as ocamlformat, merlin, odoc and so on, and the Coq
project is moving to adopt it now as well!  I’m not suggesting we rest on
our laurels, but as the year end holiday approaches, I would be delighted
if users to email their favourite tool developer a note of encouragement to
continue to work on it.  And then file an issue to get the docs fixed :-)

regards,
Anil

[-- Attachment #2: Type: text/html, Size: 16392 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Gerd Stolpmann]

[-- Attachment #1.1: Type: text/plain, Size: 4766 bytes --]

I'd recommend to be pragmatic. Making it simple for the developer and
for the end user are not goals that exclude each other. For example, we
have here a mixed development team - only a few develop in OCaml, and
the others are just consumers. The solution was to provide them a script
that installs OCaml the recommended way for all the setups we need to
consider. That script profits enormously from the capabilities of opam,
and exploits these to make life simple for the consumers.

Also, when I compare opam with other language-specific package managers
(say, npm) there is no huge difference for the users. Once you get used
to once of these managers, switching to another one is no big hurdle
anymore. I wouldn't be concerned that much.

Gerd

Am 26.11.18 um 17:57 schrieb Julia Lawall:
>
> On Mon, 26 Nov 2018, Yawar Amin wrote:
>
>> If anyone would like to chime in and say that OCaml build and packaging
>> system is not that complicated, I would recommend first looking
>> at https://github.com/rizo/awesome-ocaml#package-management . IMHO we need
>> to seriously look at consolidating efforts around OPAM for package
>> management, packaging, building, testing and running. All the serious
>> language-specific package managers do it, it's a proven strategy and it
>> simplifies life for the developer.
> I find it odd that simplifying the life of the developer is the highest
> priority.  Doesn't one want to simplify the life of the user?  Ideally the
> user who has never touched OCaml before in his life?
>
> As a simple example, the web page for installing OCaml says that the
> recommended way to install ocaml is to install opam.  There is a link to a
> page explaining how to install opam.  How should anyone even have
> confidence that they will end up with OCaml after following those
> instructions?  Even step 1 of the installation process leads the user to
> confusion.
>
> julia
>
>> This could be a typical workflow:
>>
>> cd some-ocaml-proj
>> opam install # Switches compiler if necessary and installs and locally
>> caches package dependencies
>> opam build
>> opam run # Automatically builds if necessary
>> opam test # Ditto
>> opam package # Ditto; --upload option can immediately upload to opam
>> opam doc # Builds documentation with ocamldoc or whatever
>> opam login -u user -p password
>>
>> Regards,
>>
>> Yawar
>>
>> On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel <oliver@first.in-berlin.de>
>> wrote:
>>       Hello,
>>
>>       a while ago it looked like there were not enough build- and
>>       installation-tools
>>       for OCaml. I remember some discussions about that.
>>
>>       Now it seems to me that there are a lot of them.
>>       So, developers can pick the one they know about.
>>
>>       For all these tools there might be good reasons to use them, and
>>       those
>>       developers who looked at these tools and choose them for their
>>       projects, will
>>       know them well enough.
>>
>>       The situation differs, if one wants to package the written
>>       software,
>>       and one needs to know many of those tools, just to compile the
>>       stuff.
>>       So, when one just wants to compile and install some software,
>>       just for that, it would take much effort to learn the different
>>       build-tools.
>>
>>       So, packaging has become more complicated, even though for the
>>       developers
>>       these tools may save time.
>>
>>       It would be nice if people who used one of the many new building
>>       tools
>>       could provide a Makefile that allows just to type
>>       "make" and "make install", instead of expecting everyone who
>>       wants to compile
>>       the software to first learn just-another-build-tool.
>>
>>       Also it would be good, to mention early, which installation
>>       tools (make-dependencies)
>>       are in use, and too mention needed packages (opam or others) to
>>       just build the stuff.
>>
>>       Thanks and regards,
>>         Oliver Bandel
>>
>>       --
>>       Caml-list mailing list.  Subscription management and archives:
>>       https://sympa.inria.fr/sympa/arc/caml-list
>>       https://inbox.ocaml.org/caml-list
>>       Forum: https://discuss.ocaml.org/
>>       Bug reports: http://caml.inria.fr/bin/caml-bugs
>>
>>
>>
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Louis Gesbert]

[-- Attachment #1: Type: text/plain, Size: 9128 bytes --]

Thanks all for this input. While working on the intrinsics and details of the tools, it's easy to lose the big picture, and the very important point of view of the newcomers.

So first, as the main developper of opam 2.0, I'd like to say that we have been putting a lot of work in it, and a large part of the effort was to improve for convenience, and the many use-cases that weren't supported before — including, but not limited to, reproducible build environments, and project-local sandboxes, a.k.a. switches.

The documentation, however, is severly lacking at the moment on all these new features, and the preferred and simplest ways to accomplish all the basic tasks. For all asking about the detailed formats, we have a fairly complete manual [1], and the API should be fairly well documented [2], but indeed, it's way too detailed to be the first documentation you would get to.

Let me assure you however that everything is slowly getting into place for an easier approach for everyone. I'll go below through the "typical workflow" you propose, checking what is here or not, but let's first focus on the users rather than the developers:

- installing ocaml:
  indeed, if not easily available for your system, the easiest is to install opam, then just run `opam init`. (Yes, we should be explicit in the doc that _this_ is the command that will install the compiler)

- installing a given package, and assuming opam is installed and initialised:

   * if in the repository, just one `opam install` command should be all you need

   * otherwise, if your source is available somewhere and contains an opam package definition file, `opam pin URL` is everything you need (URL pointing to an archive, or a git repository, etc.). We could merge this use-case into `opam install` for better discoverability.

   * if not and/or you want to build the project manually from a clone, the support has been much improved in opam 2, so that you can for example document specific pinned dependencies, or a "locked" development state (see opam-lock [3] to do that automatically). Then e.g. `opam switch create . --locked` will recreate a local switch with the exact same development configuration, and install the project in it. `opam install . --locked` also works, if you don't want the local sandbox.

- it has been mentionned already in this thread, but the `opam bundle` plugin can make distribution easier by including the whole OCaml + opam + package environment in a single, self-building self-extracting archive. At the cost of rebuilding everything on installation. A new release is pending [4]. Yes, it's yet another tool, but with its straight-forward interface and everyting explained in its 100-lines, included manpage, I find the criticism uncalled for. Not a silver bullet by any means, but it fits some use-cases.

As for using wrapping Makefiles, they are nice for simple build-system calls, and I like them if only to document the entry-point, but shouldn't IMHO mess with the packaging system. Note also that the main purpose of `opam` files is actually to document the building commands of any project, taking into account all OS specificities, and in an easily understandable format. I personally find that having clear and simple build instructions arout the top of the README is enough.

Once properly documented from the opam side (huh), I expect project maintainers to be able to put simple setup + installation instructions at the top of their READMEs, so that users who don't care about OCaml or opam just need to copy-paste them to get the environment setup and the project compiled. As far as I understand it now, this is where the problem really stands. To avoid having to look anything up or learning about exotic build system, this is the best compromise IMHO.

I'd also like to point out that this is not specific to OCaml, and I believe all language package managers / build systems suffer from these issues: I for one struggle every time I have to use something building with NPM, and they don't generally provide Makefiles. Of course, with a tool as popular as NPM, the problem is less visible because you have to go through it anyway. So we do need to improve documentation and simplify basic workflows as much as possible, but expecting people to work with OCaml without learning any of the tooling is unrealistic (unless they stick to an online IDE or e.g. Learn-OCaml, and even that is tooling in some form).


Let me now go through your "typical workflow":

---

> cd some-ocaml-proj
> opam install # Switches compiler if necessary and installs and locally
> caches package dependencies

You can do this with `opam switch create .`
Since "if necessary" is pretty subjective, just run `opam install .` if you prefer to share the environment with other projects.

> opam build

`opam install .`

> opam run # Automatically builds if necessary

there is no package←→executable bijection, so I don't see how this would work? (same as for OS-level packages)
see below, but this might be `dune exec <command>`

> opam test # Ditto

indeed here we enter the domain where the separation between build system and packaging system can hurt. You can run `opam install . --with-test`, but probably want `dune runtest` instead.

> opam package # Ditto; --upload option can immediately upload to opam

at this point you must already have a package definition available ? Or do you mean creating a release archive ?
If your source is hosted on Github, you only need to push a tag and run `opam publish` (you otherwise need to provide an URL for the source archive and that's it).

> opam doc # Builds documentation with ocamldoc or whatever
> opam login -u user -p password

I am not sure what you have in mind here. `opam publish` will go through Oauth authentification with Github for submitting your new package.

---


As one last note, let me mention that we are right now discussing:
- better integration of opam and dune
- integration of system dependency handling ("depexts") into opam


Hope this helps, feedback and questions welcome.

Louis Gesbert — OCamlPro




[1] https://opam.ocaml.org/doc/Manual.html
[2] https://opam.ocaml.org/doc/api/index.html
[3] https://github.com/AltGr/opam-lock
[4] https://github.com/ocaml/opam-repository/pull/13064

> - Yawar Amin, 26/11/2018 11:41 -
> If anyone would like to chime in and say that OCaml build and packaging
> system is not that complicated, I would recommend first looking at
> https://github.com/rizo/awesome-ocaml#package-management . IMHO we need to
> seriously look at consolidating efforts around OPAM for package management,
> packaging, building, testing and running. All the serious language-specific
> package managers do it, it's a proven strategy and it simplifies life for
> the developer.
> 
> This could be a typical workflow:
> 
> cd some-ocaml-proj
> opam install # Switches compiler if necessary and installs and locally
> caches package dependencies
> opam build
> opam run # Automatically builds if necessary
> opam test # Ditto
> opam package # Ditto; --upload option can immediately upload to opam
> opam doc # Builds documentation with ocamldoc or whatever
> opam login -u user -p password
> 
> Regards,
> 
> Yawar
> 
> On Mon, Nov 26, 2018 at 5:15 AM Oliver Bandel <oliver@first.in-berlin.de>
> wrote:
> 
> > Hello,
> >
> > a while ago it looked like there were not enough build- and
> > installation-tools
> > for OCaml. I remember some discussions about that.
> >
> > Now it seems to me that there are a lot of them.
> > So, developers can pick the one they know about.
> >
> > For all these tools there might be good reasons to use them, and those
> > developers who looked at these tools and choose them for their projects,
> > will
> > know them well enough.
> >
> > The situation differs, if one wants to package the written software,
> > and one needs to know many of those tools, just to compile the stuff.
> > So, when one just wants to compile and install some software,
> > just for that, it would take much effort to learn the different
> > build-tools.
> >
> > So, packaging has become more complicated, even though for the developers
> > these tools may save time.
> >
> > It would be nice if people who used one of the many new building tools
> > could provide a Makefile that allows just to type
> > "make" and "make install", instead of expecting everyone who wants to
> > compile
> > the software to first learn just-another-build-tool.
> >
> > Also it would be good, to mention early, which installation tools
> > (make-dependencies)
> > are in use, and too mention needed packages (opam or others) to just build
> > the stuff.
> >
> > Thanks and regards,
> >   Oliver Bandel
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > https://inbox.ocaml.org/caml-list
> > Forum: https://discuss.ocaml.org/
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> 
> 

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Jaap Boender]

On 26/11/2018 10:14, Oliver Bandel wrote:
> The situation differs, if one wants to package the written software,
> and one needs to know many of those tools, just to compile the stuff.
> So, when one just wants to compile and install some software,
> just for that, it would take much effort to learn the different build-tools.

I'd like to chime in here - as the person responsible for doing most of 
the OCaml packaging for pkgsrc, I've noticed that I can hardly ever just 
update a package to a new version anymore - I'll generally also need to 
update the infrastructure as some sort of new build tool will have 
become involved, with all its attendant quirks, or someone will have 
decided to change the name of packages, or some other change will have 
occurred. Things seem to be in a constant state of flux, and thus 
packaging becomes quite time-consuming.

Bootstrapping is also a problem: I don't know if the problem has been 
solved in the meantime, but not too long ago it was impossible to 
package opam, because some of the libraries needed to build opam 
actually used the opam installer themselves. I had to write my own 
version of the opam installer to get around that particular problem.

It's all well and good to say that 'using opam is the default', but can 
we please spare a thought for distribution editors as well? Using ocaml 
and friends with the package manager that comes with your distribution 
is a reasonable use case, I'd think.

best

   Jaap

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Simon Cruanes]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

Le Mon, 26 Nov 2018, Jaap Boender wrote:
> Bootstrapping is also a problem: I don't know if the problem has been solved
> in the meantime, but not too long ago it was impossible to package opam,
> because some of the libraries needed to build opam actually used the opam
> installer themselves. I had to write my own version of the opam installer to
> get around that particular problem.

That's a bit strange, I recall that opam's source archive contains
opam's dependencies and allows to build them using `make lib-ext` or
something like that. I don't recall having problems installing opam2 from
source.

> It's all well and good to say that 'using opam is the default', but can we
> please spare a thought for distribution editors as well? Using ocaml and
> friends with the package manager that comes with your distribution is a
> reasonable use case, I'd think.

The ecosystem has indeed been changing rapidly this year, but in a good
way, imho: a lot of people and projects have converged to using `dune`
as the build system, and opam2 as the package manager (and project
description). Once dune is packaged for a distribution, a lot of OCaml
projects should be buildable, I think.

-- 
Simon Cruanes

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Oliver Bandel]

Quoting  Simon Cruanes <simon.cruanes.2007@m4x.org> (snt: 2018-11-26 23:55 +0100 CET) (rcv: 2018-11-26 23:54 +0100 CET):
> Le Mon, 26 Nov 2018, Jaap Boender wrote:
[...]
> > It's all well and good to say that 'using opam is the default', but can we
> > please spare a thought for distribution editors as well? Using ocaml and
> > friends with the package manager that comes with your distribution is a
> > reasonable use case, I'd think.
> 
> The ecosystem has indeed been changing rapidly this year, but in a good
> way, imho: a lot of people and projects have converged to using `dune`
> as the build system, and opam2 as the package manager
[...]

OK, so after you know OCaml,
then you have to learn opam,
and after that, dune is just another tool that people have to know.

If developers want to use all that, because it's so much of a way to save time in
development, building and packaging - if you know all three - fine.

But if you just want to use OCaml for your programming,
why additionally need to learn opam and now dune?

Those dvelopers who want to enrich the makedependency-hell, can do it of course.
But please add a Makefile, that allows people who don't know or don't even want to know
opam, dune and so many other tools,
so that "make" and "make install" does work properly, even you never heard of all the
new tools.

To create such Makefiles should be easy for people who know the used build-/installer-tools.
But it's hard to learn any new build-/installation-tool in 5 minutes,
just to get a small package (or a bunch of them) installed, to go on with the own work.

Also documentation should mention, which tools are used and which packages are needed to build
the stuff, you just want to (compile/install) and use.

So: if "make" and "make install" just starts the toolchain to build and install,
which means a be boring stupid Makefile for those who know the toolchain,
this would be of great help for those, not interested in learing the whole
toolchain (or not having the time to do so).

Also nice would be, if opam could export the config of a package to e.g. json or even csv.
Last time I asked, it was not possible and I started reverse engineering the fileformat (and gave up after a while).
Maybe today thats possible, so that it's easy to write a tool that would convert
opam-package-files to system-installer-files.


Ciao,
  Oliver

Re: [Caml-list] Build-/Installation tools - not enough of them?

[SF Markus Elfring]

> But if you just want to use OCaml for your programming,
> why additionally need to learn opam and now dune?

Are there development competitions and varying design goals evolving
together with such software build tools?

Regards,
Markus

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Simon Cruanes]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

Le Tue, 27 Nov 2018, Oliver Bandel wrote:
> OK, so after you know OCaml,
> then you have to learn opam,
> and after that, dune is just another tool that people have to know.

Well you need to build your software. My packages tend to have a super
simple makefile around dune, because I'm used to type `make` but
claiming one can write modern OCaml without learning a bit how to build
it is naive at best. It's like *any* other language, you got to learn a
bit of tooling (even python — good luck with pip or conda — or javascript
with its pile of tools that change all the time. Oh and what about
cmake?)

> But if you just want to use OCaml for your programming,
> why additionally need to learn opam and now dune?

Because `ocamlopt foo.ml -o foo` won't scale.

-- 
Simon Cruanes

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Oliver Bandel]

Quoting  Simon Cruanes <simon.cruanes.2007@m4x.org> (snt: 2018-11-27 16:06 +0100 CET) (rcv: 2018-11-27 16:06 +0100 CET):
> Le Tue, 27 Nov 2018, Oliver Bandel wrote:
> > OK, so after you know OCaml,
> > then you have to learn opam,
> > and after that, dune is just another tool that people have to know.
> 
> Well you need to build your software. My packages tend to have a super
> simple makefile around dune, because I'm used to type `make`

That's the one thing I asked for in the thread-starting email.

So, then you are not meant to be addressed.


> but
> claiming one can write modern OCaml without learning a bit how to build
> it is naive at best.

Don't know why you are so aggressive here.

It's the problem, that the number of tools evolving around OCaml
is growing and growing, and some developers, who already are used in one of them use
them without writing the nice Makefile as a wrapper.

So, if you want to write packages for the OS's, and if you know one of the tools,
it's not enough. You have to know them all, just to create the package.
That's an extra effort for the one who wants to package, just because the
developer who already knows it's tool of choice is not willing to wrap the
stuff into a Makefile.

The minimal effort on the one single side is avoided, and that creates a lot of trouble in several other places.



> It's like *any* other language, you got to learn a
> bit of tooling (even python — good luck with pip or conda — or javascript
> with its pile of tools that change all the time. Oh and what about
> cmake?)
> 
> > But if you just want to use OCaml for your programming,
> > why additionally need to learn opam and now dune?
> 
> Because `ocamlopt foo.ml -o foo` won't scale.


I'm happy with OCamlMakefile.
So I don't need to `ocamlopt foo.ml -o foo` by hand.

Maybe it's better to abstain from native OS-packages of a lot of OCaml-software,
just because it's the "not-naive way" to insist, that people who
would like to package software of others have to have the need for learning
a lot of tools they would not use otherwise, instead of asking those who know the tools already,
to write a Makefile (which they also already know how to do that).

It's the way, how to produce work for others. (Some kind of sadistic attitude, it seems (thougthless at best).)


Ciao,
  Oliver

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Daniel Bünzli]

On 27 November 2018 at 16:49:29, Oliver Bandel (oliver@first.in-berlin.de) wrote:

> So, if you want to write packages for the OS's, and if you know one of the tools,
> it's not enough. You have to know them all, just to create the package.
> That's an extra effort for the one who wants to package, just because the
> developer who already knows it's tool of choice is not willing to wrap the
> stuff into a Makefile.
>  
> The minimal effort on the one single side is avoided, and that creates a lot of trouble  
> in several other places.

Frankly I'm not even sure about that. Given the myriads of distributions that exist out there all with their own idiosyncrasies, it is likely that a single wrapping Makefile won't be able to satisfy them all. Worse if one provides a Makefile but actually doesn't use it, it's likely it will be outdated after a few releases. You ask why are there so many build tools ? I ask why are there so many distributions ? 

More than once I suggested to operating system packagers that they should have a close look at opam metadata to automate their packaging and engage discussions on the opam issue tracker to indicate if there's anything that could be done to help them. I almost never heared anything, except from time to time we get this occasional recurring discussion about providing silly wrapping Makefiles in packages.

I don't get it. Do system packagers really prefer to manually package things rather than (semi-)automate their processes by relying on the exact information they need in machine readable format as can be found in opam files ? Aren't they interested in the ability to tap into the vast amount of (very boring, let's salute them !) effort that is being put by opam repository maintainers to ensure OCaml packages build on a large variety of platform ?

Daniel

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Jaap Boender]

On 27/11/2018 16:27, Daniel Bünzli wrote:
> More than once I suggested to operating system packagers that they should have a close look at opam metadata to automate their packaging and engage discussions on the opam issue tracker to indicate if there's anything that could be done to help them. I almost never heared anything, except from time to time we get this occasional recurring discussion about providing silly wrapping Makefiles in packages.

Apart from the bootstrap issue I mentioned), dune and the opam .install 
files work pretty well for me under pkgsrc; I've got the infrastructure 
set up now so that I can basically just create a package by pointing out 
where to download, which dune packages to build and which .install files 
to use.

Like I said, there was a time when there was a lot of tweaking involved, 
but hopefully the dust will settle now and updating will again be easy.

best,

   Jaap

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Jeremie Dimino]

On Tue, Nov 27, 2018 at 5:46 PM Jaap Boender <jaapb@kerguelen.org> wrote:
> Apart from the bootstrap issue I mentioned), dune and the opam .install
> files work pretty well for me under pkgsrc; I've got the infrastructure
> set up now so that I can basically just create a package by pointing out
> where to download, which dune packages to build and which .install files
> to use.

Hi Jaap,

Since version 1.0.0, Dune no longer requires opam to handle
installation. This means that you can build and install Dune with
nothing more than the OCaml compiler, and you can build and install
packages using Dune without opam.

Best,
Jeremie

[Caml-list] How to start with the curren toolset?

[Hendrik Boom]

On Wed, Nov 28, 2018 at 11:47:57AM +0000, Jeremie Dimino wrote:
> On Tue, Nov 27, 2018 at 5:46 PM Jaap Boender <jaapb@kerguelen.org> wrote:
> > Apart from the bootstrap issue I mentioned), dune and the opam .install
> > files work pretty well for me under pkgsrc; I've got the infrastructure
> > set up now so that I can basically just create a package by pointing out
> > where to download, which dune packages to build and which .install files
> > to use.
> 
> Hi Jaap,
> 
> Since version 1.0.0, Dune no longer requires opam to handle
> installation. This means that you can build and install Dune with
> nothing more than the OCaml compiler, and you can build and install
> packages using Dune without opam.

So, to get dune I need to install ocaml.
To get Ocaml it's recommended to start with opam (even though I don't 
need it for dune) 
To get opam ... ??

The opam in my distro (Devuan Jessie) isn't up to date.  I gather I 
should start with an up-to-date opam.

Presumably there's a way to install current opam from scratch without 
starting with dune.

Or maybe I should ask: what *is* the current starting point?

-- hendrik

Re: [Caml-list] How to start with the curren toolset?

[Ian Zimmerman]

On 2018-12-01 10:12, Hendrik Boom wrote:

> The opam in my distro (Devuan Jessie) isn't up to date.  I gather I 
> should start with an up-to-date opam.
> 
> Presumably there's a way to install current opam from scratch without 
> starting with dune.
> 
> Or maybe I should ask: what *is* the current starting point?
> 

One possibility is to add the unstable repos to your debian apt
configuration, but block them for most packages except opam.  I am quite
confident that this is possible though I no longer use debian on the
desktop.

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.

Re: [Caml-list] How to start with the curren toolset?

[Daniel Bünzli]

On 1 December 2018 at 16:12:21, Hendrik Boom (hendrik@topoi.pooq.com) wrote:

> So, to get dune I need to install ocaml.
> To get Ocaml it's recommended to start with opam (even though I don't
> need it for dune)
> To get opam ... ??

I think this discussion could benefit if people could stop mixing users who wish to 1) install OCaml to program in OCaml 2) install OCaml because an application they use is using OCaml (those may actually not even know they are installing OCaml)

If you are in 1) your best solution is to install `opam` using one of the ways documented on the opam install page and then use the opam-repository to install an ocaml compiler and the OCaml packages you'd like to use. 

If you are in 2) then *you should not* install opam nor ocaml via opam and propably don't even know that you are going to install OCaml since that will occur as a side effect of installing the application via your operating system packager. 

If you are the people who are responsible to provide the intrastructure for 2) then the best way to provide an OCaml package is *not* to install `opam`. It is to simply follow the install instruction provided in the OCaml distribution tarball. I would then suggest you make a package to install opam (*which doesn't need dune* as a short glance over the README will tell you). Having an `opam` package then allow you then to use its libraries to develop the tools you might need for your distribution to read `opam` files and generate packages from the opam-repository metadata.

I don't think that this is super complicated. A little bit of reading in the various readme of ocaml and opam should be able to convince that this is entirely possible to do without any bootstrap problems. We had boostrap problems at at a certain point with cmdliner but I believe that this is fixed, and if it's not please tell. 

Best, 

Daniel

Re: [Caml-list] How to start with the curren toolset?

[David Allsopp]

On 2 Dec 2018, at 15:27, Daniel Bünzli <daniel.buenzli@erratique.ch> wrote:
>> On 1 December 2018 at 16:12:21, Hendrik Boom (hendrik@topoi.pooq.com) wrote:
>> 
>> So, to get dune I need to install ocaml.
>> To get Ocaml it's recommended to start with opam (even though I don't
>> need it for dune)
>> To get opam ... ??
> 
> I don't think that this is super complicated. A little bit of reading in the various readme of ocaml and opam should be able to convince that this is entirely possible to do without any bootstrap problems. We had boostrap problems at at a certain point with cmdliner but I believe that this is fixed, and if it's not please tell. 

Indeed. A “pure” bootstrap of opam should always remain possible, as the “lib-pkg” bootstrap mode of opam is tested in its CI. A side-effect of this mode’s existence is that src_ext/Makefile.packages contains the manual build instructions for opam-free compilation of all of opam’s dependencies.


David

[Caml-list] let's give a try at opam-bundle

[Francois Berenger]

On 03/12/2018 00:27, Daniel Bünzli wrote:
> On 1 December 2018 at 16:12:21, Hendrik Boom (hendrik@topoi.pooq.com) 
> wrote:
> 
>> So, to get dune I need to install ocaml.
>> To get Ocaml it's recommended to start with opam (even though I don't
>> need it for dune)
>> To get opam ... ??
> 
> I think this discussion could benefit if people could stop mixing
> users who wish to 1) install OCaml to program in OCaml 2) install
> OCaml because an application they use is using OCaml (those may
> actually not even know they are installing OCaml)
> 
> If you are in 1) your best solution is to install `opam` using one of
> the ways documented on the opam install page and then use the
> opam-repository to install an ocaml compiler and the OCaml packages
> you'd like to use. 
> 
> If you are in 2) then *you should not* install opam nor ocaml via opam
> and propably don't even know that you are going to install OCaml since
> that will occur as a side effect of installing the application via
> your operating system packager.

I believe many ocaml programs available in opam don't have a package
in most Linux distributions (or it might be an old version, if there is 
one).

So, here comes opam-bundle:

https://github.com/AltGr/opam-bundle

That probably more people should know about, give it a try and report
problems, if any:

https://github.com/AltGr/opam-bundle/issues

Trying the tool and reporting problems is _very_ important, so that the 
tool
has a chance to improve, get some attention/contributions, etc.
I hope that many people will use it, so that it is really battle-tested
and production ready.

Packaging coccinelle for end-users (who should not have to bother about 
ocaml
or opam) should be as simple as

$ opam-bundle coccinelle

If it is not, maybe some opam-bundle bug has to be reported.
I did not try, I just give coccinelle as a complex software example with
probably many dependencies ('opam-bundle memcad' might also be quite a 
harsh test).

I just created a bundle for some very simple software I have
in opam. It worked, the bundle was created on Linux and I could
install and run the software on a mac. That was cool.

Regards,
Francois.

[Caml-list] confusing message in opam installer.

[Hendrik Boom]

On Sat, Dec 01, 2018 at 10:12:21AM -0500, Hendrik Boom wrote:
> On Wed, Nov 28, 2018 at 11:47:57AM +0000, Jeremie Dimino wrote:
> > On Tue, Nov 27, 2018 at 5:46 PM Jaap Boender <jaapb@kerguelen.org> wrote:
> > > Apart from the bootstrap issue I mentioned), dune and the opam .install
> > > files work pretty well for me under pkgsrc; I've got the infrastructure
> > > set up now so that I can basically just create a package by pointing out
> > > where to download, which dune packages to build and which .install files
> > > to use.
> > 
> > Hi Jaap,
> > 
> > Since version 1.0.0, Dune no longer requires opam to handle
> > installation. This means that you can build and install Dune with
> > nothing more than the OCaml compiler, and you can build and install
> > packages using Dune without opam.
> 
> So, to get dune I need to install ocaml.
> To get Ocaml it's recommended to start with opam (even though I don't 
> need it for dune) 
> To get opam ... ??
> 
> The opam in my distro (Devuan Jessie) isn't up to date.  I gather I 
> should start with an up-to-date opam.
> 
> Presumably there's a way to install current opam from scratch without 
> starting with dune.

I ended up installing opam using the script https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh

There's only one thing that gave me pause -- the message

[NOTE] Make sure that ~/.profile is well sourced in your ~/.bashrc.

It so happens that my distro does the reverse -- sources my ~/.bashrc in my ~/.profile.

Had I not checked, I would have ended up with an endless shell 
recursion.

-- hendrik

Re: [Caml-list] confusing message in opam installer.

[Julia Lawall]

On Sun, 2 Dec 2018, Hendrik Boom wrote:

> On Sat, Dec 01, 2018 at 10:12:21AM -0500, Hendrik Boom wrote:
> > On Wed, Nov 28, 2018 at 11:47:57AM +0000, Jeremie Dimino wrote:
> > > On Tue, Nov 27, 2018 at 5:46 PM Jaap Boender <jaapb@kerguelen.org> wrote:
> > > > Apart from the bootstrap issue I mentioned), dune and the opam .install
> > > > files work pretty well for me under pkgsrc; I've got the infrastructure
> > > > set up now so that I can basically just create a package by pointing out
> > > > where to download, which dune packages to build and which .install files
> > > > to use.
> > >
> > > Hi Jaap,
> > >
> > > Since version 1.0.0, Dune no longer requires opam to handle
> > > installation. This means that you can build and install Dune with
> > > nothing more than the OCaml compiler, and you can build and install
> > > packages using Dune without opam.
> >
> > So, to get dune I need to install ocaml.
> > To get Ocaml it's recommended to start with opam (even though I don't
> > need it for dune)
> > To get opam ... ??
> >
> > The opam in my distro (Devuan Jessie) isn't up to date.  I gather I
> > should start with an up-to-date opam.
> >
> > Presumably there's a way to install current opam from scratch without
> > starting with dune.
>
> I ended up installing opam using the script https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh
>
> There's only one thing that gave me pause -- the message
>
> [NOTE] Make sure that ~/.profile is well sourced in your ~/.bashrc.

The above would be better without "well".  I would have filed an issue,
but the above link doesn't seem to contain the above string, so I'm not
sure where to file it.

julia

>
> It so happens that my distro does the reverse -- sources my ~/.bashrc in my ~/.profile.
>
> Had I not checked, I would have ended up with an endless shell
> recursion.
>
> -- hendrik
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] confusing message in opam installer.

[Raja Boujbel - OCamlPro]

Jaap, Julia,

 > I ended up installing opam using the script 
https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh
 >
 > There's only one thing that gave me pause -- the message
 >
 > [NOTE] Make sure that ~/.profile is well sourced in your ~/.bashrc.
 >
 > It so happens that my distro does the reverse -- sources my ~/.bashrc 
in my ~/.profile.
 >
 > Had I not checked, I would have ended up with an endless shell
 > recursion.

It is quite difficult to make scripts that works on all distribution, 
even if we restrain to POSIX requirements. On bash environment setup, 
there is a defined precedence, but each solution doesn't suit some 
users. It was a choice to add hooks in the .profile assuming it is 
sourced in the .bashrc [1]. Discussion about it are in [2] and [3].


 > The above would be better without "well".  I would have filed an issue,
 > but the above link doesn't seem to contain the above string, so I'm not
 > sure where to file it.

This message is in the opam code. Generally, you can report opam related 
issue in the bug tracker [4], install matters too.


Raja Boujbel -- OCamlPro


[1] https://github.com/ocaml/opam/pull/3304
[2] https://github.com/ocaml/opam/issues/1447
[3] https://github.com/ocaml/opam/issues/533
[4] https://github.com/ocaml/opam/issues/

Re: [Caml-list] Build-/Installation tools - not enough of them?

[SF Markus Elfring]

> It's the problem, that the number of tools evolving around OCaml is growing and growing,

Is this just another interesting software evolution?


> and some developers, who already are used in one of them use them
> without writing the nice Makefile as a wrapper.

Can build systems work also without such wrappers?



> So, if you want to write packages for the OS's, and if you know one of the tools,
> it's not enough.

It depends on some factors, doesn't it?


> You have to know them all, just to create the package.

Which packages will require to fiddle with more build tools than you feel
comfortable with at the moment?


> That's an extra effort for the one who wants to package, just because the
> developer who already knows it's tool of choice is not willing to wrap the
> stuff into a Makefile.

This data format can trigger also specific development challenges
when you care for software portability.


> I'm happy with OCamlMakefile.

Would you like to discuss its system constraints any further?

Regards,
Markus

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Markus Mottl]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

On Tue, Nov 27, 2018 at 10:50 AM Oliver Bandel <oliver@first.in-berlin.de>
wrote:

> I'm happy with OCamlMakefile.
>

I have to say that I find it truly charming if not flattering that people
are still using my ancient OCamlMakefile (which, btw., is now more than 20
years old!).

Seriously, I think that every OCaml developer should switch to OPAM + Dune
at this point.  The ease of developing, contributing to, packaging,
releasing, and using OCaml software has improved dramatically over the
years, especially with these tools.  I'm generally reluctant to jump on the
next build tool or package manager, but my expectation is that these tools
will stay around for many years to come.

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

[-- Attachment #2: Type: text/html, Size: 1281 bytes --]

Re: [Caml-list] <DKIM> Re: Build-/Installation-Tools - not enogh of them?

[Vu Ngoc San]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]



Le 27/11/2018 à 18:09, Markus Mottl a écrit :
> On Tue, Nov 27, 2018 at 10:50 AM Oliver Bandel 
> <oliver@first.in-berlin.de <mailto:oliver@first.in-berlin.de>> wrote:
>
>     I'm happy with OCamlMakefile.
>
>
> I have to say that I find it truly charming if not flattering that 
> people are still using my ancient OCamlMakefile (which, btw., is now 
> more than 20 years old!).
>
> Seriously, I think that every OCaml developer should switch to OPAM + 
> Dune at this point.  The ease of developing, contributing to, 
> packaging, releasing, and using OCaml software has improved 
> dramatically over the years, especially with these tools.  I'm 
> generally reluctant to jump on the next build tool or package manager, 
> but my expectation is that these tools will stay around for many years 
> to come.
>
> -- 
> Markus Mottl http://www.ocaml.info markus.mottl@gmail.com 
> <mailto:markus.mottl@gmail.com>


I also was happy to use OCamlMakefile for my projects. It works, and why 
would I need something more complicated?
Well, after reading your email, I was motivated to try dune, and... 
wow... I was shocked.
In a few minutes I was able to compile my most complicated project, with 
near to zero information in the dune file!
This is actually easier than OcamlMakefile.

thanks for the incentive to switch ;)

San

[-- Attachment #2: Type: text/html, Size: 2696 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[oliver]

Quoting  Markus Mottl <markus.mottl@gmail.com> (snt: 2018-11-27 18:09 +0100 CET) (rcv: 2018-11-27 18:10 +0100 CET):
> On Tue, Nov 27, 2018 at 10:50 AM Oliver Bandel <oliver@first.in-berlin.de>
> wrote:
> 
> > I'm happy with OCamlMakefile.
> >
> 
> I have to say that I find it truly charming if not flattering that people
> are still using my ancient OCamlMakefile (which, btw., is now more than 20
> years old!).
[...]

Well, you use email here on the mailinglist.
It's much older than OCamlMakefile.
You still use it, even though there are a plenty of new possibilities in the web to talk about OCaml.
<iron>Maybe we should switch away from email then!</irony>

OCamlMakefile if fine for me: KISS principle.
If I would use come complex tools, I would provide a wrapping makefile to make default install easy.

Just for the record: I did not said, all the new tools are nonsense,
or would not help in certain projects.
I just asked those developers, who already worked into these tools,
to provide a way to make the build and install easy.

Especially if the sources of the project did not changed, or changed only minor,
switching to new build-/install-tools IMO does not make sense.
( -> never change a running install-prcedure nor change it's api )

But if one does this switch, providing the "make" / "make install" interface
for those, who had not the time to work into any new tool, would be easy.

By the way: I maintained ocaml-sqlite3 on AUR for Arch for a while.
At the moment I have not the time and nervs to look at dune, no matter
how good or bad it is.

So, I just disowned  the ocaml-sqlite3 package on AUR.
Maybe someone will pick it up and provide a new version. Maybe not.

The package had 45 votes, which is comparingly much.
So there seems to be a need for this.
opam is part of the official Arch packages since a while,
so one could ask, why 45 people voted for the PKGBUILD-based package
instead of switching to opam.
(And there will be more users than the 45 who voted for it, I would guess.)


> 
> Seriously, I think that every OCaml developer should switch to OPAM + Dune
> at this point.

Just because it's hype?

It's like: every technical discussion should be done via wh*tsapp, Sl*k, or foo or bar.
Email is so oldschool.

But of course people have different opinions on this.

OK, just ignore the 45 votes.

[...]
> I'm generally reluctant to jump on the
> next build tool or package manager,

Oh... so we are not far away here.

> but my expectation is that these tools
> will stay around for many years to come.

Heheh... maybe.

Ciao,
  Oliver

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Hendrik Boom]

On Tue, Nov 27, 2018 at 02:29:15PM +0100, Oliver Bandel wrote:
> 
> But if you just want to use OCaml for your programming,
> why additionally need to learn opam and now dune?
> 
> Those dvelopers who want to enrich the makedependency-hell, can do it of course.
> But please add a Makefile, that allows people who don't know or don't even want to know
> opam, dune and so many other tools,
> so that "make" and "make install" does work properly, even you never heard of all the
> new tools.

The Makefile could be as simple as the few commands that use the 
existing opam-specific package manager(s) to do the installation that 
we now expect the user to figure out.  The few lines this community 
seems to expect people to already know.

It could even be accompanied by the usual ./configure file that wold do 
nothing but tell the otherwise hapless user what package manager still 
needs to be installed and how to do that.

So the installation becomes
./configure
make install

> 
> To create such Makefiles should be easy for people who know the used build-/installer-tools.
> But it's hard to learn any new build-/installation-tool in 5 minutes,
> just to get a small package (or a bunch of them) installed, to go on with the own work.

Almost every time I've tried using libraries of packages or even just 
programming in ocaml, run into trouble and ask for help, I am advised 
to use different tools from the ones I am using.  Different ones each 
time.

And when I use a packaged OCaml library, I often find a bug that never 
gets fixed.  The original developer or packager has moved on to other 
things, and the package is effectively orphaned.

I've given up.  I prefer the OCaml language, but I'm more productive in 
Go.

I regret this.

-- hendrik

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Jaap Boender]

On 26/11/2018 22:55, Simon Cruanes wrote:
> Le Mon, 26 Nov 2018, Jaap Boender wrote:
>> Bootstrapping is also a problem: I don't know if the problem has been solved
>> in the meantime, but not too long ago it was impossible to package opam,
>> because some of the libraries needed to build opam actually used the opam
>> installer themselves. I had to write my own version of the opam installer to
>> get around that particular problem.
> 
> That's a bit strange, I recall that opam's source archive contains
> opam's dependencies and allows to build them using `make lib-ext` or
> something like that. I don't recall having problems installing opam2 from
> source.

The problem is that opam needs dune and cmdliner to build. cmdliner can 
be installed without opam, but dune, at the time at least, 
can't/couldn't - and adding the source code of dependencies inside a 
source archive isn't very efficient (if you have local patches in your 
distribution, you end up having to apply them twice, for starters).

It would be nice if opam were distributed in such a way that at least 
opam-installer can be built with no circular dependencies (which is 
basically what I've tried to do with opaline) so that you can bootstrap 
things easily.

best

   Jaap

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Francois Berenger]

On 27/11/2018 07:44, Jaap Boender wrote:
> On 26/11/2018 10:14, Oliver Bandel wrote:
>> The situation differs, if one wants to package the written software,
>> and one needs to know many of those tools, just to compile the stuff.
>> So, when one just wants to compile and install some software,
>> just for that, it would take much effort to learn the different 
>> build-tools.
> 
> I'd like to chime in here - as the person responsible for doing most
> of the OCaml packaging for pkgsrc, I've noticed that I can hardly ever
> just update a package to a new version anymore - I'll generally also
> need to update the infrastructure as some sort of new build tool will
> have become involved, with all its attendant quirks, or someone will
> have decided to change the name of packages, or some other change will
> have occurred. Things seem to be in a constant state of flux, and thus
> packaging becomes quite time-consuming.

There is a tool from Louis Gesbert of OCamlPro that might help people 
who want to create
a self-contained installer for some OCaml software that is already 
packaged
for opam:

https://github.com/AltGr/opam-bundle

PS: I did not use it yet, but I plan to

> Bootstrapping is also a problem: I don't know if the problem has been
> solved in the meantime, but not too long ago it was impossible to
> package opam, because some of the libraries needed to build opam
> actually used the opam installer themselves. I had to write my own
> version of the opam installer to get around that particular problem.
> 
> It's all well and good to say that 'using opam is the default', but
> can we please spare a thought for distribution editors as well? Using
> ocaml and friends with the package manager that comes with your
> distribution is a reasonable use case, I'd think.
> 
> best
> 
>   Jaap

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Oliver Bandel]

Quoting  Francois Berenger <mlists@ligand.eu> (snt: 2018-11-27 03:33 +0100 CET) (rcv: 2018-11-27 03:33 +0100 CET):
> On 27/11/2018 07:44, Jaap Boender wrote:
> > On 26/11/2018 10:14, Oliver Bandel wrote:
> > > The situation differs, if one wants to package the written software,
> > > and one needs to know many of those tools, just to compile the stuff.
> > > So, when one just wants to compile and install some software,
> > > just for that, it would take much effort to learn the different
> > > build-tools.
> > 
> > I'd like to chime in here - as the person responsible for doing most
> > of the OCaml packaging for pkgsrc, I've noticed that I can hardly ever
> > just update a package to a new version anymore - I'll generally also
> > need to update the infrastructure as some sort of new build tool will
> > have become involved, with all its attendant quirks, or someone will
> > have decided to change the name of packages, or some other change will
> > have occurred. Things seem to be in a constant state of flux, and thus
> > packaging becomes quite time-consuming.
> 
> There is a tool from Louis Gesbert of OCamlPro that might help people who
> want to create
> a self-contained installer for some OCaml software that is already packaged
> for opam:
[...]

Hey, just another tool, yay!  :->

And if it's too complicated, just invent another tool on top of that! :->
(of course with different syntax and clis-witches)


Ciao,
  Oliver

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[John F Carr]

I have a related request.  I am not a trusting person.  I do not like "curl | sudo sh" type installation methods.  Whatever one true packaging system we use, I want to trust it not to let the build process mess up my system.  For example, I see opam makes some attempt to contain the build process on some systems.  It's not clear to me how much it does and how effective.  And it appaerntly does not work on BSD.  If a package has 'rm -rf $BUILD/', or equivalent ocaml code, are its ill effects confined when BUILD is unset?  Can the build process grab screenshots from the background?  And so on.

One reason I like make is, if the Makefile is simple you know what it's going to do.

Also, the xkcd on standards seems relevant: https://xkcd.com/927/

> On Nov 26, 2018, at 05:14 , Oliver Bandel <oliver@first.in-berlin.de> wrote:
> 
> Hello,
> 
> a while ago it looked like there were not enough build- and installation-tools
> for OCaml. I remember some discussions about that.
> 
> Now it seems to me that there are a lot of them.
> So, developers can pick the one they know about.
> 
> For all these tools there might be good reasons to use them, and those
> developers who looked at these tools and choose them for their projects, will
> know them well enough.
> 
> The situation differs, if one wants to package the written software,
> and one needs to know many of those tools, just to compile the stuff.
> So, when one just wants to compile and install some software,
> just for that, it would take much effort to learn the different build-tools.
> 
> So, packaging has become more complicated, even though for the developers
> these tools may save time.
> 
> It would be nice if people who used one of the many new building tools
> could provide a Makefile that allows just to type
> "make" and "make install", instead of expecting everyone who wants to compile
> the software to first learn just-another-build-tool.
> 
> Also it would be good, to mention early, which installation tools (make-dependencies)
> are in use, and too mention needed packages (opam or others) to just build the stuff.
> 
> Thanks and regards,
>  Oliver Bandel
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list https://inbox.ocaml.org/caml-list
> Forum: https://discuss.ocaml.org/
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Louis Gesbert]

[-- Attachment #1: Type: text/plain, Size: 5603 bytes --]

> - John F Carr, 27/11/2018 13:40 -
> I have a related request.  I am not a trusting person.  I do not like "curl | sudo sh" type installation methods.

You're not the only one :)
Some notes on opam's security model:

- opam 2.0 uses, by default `bubblewrap` [1] on Linux and `sandbox-exec` on OSX to ensure that package scripts:
  * don't make any network access
  * don't interact with other processes
  * don't write outside of their build dir, /tmp, and (in the case of install) the switch prefix (excl. opam files)

- this is done using simple wrapper scripts [2] and some default hooks configuration in ~/.opam/config, so if you know about built-in sandboxing engines for other OSes, it is fairly easy to experiment with them, and a contribution would be very welcome.

- while I expect this to be reasonably secure, it's intended first and foremost to avoid dramatic errors, not to protect against malicious repositories

- package scripts are protected but **not any use made by the users of the programs or libraries that were installed through opam**. In other words, building should be safe, but there is no guarantee about what the result of the build will do: that is not restrained by opam in any way

- the effort to provide end-to-end package signatures in the repository [3] is still ongoing. Cheers to Hannes Mehnert for the awesome work he has already done here. Most of the work should be done, but then we need to integrate all that, and there is a lot of work on the tooling so that it won't add to much burden on users and repository maintainers (this commonly results in most disabling the security features, which is as good has having no security features to begin with).

- we do advertise `curl | sh` on the installation page as the easiest entry point, but the script is quite trivial and only uses root to copy to your prefix; it's very easy to fetch the binary by hand from Github if you prefer not to run it, and of course, you can also build from source using the bootstrap scripts.

> If a package has 'rm -rf $BUILD/', or equivalent ocaml code, are its ill
> effects confined when BUILD is unset?

yes, that's the whole point of the sandboxing that was introduced in 2.0

> Can the build process grab screenshots from the background?

not sure. Probably not on Linux since we use a different process space, but maybe on OSX. In anycase, since network access is blocked in both cases, that wouldn't do much harm.

> One reason I like make is, if the Makefile is simple you know what it's going to do.

I would object that opam package definition files (`opam` or `foo.opam`) should be at least as straightforward to read even if you have never seen the syntax, are less error-prone, and are generally much shorter. Just look for the "build:" and "install:" parts. But I agree you need to know first to look at them, and since they are generally an indirection to some build-system (`make`, `dune`, `topkg`...), you would just start digging...

> Also, the xkcd on standards seems relevant: https://xkcd.com/927/

We have _actually_ been converging as of late, though.


Best,
Louis Gesbert — OCamlPro



[1] https://github.com/projectatomic/bubblewrap
[2] https://github.com/ocaml/opam/blob/master/src/state/shellscripts/bwrap.sh
and https://github.com/ocaml/opam/blob/master/src/state/shellscripts/sandbox_exec.sh
[3] https://github.com/hannesm/conex

> Whatever one true packaging system we use, I want to trust it not to let the build process mess up my system.  For example, I see opam makes some attempt to contain the build process on some systems.  It's not clear to me how much it does and how effective.  And it appaerntly does not work on BSD.  If a package has 'rm -rf $BUILD/', or equivalent ocaml code, are its ill effects confined when BUILD is unset?  Can the build process grab screenshots from the background?  And so on.
> 
> One reason I like make is, if the Makefile is simple you know what it's going to do.
> 
> Also, the xkcd on standards seems relevant: https://xkcd.com/927/
> 
> > On Nov 26, 2018, at 05:14 , Oliver Bandel <oliver@first.in-berlin.de> wrote:
> > 
> > Hello,
> > 
> > a while ago it looked like there were not enough build- and installation-tools
> > for OCaml. I remember some discussions about that.
> > 
> > Now it seems to me that there are a lot of them.
> > So, developers can pick the one they know about.
> > 
> > For all these tools there might be good reasons to use them, and those
> > developers who looked at these tools and choose them for their projects, will
> > know them well enough.
> > 
> > The situation differs, if one wants to package the written software,
> > and one needs to know many of those tools, just to compile the stuff.
> > So, when one just wants to compile and install some software,
> > just for that, it would take much effort to learn the different build-tools.
> > 
> > So, packaging has become more complicated, even though for the developers
> > these tools may save time.
> > 
> > It would be nice if people who used one of the many new building tools
> > could provide a Makefile that allows just to type
> > "make" and "make install", instead of expecting everyone who wants to compile
> > the software to first learn just-another-build-tool.
> > 
> > Also it would be good, to mention early, which installation tools (make-dependencies)
> > are in use, and too mention needed packages (opam or others) to just build the stuff.
> > 
> > Thanks and regards,
> >  Oliver Bandel
> > 
> 
> 
> 

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Caml-list] Build-/Installation-Tools - not enogh of them?

[Louis Roché]

[-- Attachment #1: Type: text/plain, Size: 6253 bytes --]

Shameless plug, I tried to document basic opam usage for people who have
experience with npm. It turns ok it can also be an introduction to total
newcomers. Hope it can help some people.
https://khady.info/opam-npm.html


On Sat, Dec 1, 2018, at 2:31 AM, Louis Gesbert wrote:
>> - John F Carr, 27/11/2018 13:40 -
>> I have a related request.  I am not a trusting person.  I do not like
>> "curl | sudo sh" type installation methods.> 
> You're not the only one :)
> Some notes on opam's security model:
> 
> - opam 2.0 uses, by default `bubblewrap` [1] on Linux and `sandbox-
>   exec` on OSX to ensure that package scripts:>   * don't make any network access
>   * don't interact with other processes
>   * don't write outside of their build dir, /tmp, and (in the case of
>     install) the switch prefix (excl. opam files)> 
> - this is done using simple wrapper scripts [2] and some default hooks
>   configuration in ~/.opam/config, so if you know about built-in
>   sandboxing engines for other OSes, it is fairly easy to experiment
>   with them, and a contribution would be very welcome.> 
> - while I expect this to be reasonably secure, it's intended first and
>   foremost to avoid dramatic errors, not to protect against malicious
>   repositories> 
> - package scripts are protected but **not any use made by the users of
>   the programs or libraries that were installed through opam**. In
>   other words, building should be safe, but there is no guarantee
>   about what the result of the build will do: that is not restrained
>   by opam in any way> 
> - the effort to provide end-to-end package signatures in the
>   repository [3] is still ongoing. Cheers to Hannes Mehnert for the
>   awesome work he has already done here. Most of the work should be
>   done, but then we need to integrate all that, and there is a lot of
>   work on the tooling so that it won't add to much burden on users and
>   repository maintainers (this commonly results in most disabling the
>   security features, which is as good has having no security features
>   to begin with).> 
> - we do advertise `curl | sh` on the installation page as the easiest
>   entry point, but the script is quite trivial and only uses root to
>   copy to your prefix; it's very easy to fetch the binary by hand from
>   Github if you prefer not to run it, and of course, you can also
>   build from source using the bootstrap scripts.> 
>> If a package has 'rm -rf $BUILD/', or equivalent ocaml code, are
>> its ill>> effects confined when BUILD is unset?
> 
> yes, that's the whole point of the sandboxing that was
> introduced in 2.0> 
>> Can the build process grab screenshots from the background?
> 
> not sure. Probably not on Linux since we use a different process
> space, but maybe on OSX. In anycase, since network access is blocked
> in both cases, that wouldn't do much harm.> 
>> One reason I like make is, if the Makefile is simple you know what
>> it's going to do.> 
> I would object that opam package definition files (`opam` or
> `foo.opam`) should be at least as straightforward to read even if you
> have never seen the syntax, are less error-prone, and are generally
> much shorter. Just look for the "build:" and "install:" parts. But I
> agree you need to know first to look at them, and since they are
> generally an indirection to some build-system (`make`, `dune`,
> `topkg`...), you would just start digging...> 
>> Also, the xkcd on standards seems relevant: https://xkcd.com/927/
> 
> We have __actually__ been converging as of late, though.
> 
> 
> Best,
> Louis Gesbert — OCamlPro
> 
> 
> 
> [1] https://github.com/projectatomic/bubblewrap
> [2] https://github.com/ocaml/opam/blob/master/src/state/shellscripts/bwrap.sh> and
> https://github.com/ocaml/opam/blob/master/src/state/shellscripts/sandbox_exec.sh> [3] https://github.com/hannesm/conex
> 
>> Whatever one true packaging system we use, I want to trust it not to
>> let the build process mess up my system.  For example, I see opam
>> makes some attempt to contain the build process on some systems.
>> It's not clear to me how much it does and how effective.  And it
>> appaerntly does not work on BSD.  If a package has 'rm -rf $BUILD/',
>> or equivalent ocaml code, are its ill effects confined when BUILD is
>> unset?  Can the build process grab screenshots from the background?
>> And so on.>> 
>> One reason I like make is, if the Makefile is simple you know what
>> it's going to do.>> 
>> Also, the xkcd on standards seems relevant: https://xkcd.com/927/
>> 
>>> On Nov 26, 2018, at 05:14 , Oliver Bandel <oliver@first.in-
>>> berlin.de> wrote:>>> 
>>> Hello,
>>> 
>>> a while ago it looked like there were not enough build- and
>>> installation-tools>>> for OCaml. I remember some discussions about that.
>>> 
>>> Now it seems to me that there are a lot of them.
>>> So, developers can pick the one they know about.
>>> 
>>> For all these tools there might be good reasons to use them,
>>> and those>>> developers who looked at these tools and choose them for their
>>> projects, will>>> know them well enough.
>>> 
>>> The situation differs, if one wants to package the written software,>>> and one needs to know many of those tools, just to compile the
>>> stuff.>>> So, when one just wants to compile and install some software,
>>> just for that, it would take much effort to learn the different build-
>>> tools.>>> 
>>> So, packaging has become more complicated, even though for the
>>> developers>>> these tools may save time.
>>> 
>>> It would be nice if people who used one of the many new
>>> building tools>>> could provide a Makefile that allows just to type
>>> "make" and "make install", instead of expecting everyone who wants
>>> to compile>>> the software to first learn just-another-build-tool.
>>> 
>>> Also it would be good, to mention early, which installation tools
>>> (make-dependencies)>>> are in use, and too mention needed packages (opam or others) to just
>>> build the stuff.>>> 
>>> Thanks and regards,
>>> Oliver Bandel
>>> 
>> 
>> 
>> 
> Email had 1 attachment:


>  * signature.asc 1k (application/pgp-signature)

[-- Attachment #2: Type: text/html, Size: 8228 bytes --]

[Caml-list] Build-/Installation-Tools - not enogh of them?

[Edwin Török]

On 30/11/2018 16:31, Louis Gesbert wrote:
>> - John F Carr, 27/11/2018 13:40 -
>> I have a related request.  I am not a trusting person.  I do not like "curl | sudo sh" type installation methods.
> You're not the only one :)
> Some notes on opam's security model:
>
> [...]
> - we do advertise `curl | sh` on the installation page as the easiest entry point, but the script is quite trivial and only uses root to copy to your prefix; it's very easy to fetch the binary by hand from Github if you prefer not to run it, and of course, you can also build from source using the bootstrap scripts.


Malicious attacks aren't the only reason why 'curl | sh' is dangerous,
it lacks basic integrity checks before execution:

* the curl download could be interrupted due to any number reasons, and
then the shell will attempt to execute the incomplete lines. Most of the
time that is harmless, but what if the script contained a 'rm -rf
${FOO}/${BAR}' that gets truncated to 'rm -rf ${FOO}/', such truncation
scenarios are probably not tested by a CI

* the server could reply with something else than your file, not
necessarily due to malicious reasons. Consider what would happen if
github has a hiccup and shows replies with https://github.com/503.html
that gets piped into the shell, which attempts to execute it: are we
sure it doesn't have any unintended consequences (or someone might put a
joke about rm in one of the error pages some day)


A slightly better approach would be:

curl -sL --fail
https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh -o
install-opam.sh && sh install-opam.sh


Although curl's manpage says that --fail is not fail-safe, and it could
still return success for a 401 error sometimes.


This command would solve the concerns above but then you'd have to keep
the webpage and the repo in sync:

curl -sL
https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh -o
install.sh-opam.tmp && echo
'0ebdb94df8940f838727bd12728d778a4a67e8006db3df330b4e0171c7f29a81 
install.sh-opam.tmp' | sha256sum -c && mv install.sh-opam.tmp
install-opam.sh && sh install-opam.sh

Perhaps the checksum file could be hosted in the repo, and point people
to run the script from a particular tag on the opam repo (to avoid race
conditions between fetching the checksum file and the file itself).


In the end I'm not sure if the added complication is worth it, might be
easier to just point people to the releases page where the install.sh
could be downloaded from and executed.

If you do plan to keep 'curl | sh', I'd suggest to include at least the
--fail from above.


Best regards,

--Edwin