[Caml-list] We need a rich standard library distributed with OCaml, really

[Caml-list] We need a rich standard library distributed with OCaml, really

[Hongbo Zhang]

[-- Attachment #1: Type: text/plain, Size: 2772 bytes --]

Dear OCaml developers,
    I would like to spend one hour in writing down my experience that why I had to write some small utilities again and again, since this happened so many times that I believe you might come across such issues before.
    I am doing some compiler hacking tonight, I needed a utility function “String.split” which split a string into a list of strings by whitespace, it is just one liner if you use str library. However, since I am doing some low level stuff, I would try to avoid such big dependency, and I am pretty sure that I have ever written it  for at least three times, I just hoped that I could get it done quickly, so I am looking into batteries that if I can steal some code, I have to copy some code instead of depend on batteries, batteries is too big for my projects. `BatString.nsplit` seems to fit for what I needed, I copied the definition of `BatString.nsplit` into REPL, no luck, it depends on some previously defined functions, then I copied the whole module `BatString` into REPL, still no luck, it depended on another module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc thrown-away `String.split` function again.
   OCaml is my favorite programming language, and I am very productive at it, however, I was annoyed by such things from time to time. We do have four *standard libraries* alternatives: batteries, core, extlib and ocaml-containers. In my opinion, none of them matches the beauty of the OCaml language itself and probably will never catch up if we don’t do anything. 
    Note that I don’t want to be offensive to any of these libraries, just my personal feedback that why I think it is not a good standard library, I appreciated a lot to people who contribute their precious time in maintaining these libraries, better than nothing : )
    - Batteries(extlib)
      It’s big with dependencies between different modules (OCaml does not have a good story in dead code elimination), some of its modules are of low quality, for example, batEnum is used everywhere while its implementation is buggy. batIO makes things even worse since it is not compatible with standard library, some type signatures mismatched IIRC.
    - ocaml-containers
      Mostly one man’s project
    - core
      I believe core has high quality, however, it suffers the same problem as batteries, a big dependency. There is a blocking issue, its development process is opaque, for an open source community, I would prefer a standard library developed in an open environment. 
    I am not expecting that we could have a  standard library as rich as python, but for some utilities, I do believe that shipped with standard library or officially supported is the best solution.
   Thanks — Hongbo 

[-- Attachment #2: Type: text/html, Size: 3390 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Christoph Höger]

That just came to mind:

https://xkcd.com/927/

I'd say the best way to deal with this (rather strange) situation would
be to merge the existing libraries (maybe gradually) into the std library.

If the community at large is indeed interested in this, one should
probably think about crowdfunding such an effort. However, to avoid the
xkcd solution, first and formost at least 2 of the libraries above
should agree on that merger in principle.

and btw: core is openly developed here:
https://github.com/janestreet/core

-- 
Christoph Höger

Technische Universität Berlin
Fakultät IV - Elektrotechnik und Informatik
Übersetzerbau und Programmiersprachen

Sekr. TEL12-2, Ernst-Reuter-Platz 7, 10587 Berlin

Tel.: +49 (30) 314-24890
E-Mail: christoph.hoeger@tu-berlin.de

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Anthony Tavener]

[-- Attachment #1: Type: text/plain, Size: 4695 bytes --]

I tend to live-in and build my own ecosystem anyway, so I haven't used any
of these libraries. But I've looked at them... and I think "containers"
goes for a more self-contained-module approach like you were looking for
with string split... Looking at it now I see the 'sep' function does use a
bit from within the CCParse module it's defined in -- but that module is
built on the parser combinators it defines. I think the individual module
can be taken as-is, or parts of it, without hunting down other module
dependencies.

This doesn't address the core problem you raise. Haven't thought much about
it, as I'm one of the few(?) who is okay with the sparse standard library.
For games, it's pretty much tradition that you write your own stdlib anyway
-- things end up being different... stressing particular use-cases or
performance characteristics. And, overall, not suitable for general
purposes. DBuenzli has contributed a lot which is actually quite
game-relevant, and his style is standalone modules. You could consider his
collection of "libraries" (modules really) as a library in it's own right.
:) Maybe that style is a good model? Come to think of it, the OCaml stdlib
modules are generally quite standalone, aside from Pervasives, aren't they?
So maybe ccube-containers and dbuenzli's style are really very OCaml-ish.

My own stuff, by comparison, is horribly layered with dependencies. I
generated a dot-graph showing dependencies and nearly everything depends on
another module, often which depend on others. I don't like redundant code,
and this is the result... but it makes for modules which cannot be easily
teased apart. Probably similar to Batteries-style.

-Tony


On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com>
wrote:

> Dear OCaml developers,
>     I would like to spend one hour in writing down my experience that why
> I had to write some small utilities again and again, since this happened so
> many times that I believe you might come across such issues before.
>     I am doing some compiler hacking tonight, I needed a utility function
> “String.split” which split a string into a list of strings by whitespace,
> it is just one liner if you use str library. However, since I am doing some
> low level stuff, I would try to avoid such big dependency, and I am pretty
> sure that I have ever written it  for at least three times, I just hoped
> that I could get it done quickly, so I am looking into batteries that if I
> can steal some code, I have to copy some code instead of depend on
> batteries, batteries is too big for my projects. `BatString.nsplit` seems
> to fit for what I needed, I copied the definition of `BatString.nsplit`
> into REPL, no luck, it depends on some previously defined functions, then I
> copied the whole module `BatString` into REPL, still no luck, it depended
> on another module `BatReturn`, then I stopped here, it’s time to write my
> own ad-hoc thrown-away `String.split` function again.
>    OCaml is my favorite programming language, and I am very productive at
> it, however, I was annoyed by such things from time to time. We do have
> four *standard libraries* alternatives: batteries, core, extlib and
> ocaml-containers. In my opinion, none of them matches the beauty of the
> OCaml language itself and probably will never catch up if we don’t do
> anything.
>     Note that I don’t want to be offensive to any of these libraries, just
> my personal feedback that why I think it is not a good standard library, *I
> appreciated a lot to people who contribute their precious time in
> maintaining these libraries, better than nothing* : )
>     - Batteries(extlib)
>       It’s big with dependencies between different modules (OCaml does not
> have a good story in dead code elimination), some of its modules are of low
> quality, for example, batEnum is used everywhere while its implementation
> is buggy. batIO makes things even worse since it is not compatible with
> standard library, some type signatures mismatched IIRC.
>     - ocaml-containers
>       Mostly one man’s project
>     - core
>       I believe core has high quality, however, it suffers the same
> problem as batteries, a big dependency. There is a blocking issue, its
> development process is opaque, for an open source community, I would prefer
> a standard library developed in an open environment.
>     I am not expecting that we could have a  standard library as rich as
> python, but for some utilities, I do believe that shipped with standard
> library or officially supported is the best solution.
>    Thanks — Hongbo
>

[-- Attachment #2: Type: text/html, Size: 5170 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Gabriel Scherer]

Hongbo, you are saying at the same time that:
- you want a *rich* (in particular, large) standard library
- existing third-party base libraries are too large for your taste

If we magically imported one of those libraries in the OCaml
distribution, it would still be large, and it would still have
inter-dependencies between its modules. Why are you unwilling to
depend on a library provided outside the distribution?

Depending on external libraries used to be a problem when there was no
consensus on OCaml packaging (no longer than a few years ago people
where reluctant to even use ocamlfind). We now have a reasonable
consensus on a packaging story, and installing third-party libraries
has never been as easy as it is today -- except on Windows, meh.
I think you should embrace the idea of depending on software outside
the OCaml distribution.

(I understand there are legal issues that make it difficult for some
companies to use third-party software. It is not our job to work in
stead of their lawyers.)

Many of the issues people have with the library distributed with in
the compiler distribution come from the fact that it is included in
the compiler distribution -- and thus imposes a maintenance burden on
the same group, same backward-compatibility constraints, etc. The
general movements in the current years is to make the compiler
distribution *smaller*, not *larger*, to avoid these problems.


I think the criticism you make of existing standard library
alternatives is fair (at least the Batteries part corresponds to
something that I think is consensual¹). My preferred solution to help
solve these issues is to contribute work to improve these libraries.

¹: Simon Cruanes has started good work to split Batteries in smaller
modules that do not depend on each other. This effort is currently
stalled, in large part by my personal fault (I was too
perfectionist in hoping for doing this and at the same time preserving
backward-functionality). Simon may have made the good call in starting
his own parallel library effort suiting his relatively opinionated
view of how things should be. I hope to get more time to devote to
Batteries at some point, and resolve that tension by proposing a good
compromise (integrating most of Simon's work) for a v3 release.

There remain the issue that having several "base libraries" risks
fragmenting the community in incompatible islands of software usage.
It is true that shoving stuff into the compiler distribution is a way
to resolve this excess of choice by authority, but it is manifest that
no one currently wants to bear this authority and the responsibilities
that come with it. (Except possibly on very localized issues, as the
`result` type that is being integrated in 4.03+dev).

I think the way forward is to have smaller independent packages that
do not require so large a buy-in and consensus. We could have *one*
package that provides many useful functions for lists, and one
separate package with many useful functions for strings. In this style
Daniel Bünzli beautifully documented small packages, or David Sheets
doing-one-thing libraries would be role models to follow. This wasn't
an option when Batteries or Core were started, because the packaging
story was too confused, but it is now and we should take advantage of
it. Given time (and help?) I hope to evolve Batteries towards this
model.

(Small independent packages have their own issues ("cabal hell"), so
maybe it's a case of seeing the greener pasture on the other side of
the fence. I think that's at least worth trying.)

On Thu, Aug 27, 2015 at 9:18 AM, Anthony Tavener
<anthony.tavener@gmail.com> wrote:
> I tend to live-in and build my own ecosystem anyway, so I haven't used any
> of these libraries. But I've looked at them... and I think "containers" goes
> for a more self-contained-module approach like you were looking for with
> string split... Looking at it now I see the 'sep' function does use a bit
> from within the CCParse module it's defined in -- but that module is built
> on the parser combinators it defines. I think the individual module can be
> taken as-is, or parts of it, without hunting down other module dependencies.
>
> This doesn't address the core problem you raise. Haven't thought much about
> it, as I'm one of the few(?) who is okay with the sparse standard library.
> For games, it's pretty much tradition that you write your own stdlib anyway
> -- things end up being different... stressing particular use-cases or
> performance characteristics. And, overall, not suitable for general
> purposes. DBuenzli has contributed a lot which is actually quite
> game-relevant, and his style is standalone modules. You could consider his
> collection of "libraries" (modules really) as a library in it's own right.
> :) Maybe that style is a good model? Come to think of it, the OCaml stdlib
> modules are generally quite standalone, aside from Pervasives, aren't they?
> So maybe ccube-containers and dbuenzli's style are really very OCaml-ish.
>
> My own stuff, by comparison, is horribly layered with dependencies. I
> generated a dot-graph showing dependencies and nearly everything depends on
> another module, often which depend on others. I don't like redundant code,
> and this is the result... but it makes for modules which cannot be easily
> teased apart. Probably similar to Batteries-style.
>
> -Tony
>
>
> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com>
> wrote:
>>
>> Dear OCaml developers,
>>     I would like to spend one hour in writing down my experience that why
>> I had to write some small utilities again and again, since this happened so
>> many times that I believe you might come across such issues before.
>>     I am doing some compiler hacking tonight, I needed a utility function
>> “String.split” which split a string into a list of strings by whitespace, it
>> is just one liner if you use str library. However, since I am doing some low
>> level stuff, I would try to avoid such big dependency, and I am pretty sure
>> that I have ever written it  for at least three times, I just hoped that I
>> could get it done quickly, so I am looking into batteries that if I can
>> steal some code, I have to copy some code instead of depend on batteries,
>> batteries is too big for my projects. `BatString.nsplit` seems to fit for
>> what I needed, I copied the definition of `BatString.nsplit` into REPL, no
>> luck, it depends on some previously defined functions, then I copied the
>> whole module `BatString` into REPL, still no luck, it depended on another
>> module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
>> thrown-away `String.split` function again.
>>    OCaml is my favorite programming language, and I am very productive at
>> it, however, I was annoyed by such things from time to time. We do have four
>> *standard libraries* alternatives: batteries, core, extlib and
>> ocaml-containers. In my opinion, none of them matches the beauty of the
>> OCaml language itself and probably will never catch up if we don’t do
>> anything.
>>     Note that I don’t want to be offensive to any of these libraries, just
>> my personal feedback that why I think it is not a good standard library, I
>> appreciated a lot to people who contribute their precious time in
>> maintaining these libraries, better than nothing : )
>>     - Batteries(extlib)
>>       It’s big with dependencies between different modules (OCaml does not
>> have a good story in dead code elimination), some of its modules are of low
>> quality, for example, batEnum is used everywhere while its implementation is
>> buggy. batIO makes things even worse since it is not compatible with
>> standard library, some type signatures mismatched IIRC.
>>     - ocaml-containers
>>       Mostly one man’s project
>>     - core
>>       I believe core has high quality, however, it suffers the same
>> problem as batteries, a big dependency. There is a blocking issue, its
>> development process is opaque, for an open source community, I would prefer
>> a standard library developed in an open environment.
>>     I am not expecting that we could have a  standard library as rich as
>> python, but for some utilities, I do believe that shipped with standard
>> library or officially supported is the best solution.
>>    Thanks — Hongbo
>
>

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Romain Bardou]

> There remain the issue that having several "base libraries" risks
> fragmenting the community in incompatible islands of software usage.
> It is true that shoving stuff into the compiler distribution is a way
> to resolve this excess of choice by authority, but it is manifest that
> no one currently wants to bear this authority and the responsibilities
> that come with it. (Except possibly on very localized issues, as the
> `result` type that is being integrated in 4.03+dev).
>
> I think the way forward is to have smaller independent packages that
> do not require so large a buy-in and consensus. We could have *one*
> package that provides many useful functions for lists, and one
> separate package with many useful functions for strings. In this style
> Daniel Bünzli beautifully documented small packages, or David Sheets
> doing-one-thing libraries would be role models to follow. This wasn't
> an option when Batteries or Core were started, because the packaging
> story was too confused, but it is now and we should take advantage of
> it. Given time (and help?) I hope to evolve Batteries towards this
> model.

I agree about smaller, independent packages. This is a very general API 
design principle: avoid coupling (the fact that using a module implies 
using another). This may be the main reason I avoid external libraries. 
For instance, Martin Jambon's Yojson depends on biniou, cppo and 
easy-format. I believe Martin is an awesome programmer but this 
particular point just baffles me as there is absolutely no need for 
*any* external dependency to solve such a simple problem (JSON parsing, 
pretty-printing and AST constructors). I understand that Martin wants to 
reuse its own code and be able to integrate Yojson easily with other 
libraries of his, and that is great. For him and users of his other 
libraries. Not for those who just want a JSON parser and have had to 
install all dependencies manually on Windows.

Lwt is another example. While it is obviously an awesome library, using 
it is not a decision to take lightly.

There is something that I love about Yojson though: the main type (the 
JSON AST) uses polymorphic variants. This means that it is possible to 
write a library which is capable of integrating with Yojson without 
actually depending on it. You just copy-paste the type definition and 
use it. Then your user can use any JSON parser and printer. He can use 
Yojson, or another library which happens to use a supertype. Worst case 
scenario, he takes 5 minutes to write functions to convert the Yojson 
AST to and from another JSON library's type.

Writing code which can integrate with another library without actually 
depending on it will scare users less and make more people use your 
library. Here are some more examples.

- Use standard library types like option, list, or array instead of 
trying to generalize to some enumeration concept.

- If the previous point is not possible because the enumeration is too 
big to be constructed before use, consider providing an iter function. 
Such a function can be easily used with a library such as Simon Cruanes' 
sequence library, but your library does not need to depend on it.

- Functors allow your code to depend on any implementation. 
Unfortunately functors are a pain both to write and to use. One of the 
promises of the module system is to be able to replace a module with 
another one with the same signature, but how many times have you seen 
that being done in practice (without functors)? The build system and 
namespace issues do not help you to do that. Especially if you want to 
replace the dependency of a third-party library. Say, use a library 
which depends on Lwt but make it use your implementation of Lwt instead, 
without having to manually reconfigure and recompile the library, just 
by using the compiled version from opam. This is even harder if your Lwt 
implementation does not provide the same interface as Lwt except for the 
part which is used by the library.

Coupling is not bad just between libraries. For instance, you may be 
tempted to write a function "split_list" to split a string into a list, 
by using another function "split_pair" which splits a string into a pair 
(at the first occurrence only). But an independent implementation of 
split_list is much more efficient (if split_pair copies the two 
substrings), can more easily be sent to the toplevel for testing, and 
can be copy-pasted easily between programs (until you decide to put it 
into a library). But more importantly, if you need to change 
"split_pair" for some reason, you will have to check all functions which 
use it, including "split_list". Yes, sharing code allows to share bug 
fixes. But it also makes it easier to break stuff accidentally.

-- 
Romain Bardou

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Martin DeMello]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

On Thu, Aug 27, 2015 at 3:35 AM, Romain Bardou <romain.bardou@inria.fr>
wrote:
>
> I agree about smaller, independent packages. This is a very general API
> design principle: avoid coupling (the fact that using a module implies
> using another). This may be the main reason I avoid external libraries. For
> instance, Martin Jambon's Yojson depends on biniou, cppo and easy-format. I
> believe Martin is an awesome programmer but this particular point just
> baffles me as there is absolutely no need for *any* external dependency to
> solve such a simple problem (JSON parsing, pretty-printing and AST
> constructors). I understand that Martin wants to reuse its own code and be
> able to integrate Yojson easily with other libraries of his, and that is
> great. For him and users of his other libraries. Not for those who just
> want a JSON parser and have had to install all dependencies manually on
> Windows.
>

Part of the promise of an ecosystem of small libraries is that you should
be able to use them for any code you write, including other libraries. This
is not the same thing as API coupling; as an end user of library C you
should be able to use it without caring about whether it is self-contained
in terms of code or whether it uses libraries A and B internally. The fact
that dependencies need to be installed manually on windows is a failure of
the ocaml windows ecosystem (which I'm definitely sympathetic towards; I
once had to manually copy a bunch of code from batteries into my own
project just to avoid depending on it); it is not a sign that libraries
need not to depend on each other.

martin

[-- Attachment #2: Type: text/html, Size: 1940 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Yotam Barnoy]

[-- Attachment #1: Type: text/plain, Size: 3419 bytes --]

I'd like to mention the merits of not having a large standard library.
Consider the evolution of OCaml. Many of the paradigms with which OCaml was
born, such as using exceptions everywhere, have gone out of favor.
Hopefully, sometime in the near future, we'll have Modular Implicits
integrated into the language. Assuming this happens, it will almost
certainly impact what would be expected to belong in a standard library.
The official standard library already carries around with it vestigial
organs, such as the Stream module. This will only get worse if we add to it.

At the same time, I recognize a need for a library to represent a large
collection of data types and the functions over said types. It cannot all
be miniature libraries in opam IMO -- for basic programming, there should
be a curated source of basic and even some extended functionality.

What seems to me better than a built-in standard library, though, is a
reference to the best currently available libraries in each area, including
a 'standard' collection of utilities/data structures. Such a reference
could include space for the community to debate the pros and cons of
various libraries, and perhaps even a voting system to indicate to
potential users what the community thinks about said libraries. This is
something I currently have trouble with in opam, since I have no idea if a
given library is a) ancient and unmaintained b) the best in its class c)
rising in popularity d) written by a pro and solid, even if not used much.
The closest I have to that in opam is number of downloads, but given how
much the ecosystem now relies on opam, I think a more advanced display is
needed.

-Yotam

On Thu, Aug 27, 2015 at 3:55 PM, Martin DeMello <martindemello@gmail.com>
wrote:

> On Thu, Aug 27, 2015 at 3:35 AM, Romain Bardou <romain.bardou@inria.fr>
> wrote:
>>
>> I agree about smaller, independent packages. This is a very general API
>> design principle: avoid coupling (the fact that using a module implies
>> using another). This may be the main reason I avoid external libraries. For
>> instance, Martin Jambon's Yojson depends on biniou, cppo and easy-format. I
>> believe Martin is an awesome programmer but this particular point just
>> baffles me as there is absolutely no need for *any* external dependency to
>> solve such a simple problem (JSON parsing, pretty-printing and AST
>> constructors). I understand that Martin wants to reuse its own code and be
>> able to integrate Yojson easily with other libraries of his, and that is
>> great. For him and users of his other libraries. Not for those who just
>> want a JSON parser and have had to install all dependencies manually on
>> Windows.
>>
>
> Part of the promise of an ecosystem of small libraries is that you should
> be able to use them for any code you write, including other libraries. This
> is not the same thing as API coupling; as an end user of library C you
> should be able to use it without caring about whether it is self-contained
> in terms of code or whether it uses libraries A and B internally. The fact
> that dependencies need to be installed manually on windows is a failure of
> the ocaml windows ecosystem (which I'm definitely sympathetic towards; I
> once had to manually copy a bunch of code from batteries into my own
> project just to avoid depending on it); it is not a sign that libraries
> need not to depend on each other.
>
> martin
>

[-- Attachment #2: Type: text/html, Size: 4161 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Drup]

[-- Attachment #1: Type: text/plain, Size: 3959 bytes --]

https://github.com/rizo/awesome-ocaml ?

(my opinion is that awesome-ocaml is not curated enough and lists things 
that are not stable nor usable, but at least it's a good step)


Le 27/08/2015 22:10, Yotam Barnoy a écrit :
> I'd like to mention the merits of not having a large standard library. 
> Consider the evolution of OCaml. Many of the paradigms with which 
> OCaml was born, such as using exceptions everywhere, have gone out of 
> favor. Hopefully, sometime in the near future, we'll have Modular 
> Implicits integrated into the language. Assuming this happens, it will 
> almost certainly impact what would be expected to belong in a standard 
> library. The official standard library already carries around with it 
> vestigial organs, such as the Stream module. This will only get worse 
> if we add to it.
>
> At the same time, I recognize a need for a library to represent a 
> large collection of data types and the functions over said types. It 
> cannot all be miniature libraries in opam IMO -- for basic 
> programming, there should be a curated source of basic and even some 
> extended functionality.
>
> What seems to me better than a built-in standard library, though, is a 
> reference to the best currently available libraries in each area, 
> including a 'standard' collection of utilities/data structures. Such a 
> reference could include space for the community to debate the pros and 
> cons of various libraries, and perhaps even a voting system to 
> indicate to potential users what the community thinks about said 
> libraries. This is something I currently have trouble with in opam, 
> since I have no idea if a given library is a) ancient and unmaintained 
> b) the best in its class c) rising in popularity d) written by a pro 
> and solid, even if not used much. The closest I have to that in opam 
> is number of downloads, but given how much the ecosystem now relies on 
> opam, I think a more advanced display is needed.
>
> -Yotam
>
> On Thu, Aug 27, 2015 at 3:55 PM, Martin DeMello 
> <martindemello@gmail.com <mailto:martindemello@gmail.com>> wrote:
>
>     On Thu, Aug 27, 2015 at 3:35 AM, Romain Bardou
>     <romain.bardou@inria.fr <mailto:romain.bardou@inria.fr>> wrote:
>
>         I agree about smaller, independent packages. This is a very
>         general API design principle: avoid coupling (the fact that
>         using a module implies using another). This may be the main
>         reason I avoid external libraries. For instance, Martin
>         Jambon's Yojson depends on biniou, cppo and easy-format. I
>         believe Martin is an awesome programmer but this particular
>         point just baffles me as there is absolutely no need for *any*
>         external dependency to solve such a simple problem (JSON
>         parsing, pretty-printing and AST constructors). I understand
>         that Martin wants to reuse its own code and be able to
>         integrate Yojson easily with other libraries of his, and that
>         is great. For him and users of his other libraries. Not for
>         those who just want a JSON parser and have had to install all
>         dependencies manually on Windows.
>
>
>     Part of the promise of an ecosystem of small libraries is that you
>     should be able to use them for any code you write, including other
>     libraries. This is not the same thing as API coupling; as an end
>     user of library C you should be able to use it without caring
>     about whether it is self-contained in terms of code or whether it
>     uses libraries A and B internally. The fact that dependencies need
>     to be installed manually on windows is a failure of the ocaml
>     windows ecosystem (which I'm definitely sympathetic towards; I
>     once had to manually copy a bunch of code from batteries into my
>     own project just to avoid depending on it); it is not a sign that
>     libraries need not to depend on each other.
>
>     martin
>
>


[-- Attachment #2: Type: text/html, Size: 6510 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 3294 bytes --]

>
>
> What seems to me better than a built-in standard library, though, is a
> reference to the best currently available libraries in each area, including
> a 'standard' collection of utilities/data structures.
>
Actually, even getting to know the list of libraries in a given area would
already be great. The OPAM repo has a search functionality which is already
helpful for that. Having a more systematic and principled [1] use of the
"tag" field could help too.


> Such a reference could include space for the community to debate the pros
> and cons of various libraries, and perhaps even a voting system to indicate
> to potential users what the community thinks about said libraries. This is
> something I currently have trouble with in opam, since I have no idea if a
> given library is a) ancient and unmaintained b) the best in its class c)
> rising in popularity d) written by a pro and solid, even if not used much.
> The closest I have to that in opam is number of downloads, but given how
> much the ecosystem now relies on opam, I think a more advanced display is
> needed.
>

Another indicator of the popularity of a library is the number of reverse
dependencies. This is something that can be checked on the OPAM repository
(see [0] for instance). I guess having this information summarized in a
table  (possibly filtered with tags) could already give a hint.

[0] http://opam.ocaml.org/packages/batteries/batteries.2.3.1/
[1] I mean choosing tags from a controlled vocabulary.



>
> -Yotam
>
> On Thu, Aug 27, 2015 at 3:55 PM, Martin DeMello <martindemello@gmail.com>
> wrote:
>
>> On Thu, Aug 27, 2015 at 3:35 AM, Romain Bardou <romain.bardou@inria.fr>
>> wrote:
>>>
>>> I agree about smaller, independent packages. This is a very general API
>>> design principle: avoid coupling (the fact that using a module implies
>>> using another). This may be the main reason I avoid external libraries. For
>>> instance, Martin Jambon's Yojson depends on biniou, cppo and easy-format. I
>>> believe Martin is an awesome programmer but this particular point just
>>> baffles me as there is absolutely no need for *any* external dependency to
>>> solve such a simple problem (JSON parsing, pretty-printing and AST
>>> constructors). I understand that Martin wants to reuse its own code and be
>>> able to integrate Yojson easily with other libraries of his, and that is
>>> great. For him and users of his other libraries. Not for those who just
>>> want a JSON parser and have had to install all dependencies manually on
>>> Windows.
>>>
>>
>> Part of the promise of an ecosystem of small libraries is that you should
>> be able to use them for any code you write, including other libraries. This
>> is not the same thing as API coupling; as an end user of library C you
>> should be able to use it without caring about whether it is self-contained
>> in terms of code or whether it uses libraries A and B internally. The fact
>> that dependencies need to be installed manually on windows is a failure of
>> the ocaml windows ecosystem (which I'm definitely sympathetic towards; I
>> once had to manually copy a bunch of code from batteries into my own
>> project just to avoid depending on it); it is not a sign that libraries
>> need not to depend on each other.
>>
>> martin
>>
>
>

[-- Attachment #2: Type: text/html, Size: 4752 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Raoul Duke]

As a thought experiment, consider each function being its own
component. That isn't really an idea beyond the pale, in my opinion.
So if that's the case, then why can't one component depend upon
another. The boundaries we assume are reasonable (everything in a
namespace or module or whatever) are completely arbitrary. So yes if
library X that is 4k relies on 50 other libs that are around 1MB each,
sure that is (perhaps, even there it is context sensitive) lame. But
1k + 2 other 1K libs, not so much. etc.

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Martin Jambon]

On 08/27/2015 03:35 AM, Romain Bardou wrote:
> I agree about smaller, independent packages. This is a very general API
> design principle: avoid coupling (the fact that using a module implies
> using another). This may be the main reason I avoid external libraries.
> For instance, Martin Jambon's Yojson depends on biniou, cppo and
> easy-format. I believe Martin is an awesome programmer but this
> particular point just baffles me as there is absolutely no need for
> *any* external dependency to solve such a simple problem (JSON parsing,
> pretty-printing and AST constructors).

Thanks for the praise!

The reason why there are several packages is that they offer different 
functionality. Not all of them are needed everywhere. Not all of them 
can be built on every platform.

> I understand that Martin wants to
> reuse its own code and be able to integrate Yojson easily with other
> libraries of his, and that is great. For him and users of his other
> libraries. Not for those who just want a JSON parser and have had to
> install all dependencies manually on Windows.

Yojson is just the runtime for atdgen, but some people find it useful to 
use it as is. I'm sure atdgen runs on Windows, but if it didn't and it 
came with all its dependencies and the dependencies weren't available 
anywhere else, you wouldn't have any Yojson at all.

I was under the impression that the opam package manager was available 
for Windows. It takes care of the dependencies for the user, as package 
managers do.

Martin

[Caml-list] Fwd: We need a rich standard library distributed with OCaml, really

[Emmanuel Surleau]

[-- Attachment #1: Type: text/plain, Size: 5357 bytes --]

---------- Forwarded message ----------
From: Emmanuel Surleau <emmanuel.surleau@gmail.com>
Date: Thu, Aug 27, 2015 at 7:45 PM
Subject: Re: [Caml-list] We need a rich standard library distributed with
OCaml, really
To: Gabriel Scherer <gabriel.scherer@gmail.com>


Quoting Gabriel Scherer (2015-08-27 10:17:08)
> Hongbo, you are saying at the same time that:
> - you want a *rich* (in particular, large) standard library
> - existing third-party base libraries are too large for your taste
>
> If we magically imported one of those libraries in the OCaml
> distribution, it would still be large, and it would still have
> inter-dependencies between its modules. Why are you unwilling to
> depend on a library provided outside the distribution?
>
> Depending on external libraries used to be a problem when there was no
> consensus on OCaml packaging (no longer than a few years ago people
> where reluctant to even use ocamlfind). We now have a reasonable
> consensus on a packaging story, and installing third-party libraries
> has never been as easy as it is today -- except on Windows, meh.
> I think you should embrace the idea of depending on software outside
> the OCaml distribution.
>
> (I understand there are legal issues that make it difficult for some
> companies to use third-party software. It is not our job to work in
> stead of their lawyers.)
>
> Many of the issues people have with the library distributed with in
> the compiler distribution come from the fact that it is included in
> the compiler distribution -- and thus imposes a maintenance burden on
> the same group, same backward-compatibility constraints, etc. The
> general movements in the current years is to make the compiler
> distribution *smaller*, not *larger*, to avoid these problems.
>
>
> I think the criticism you make of existing standard library
> alternatives is fair (at least the Batteries part corresponds to
> something that I think is consensual¹). My preferred solution to help
> solve these issues is to contribute work to improve these libraries.
>
> ¹: Simon Cruanes has started good work to split Batteries in smaller
> modules that do not depend on each other. This effort is currently
> stalled, in large part by my personal fault (I was too
> perfectionist in hoping for doing this and at the same time preserving
> backward-functionality). Simon may have made the good call in starting
> his own parallel library effort suiting his relatively opinionated
> view of how things should be. I hope to get more time to devote to
> Batteries at some point, and resolve that tension by proposing a good
> compromise (integrating most of Simon's work) for a v3 release.
>
> There remain the issue that having several "base libraries" risks
> fragmenting the community in incompatible islands of software usage.
> It is true that shoving stuff into the compiler distribution is a way
> to resolve this excess of choice by authority, but it is manifest that
> no one currently wants to bear this authority and the responsibilities
> that come with it. (Except possibly on very localized issues, as the
> `result` type that is being integrated in 4.03+dev).
>
> I think the way forward is to have smaller independent packages that
> do not require so large a buy-in and consensus. We could have *one*
> package that provides many useful functions for lists, and one
> separate package with many useful functions for strings. In this style
> Daniel Bünzli beautifully documented small packages, or David Sheets
> doing-one-thing libraries would be role models to follow. This wasn't
> an option when Batteries or Core were started, because the packaging
> story was too confused, but it is now and we should take advantage of
> it. Given time (and help?) I hope to evolve Batteries towards this
> model.

(forwarding the original email to the list as I had sent only to Gabriel by
accident)

Well, as far collections go, you don't need "large libraries". You just
need a
to_gen method in collection types in the stdblib. The gen type itself
is trivially reimplementable, so it's not a hard dependency on ccube's gen
package, and gen itself is a single module that could, if there was the
will, be
integrated in the stdlib at little cost. This would greatly extend the
flexibility of existing data structures.

Another issue to consider is that you need to distinguish what is "sugar"
and
what is essential infrastructure. You do not want to farm out
infrastructure to
third parties. Otherwise, this puts maintainers of OCaml libraries in the
difficult position of choosing for their users. It's mostly a question of
data
types. If, for instance, the stdlib had sensible data types for handling
time
information, it would be acceptable to leave advanced time-handling
mechanics to
specialized libraries outside of the stdlib, as long as they were able to
rely
on a *lingua franca* of standard types.

Of course, all that wouldn't solve the design issues of the standard library
(eg, Str, exceptions everywhere, etc.), but adding gen (or at least hooks
for
gen) and a number of common data structures would really go a long way.

I wouldn't mind contributing a few hours working on that, assuming we could
reach some kind of consensus.

Cheers,

Emm

[-- Attachment #2: Type: text/html, Size: 6258 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Paolo Donadeo]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]

I'm possibly the only one considering a small, essential, base library a
*value* for the OCaml language and community?

The OCaml standard library let the user free to decide what library is the
best choice for each project. I'm currently using Batteries, or Core, for
very different tasks. And sometimes none of the two.

I'm very happy and grateful not to have an OCaml counterpart of the STL or,
worse, the Java Platform!


-- 
*Paolo*

[-- Attachment #2: Type: text/html, Size: 671 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Oliver Bandel]

Zitat von Paolo Donadeo <p.donadeo@gmail.com> (Thu, 27 Aug 2015  
23:17:50 +0200)

> I'm possibly the only one considering a small, essential, base library a
> *value* for the OCaml language and community?

No, I also prefer the slim lib approach ;-)


But nevertheless, some more functions would make sense to me too.

The String.split, mentioned by the orig-poster, is a good idea, IMHO.
Adding this as Feature-wish in the Bugtracking system would be a good  
idea, I think.

Some things need a while... sometimes too long, IMHO, e.g. feature  
wish Hashtbl.keys
is in the bugtracker since june 2013. :(
I hope it will be added before 2020 ;-)

But sometimes, it's surprising, what has ben added.
After looking at the documentation after long time (didn't awaited new  
things *ahem*)
I found String.trim for example.
Very useful, but I did not expected that this would be added.

So, even though I prefer a slim std-lib, some functions are still so common,
that they should be added.

I also was surprised to see ( |> ) was added.


Someone in this thread mentioned opt_apply (opt_eval) functions.
Very useful, and even though very simple, it's very powerful.
Especially combined with ( |> ).
( ( |> ) also is easy to write, but soooo helpful, that it's a good  
idea, to have it
  coming with OCaml system itself.)

So, opt_apply functions are of the same kind as ( |> ) ---> easy to write, but
should be part of the OCaml distribution.


For the advantages of opt_apply, see

    "Scott Wlaschin - Railway Oriented Programming -- error handling  
in functional languages"
       https://vimeo.com/97344498

I really enjoyed that talk.
It started me thinking about my code, seeing option-type not as
something strange, that sometimes makes sense, but as something,
which is really great and might be used more frequently ...

There are also slides accompaning the talk somewhere, but I don't have  
the link at hand.



>
> The OCaml standard library let the user free to decide what library is the
> best choice for each project. I'm currently using Batteries, or Core, for
> very different tasks. And sometimes none of the two.

I rely on OCaml's standard lib and a lot of smaller libs.
When thinking about Batteries, Core and others, I just could not decide
for one of them.
Looking at them in detail might eat more time than writing the needed  
code by myself.
And it saves much space, NOT to use BIG libraries... especially, if  
most of that stuff
is not needed.
I prefer finer grained solutions.
But of course, it's good to have all these libraries, just in case, it would
match the needs (of someone).


>
> I'm very happy and grateful not to have an OCaml counterpart of the STL or,
> worse, the Java Platform!

heheh.


Ciao,
   Oliver

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Oliver Bandel]

Zitat von Oliver Bandel <oliver@first.in-berlin.de> (Thu, 27 Aug 2015  
23:51:49 +0200)

[...]
> For the advantages of opt_apply, see
>
>    "Scott Wlaschin - Railway Oriented Programming -- error handling  
> in functional languages"
>       https://vimeo.com/97344498
>
> I really enjoyed that talk.
[...]

Seems to be a different talk, than the one which I remember.

Maybe this one I saw:


   Functional programming design patterns by Scott Wlaschin
     https://vimeo.com/113588389


But both cover the issue with option-type.


Ciao,
    Oliver

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Oliver Bandel]

Ah, this one was it:

Railway oriented programming: Error handling in functional languages  
by Scott Wlaschin
   https://vimeo.com/113707214


Slides:
   http://www.slideshare.net/ScottWlaschin/railway-oriented-programming



Ciao,
    Oliver


Zitat von Oliver Bandel <oliver@first.in-berlin.de> (Thu, 27 Aug 2015  
23:56:57 +0200)

> Zitat von Oliver Bandel <oliver@first.in-berlin.de> (Thu, 27 Aug  
> 2015 23:51:49 +0200)
>
> [...]
>> For the advantages of opt_apply, see
>>
>>   "Scott Wlaschin - Railway Oriented Programming -- error handling  
>> in functional languages"
>>      https://vimeo.com/97344498
>>
>> I really enjoyed that talk.
> [...]
>
> Seems to be a different talk, than the one which I remember.
>
> Maybe this one I saw:
>
>
>   Functional programming design patterns by Scott Wlaschin
>     https://vimeo.com/113588389
>
>
> But both cover the issue with option-type.
>
>
> Ciao,
>    Oliver
>
>
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Hongbo Zhang]

Hi Gabriel,
   To answer your questions in particular, standard library should not be considered a dependency, since for people who use your library, there is no cost. Yes, I agree that it would be even better when enriching the standard library, we would try to make it self contained.
   When I was at school, I can play with OPAM, and install any dependency if I like(thanks to OPAM), however, in the industry setting, esepcially a big company which takes IP in a serious way, it is damn hard to introduce a third party library, the time spent in convincing your legal department is more than you roll your own version.
   Thanks -- Hongbo
> On Aug 27, 2015, at 4:17 AM, Gabriel Scherer <gabriel.scherer@gmail.com> wrote:
> 
> Hongbo, you are saying at the same time that:
> - you want a *rich* (in particular, large) standard library
> - existing third-party base libraries are too large for your taste
> 
> If we magically imported one of those libraries in the OCaml
> distribution, it would still be large, and it would still have
> inter-dependencies between its modules. Why are you unwilling to
> depend on a library provided outside the distribution?
> 
> Depending on external libraries used to be a problem when there was no
> consensus on OCaml packaging (no longer than a few years ago people
> where reluctant to even use ocamlfind). We now have a reasonable
> consensus on a packaging story, and installing third-party libraries
> has never been as easy as it is today -- except on Windows, meh.
> I think you should embrace the idea of depending on software outside
> the OCaml distribution.
> 
> (I understand there are legal issues that make it difficult for some
> companies to use third-party software. It is not our job to work in
> stead of their lawyers.)
> 
> Many of the issues people have with the library distributed with in
> the compiler distribution come from the fact that it is included in
> the compiler distribution -- and thus imposes a maintenance burden on
> the same group, same backward-compatibility constraints, etc. The
> general movements in the current years is to make the compiler
> distribution *smaller*, not *larger*, to avoid these problems.
> 
> 
> I think the criticism you make of existing standard library
> alternatives is fair (at least the Batteries part corresponds to
> something that I think is consensual¹). My preferred solution to help
> solve these issues is to contribute work to improve these libraries.
> 
> ¹: Simon Cruanes has started good work to split Batteries in smaller
> modules that do not depend on each other. This effort is currently
> stalled, in large part by my personal fault (I was too
> perfectionist in hoping for doing this and at the same time preserving
> backward-functionality). Simon may have made the good call in starting
> his own parallel library effort suiting his relatively opinionated
> view of how things should be. I hope to get more time to devote to
> Batteries at some point, and resolve that tension by proposing a good
> compromise (integrating most of Simon's work) for a v3 release.
> 
> There remain the issue that having several "base libraries" risks
> fragmenting the community in incompatible islands of software usage.
> It is true that shoving stuff into the compiler distribution is a way
> to resolve this excess of choice by authority, but it is manifest that
> no one currently wants to bear this authority and the responsibilities
> that come with it. (Except possibly on very localized issues, as the
> `result` type that is being integrated in 4.03+dev).
> 
> I think the way forward is to have smaller independent packages that
> do not require so large a buy-in and consensus. We could have *one*
> package that provides many useful functions for lists, and one
> separate package with many useful functions for strings. In this style
> Daniel Bünzli beautifully documented small packages, or David Sheets
> doing-one-thing libraries would be role models to follow. This wasn't
> an option when Batteries or Core were started, because the packaging
> story was too confused, but it is now and we should take advantage of
> it. Given time (and help?) I hope to evolve Batteries towards this
> model.
> 
> (Small independent packages have their own issues ("cabal hell"), so
> maybe it's a case of seeing the greener pasture on the other side of
> the fence. I think that's at least worth trying.)
> 
> On Thu, Aug 27, 2015 at 9:18 AM, Anthony Tavener
> <anthony.tavener@gmail.com> wrote:
>> I tend to live-in and build my own ecosystem anyway, so I haven't used any
>> of these libraries. But I've looked at them... and I think "containers" goes
>> for a more self-contained-module approach like you were looking for with
>> string split... Looking at it now I see the 'sep' function does use a bit
>> from within the CCParse module it's defined in -- but that module is built
>> on the parser combinators it defines. I think the individual module can be
>> taken as-is, or parts of it, without hunting down other module dependencies.
>> 
>> This doesn't address the core problem you raise. Haven't thought much about
>> it, as I'm one of the few(?) who is okay with the sparse standard library.
>> For games, it's pretty much tradition that you write your own stdlib anyway
>> -- things end up being different... stressing particular use-cases or
>> performance characteristics. And, overall, not suitable for general
>> purposes. DBuenzli has contributed a lot which is actually quite
>> game-relevant, and his style is standalone modules. You could consider his
>> collection of "libraries" (modules really) as a library in it's own right.
>> :) Maybe that style is a good model? Come to think of it, the OCaml stdlib
>> modules are generally quite standalone, aside from Pervasives, aren't they?
>> So maybe ccube-containers and dbuenzli's style are really very OCaml-ish.
>> 
>> My own stuff, by comparison, is horribly layered with dependencies. I
>> generated a dot-graph showing dependencies and nearly everything depends on
>> another module, often which depend on others. I don't like redundant code,
>> and this is the result... but it makes for modules which cannot be easily
>> teased apart. Probably similar to Batteries-style.
>> 
>> -Tony
>> 
>> 
>> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com>
>> wrote:
>>> 
>>> Dear OCaml developers,
>>>    I would like to spend one hour in writing down my experience that why
>>> I had to write some small utilities again and again, since this happened so
>>> many times that I believe you might come across such issues before.
>>>    I am doing some compiler hacking tonight, I needed a utility function
>>> “String.split” which split a string into a list of strings by whitespace, it
>>> is just one liner if you use str library. However, since I am doing some low
>>> level stuff, I would try to avoid such big dependency, and I am pretty sure
>>> that I have ever written it  for at least three times, I just hoped that I
>>> could get it done quickly, so I am looking into batteries that if I can
>>> steal some code, I have to copy some code instead of depend on batteries,
>>> batteries is too big for my projects. `BatString.nsplit` seems to fit for
>>> what I needed, I copied the definition of `BatString.nsplit` into REPL, no
>>> luck, it depends on some previously defined functions, then I copied the
>>> whole module `BatString` into REPL, still no luck, it depended on another
>>> module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
>>> thrown-away `String.split` function again.
>>>   OCaml is my favorite programming language, and I am very productive at
>>> it, however, I was annoyed by such things from time to time. We do have four
>>> *standard libraries* alternatives: batteries, core, extlib and
>>> ocaml-containers. In my opinion, none of them matches the beauty of the
>>> OCaml language itself and probably will never catch up if we don’t do
>>> anything.
>>>    Note that I don’t want to be offensive to any of these libraries, just
>>> my personal feedback that why I think it is not a good standard library, I
>>> appreciated a lot to people who contribute their precious time in
>>> maintaining these libraries, better than nothing : )
>>>    - Batteries(extlib)
>>>      It’s big with dependencies between different modules (OCaml does not
>>> have a good story in dead code elimination), some of its modules are of low
>>> quality, for example, batEnum is used everywhere while its implementation is
>>> buggy. batIO makes things even worse since it is not compatible with
>>> standard library, some type signatures mismatched IIRC.
>>>    - ocaml-containers
>>>      Mostly one man’s project
>>>    - core
>>>      I believe core has high quality, however, it suffers the same
>>> problem as batteries, a big dependency. There is a blocking issue, its
>>> development process is opaque, for an open source community, I would prefer
>>> a standard library developed in an open environment.
>>>    I am not expecting that we could have a  standard library as rich as
>>> python, but for some utilities, I do believe that shipped with standard
>>> library or officially supported is the best solution.
>>>   Thanks — Hongbo
>> 
>> 

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Stéphane Glondu]

Le 27/08/2015 10:17, Gabriel Scherer a écrit :
> Depending on external libraries used to be a problem when there was no
> consensus on OCaml packaging (no longer than a few years ago people
> where reluctant to even use ocamlfind). We now have a reasonable
> consensus on a packaging story, and installing third-party libraries
> has never been as easy as it is today -- except on Windows, meh.
> I think you should embrace the idea of depending on software outside
> the OCaml distribution.

Depending on a multitude of external libraries makes packaging more
difficult. I am talking about "system" packaging (deb, rpm...) here, not
OPAM. And "install OPAM first" is not a very satisfactory way of
installing some software written in OCaml in a stable environment.


Cheers,

-- 
Stéphane

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Francois Berenger]

On 08/31/2015 06:06 PM, Stéphane Glondu wrote:
> Le 27/08/2015 10:17, Gabriel Scherer a écrit :
>> Depending on external libraries used to be a problem when there was no
>> consensus on OCaml packaging (no longer than a few years ago people
>> where reluctant to even use ocamlfind). We now have a reasonable
>> consensus on a packaging story, and installing third-party libraries
>> has never been as easy as it is today -- except on Windows, meh.
>> I think you should embrace the idea of depending on software outside
>> the OCaml distribution.
>
> Depending on a multitude of external libraries makes packaging more
> difficult. I am talking about "system" packaging (deb, rpm...) here, not
> OPAM. And "install OPAM first" is not a very satisfactory way of
> installing some software written in OCaml in a stable environment.

Why not?

In my experience, opam installs are quite reproducible.
You can even specify the version number you want to install.

The only problem I see is that opam install things in a user's home 
directory, which is not what you want for a system-wise installation of 
a software/library.

-- 
Regards,
Francois.
"When in doubt, use more types"

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Hendrik Boom]

On Mon, Aug 31, 2015 at 06:14:18PM +0200, Francois Berenger wrote:
> On 08/31/2015 06:06 PM, Stéphane Glondu wrote:
> >Le 27/08/2015 10:17, Gabriel Scherer a écrit :
> >>Depending on external libraries used to be a problem when there was no
> >>consensus on OCaml packaging (no longer than a few years ago people
> >>where reluctant to even use ocamlfind). We now have a reasonable
> >>consensus on a packaging story, and installing third-party libraries
> >>has never been as easy as it is today -- except on Windows, meh.
> >>I think you should embrace the idea of depending on software outside
> >>the OCaml distribution.
> >
> >Depending on a multitude of external libraries makes packaging more
> >difficult. I am talking about "system" packaging (deb, rpm...) here, not
> >OPAM. And "install OPAM first" is not a very satisfactory way of
> >installing some software written in OCaml in a stable environment.
> 
> Why not?
> 
> In my experience, opam installs are quite reproducible.
> You can even specify the version number you want to install.
> 
> The only problem I see is that opam install things in a user's home
> directory, which is not what you want for a system-wise installation
> of a software/library.

So there should be a superuser option, telling opam to install in the
system-wide installation area.  Of course that wouldn't work if you 
don't have the relevant permissions, not should it.  And the details of 
the superuser option (such as where to install what) may be 
system-dependent.

There are Debian packages that access the net during installation; most 
notably some device drivers on Debian, where the manufacturer releases 
its own installation procedure, but does not allow redistribution.  So 
the Debian package, during installation, downloads the manufacturer's 
installer and executes it.

No reason why the, say ocaml packages shouldn't do likewise using 
opam.  One bit of hackery could wrap all the available opam packages.

--- hendrik

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Ian Zimmerman]

On 2015-08-31 12:44 -0400, Hendrik Boom wrote:

> There are Debian packages that access the net during installation;
> most notably some device drivers on Debian, where the manufacturer
> releases its own installation procedure, but does not allow
> redistribution.  So the Debian package, during installation, downloads
> the manufacturer's installer and executes it.

Those are all in non-free (AFAIK).  So not really in Debian.

-- 
Please *no* private copies of mailing list or newsgroup messages.
Rule 420: All persons more than eight miles high to leave the court.

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Stéphane Glondu]

Le 31/08/2015 18:14, Francois Berenger a écrit :
>>> Depending on external libraries used to be a problem when there was no
>>> consensus on OCaml packaging (no longer than a few years ago people
>>> where reluctant to even use ocamlfind). We now have a reasonable
>>> consensus on a packaging story, and installing third-party libraries
>>> has never been as easy as it is today -- except on Windows, meh.
>>> I think you should embrace the idea of depending on software outside
>>> the OCaml distribution.
>>
>> Depending on a multitude of external libraries makes packaging more
>> difficult. I am talking about "system" packaging (deb, rpm...) here, not
>> OPAM. And "install OPAM first" is not a very satisfactory way of
>> installing some software written in OCaml in a stable environment.
> 
> Why not?

Packages in OPAM do not benefit from QA done in e.g. Debian or Fedora.

Besides, a regular Debian user (for example) is used to Debian policies,
not OPAM ones. Don't assume that anyone wanting to use software written
in OCaml is part of the OCaml community. I am happy as a Debian user to
not have to deal with CPAN / CTAN / PyPI / whatever.

> In my experience, opam installs are quite reproducible.

In my experience, they are not...

> You can even specify the version number you want to install.

... but I've never tried that. Aren't old versions of packages removed
at some point?

> [...] opam install things in a user's home
> directory, which is not what you want for a system-wise installation of
> a software/library.

AFAICT, opam only "trusts" packages to install things in a user's home.


Cheers,

-- 
Stéphane

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Anil Madhavapeddy]

On 31 Aug 2015, at 18:26, Stéphane Glondu <steph@glondu.net> wrote:
> 
> Le 31/08/2015 18:14, Francois Berenger a écrit :
>>>> Depending on external libraries used to be a problem when there was no
>>>> consensus on OCaml packaging (no longer than a few years ago people
>>>> where reluctant to even use ocamlfind). We now have a reasonable
>>>> consensus on a packaging story, and installing third-party libraries
>>>> has never been as easy as it is today -- except on Windows, meh.
>>>> I think you should embrace the idea of depending on software outside
>>>> the OCaml distribution.
>>> 
>>> Depending on a multitude of external libraries makes packaging more
>>> difficult. I am talking about "system" packaging (deb, rpm...) here, not
>>> OPAM. And "install OPAM first" is not a very satisfactory way of
>>> installing some software written in OCaml in a stable environment.
>> 
>> Why not?
> 
> Packages in OPAM do not benefit from QA done in e.g. Debian or Fedora.
> 
> Besides, a regular Debian user (for example) is used to Debian policies,
> not OPAM ones. Don't assume that anyone wanting to use software written
> in OCaml is part of the OCaml community. I am happy as a Debian user to
> not have to deal with CPAN / CTAN / PyPI / whatever.

The opposite also holds true -- OPAM remains relatively consistent across
operating system distributions, in an age when students are bringing their
own laptops running various strands of Linux, *BSD and MacOS X.  Only
Windows languishes behind at the moment.  Our experience in Cambridge is
that our sys-admins have had no problem installing OPAM on our centrally
managed (Ubuntu LTS) lab machines, while students followed the online
instructions to get it running on their own flavours.  Windows students
run a Vagrant or Virtualbox VM, or can soon use Boot2Docker.

My view on packaging is that it ultimately needs to be automated from
a central metadata source held in the package, with only OS-specific
concerns being put into the upstream port.  Right now, individual
distributions go to great pains to separately calculate packing lists
for (e.g.) native vs bytecode, where this should all really be information
provided by a standardised build system.  That's the intention of various
efforts such as OASIS, oasis2opam, and the early efforts to get opam2deb
and opam2rpm working: https://github.com/ocaml/opam/wiki/opam2%7Brpm,deb%7D
The OPAM `depext` mechanism also tracks external OS packaging metadata
in order to do a better job of continuous integration.

Don't get me wrong -- I have a huge respect for the efforts poured into
Debian and Fedora packaging of individual libraries, and I also spend
significant time myself maintaining OpenBSD ports of OCaml libraries.
But I also appreciate being able to get the latest libraries and
manipulate them at a source-code level much more easily via OPAM.
These are all complementary efforts :-)

>> In my experience, opam installs are quite reproducible.
> 
> In my experience, they are not...

Any specific concerns here?  We run quite a bit of automated testing
on the OPAM repository, and the only real source of non-reproducibility
in a well-controlled base environment (as found when running in a Linux
container or BSD VM) is if the external aspcud package solver is changed.

> 
>> You can even specify the version number you want to install.
> 
> ... but I've never tried that. Aren't old versions of packages removed
> at some point?

No, old versions are never deleted from the OPAM repository and remain
available to install.

> 
>> [...] opam install things in a user's home
>> directory, which is not what you want for a system-wise installation of
>> a software/library.
> 
> AFAICT, opam only "trusts" packages to install things in a user's home.

The build is also never run as root, only as the user uid that invoked
OPAM.  The `OPAMROOT` variable can be set to a system-wide area if
desired.  See `opam --help` for a full list.

regards,
Anil

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Oliver Bandel]

Zitat von Stéphane Glondu <steph@glondu.net> (Mon, 31 Aug 2015 18:06:22 +0200)

> Le 27/08/2015 10:17, Gabriel Scherer a écrit :
>> Depending on external libraries used to be a problem when there was no
>> consensus on OCaml packaging (no longer than a few years ago people
>> where reluctant to even use ocamlfind). We now have a reasonable
>> consensus on a packaging story, and installing third-party libraries
>> has never been as easy as it is today -- except on Windows, meh.
>> I think you should embrace the idea of depending on software outside
>> the OCaml distribution.
>
> Depending on a multitude of external libraries makes packaging more
> difficult. I am talking about "system" packaging (deb, rpm...) here, not
> OPAM. And "install OPAM first" is not a very satisfactory way of
> installing some software written in OCaml in a stable environment.
[...]

I imagine some tools here:


opam2deb
opam2rpm
opam2pkgbuild

deb2opam
rpm2opam
pkgbuild2opam



Ciao,
    Oliver

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Gabriel Scherer]

On Mon, Aug 31, 2015 at 9:06 AM, Stéphane Glondu <steph@glondu.net> wrote:
> And "install OPAM first" is not a very satisfactory way of
> installing some software written in OCaml in a stable environment.

I wholly agree. My personal practice is to use OPAM to set up
*development* environment(s) for OCaml hacking, and rely on the system
package manager to install any end-user software (including those
written in OCaml). For example, I do not recommend installing Coq
through OPAM (unless one need experimental versions or plans to
contribute to Coq's development), but rather through the package
manager.

It remains, though, that we cannot assume that everyone is using a
distribution with a good OCaml packaging story (I know only of Fedora
and Debian although I haven't tried OSX distributions or the BSD ports
-- on other systems I've been a satisfied user of GODI for a while),
so for a long time the difficulty of *some* users and developers to
install OCaml (development) libraries effectively discouraged people
to rely on third-party libraries, and I think it is not the case
anymore.

I believe the Debian QA process is very useful and important. For
example, the handling of software license is still in its infancy in
OPAM, while there is a well-established checking process which is part
of Debian integration; I mentioned before that legal issues
discouraged some industrial users from using third-party libraries,
but I think reasonable users should feel rather safe using
Debian-packaged software.

The OCaml community also benefits from the fact that many consider
"supports all the official Debian packages" as a milestone (when
evaluating the readiness of a new hardware architecture for example),
and will thus foster development across all of the Debian user-space,
including packaged OCaml projects. More generally, we benefit from a
lot of patches and platform support developed by packagers for various
operating system (BSD port comes to mind), and this is part of the
continuing added value of packaging OCaml software for many systems.

On Mon, Aug 31, 2015 at 9:06 AM, Stéphane Glondu <steph@glondu.net> wrote:
> Le 27/08/2015 10:17, Gabriel Scherer a écrit :
>> Depending on external libraries used to be a problem when there was no
>> consensus on OCaml packaging (no longer than a few years ago people
>> where reluctant to even use ocamlfind). We now have a reasonable
>> consensus on a packaging story, and installing third-party libraries
>> has never been as easy as it is today -- except on Windows, meh.
>> I think you should embrace the idea of depending on software outside
>> the OCaml distribution.
>
> Depending on a multitude of external libraries makes packaging more
> difficult. I am talking about "system" packaging (deb, rpm...) here, not
> OPAM. And "install OPAM first" is not a very satisfactory way of
> installing some software written in OCaml in a stable environment.
>
>
> Cheers,
>
> --
> Stéphane

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Sébastien Hinderer]

Hi,

Thanks for having started this thread.

I am wondering to which extent the standard library of a language such
as Haskell could be a source of inspiration for OCaml?

Cheers,

Sébastien.

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Daniil Baturin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

It often provokes exact same feelings as OCaml standard library,
especially the prelude.  There are also reimplementations of it, e.g.
https://hackage.haskell.org/package/classy-prelude

GHC authors also keep the old stuff for backwards compatibility.

What I think can serve as a source of inspiration is the Haskell
Platform or the Python distribution. They are distributed with a set of
more or less self-contained modules that once were or still are
third-party libraries. This is especially convenient on platforms not
friendly to automatic package management.

On 08/27/2015 02:07 PM, Sébastien Hinderer wrote:
> Hi,
>
> Thanks for having started this thread.
>
> I am wondering to which extent the standard library of a language such
> as Haskell could be a source of inspiration for OCaml?
>
> Cheers,
>
> Sébastien.
>


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJV3shSAAoJEEcm35UR4K8fSmwP+wURGWzi/onbTmr0dltA8aRK
XB6FKEyiwLarEMHKnLPiT3uZhDVbaSvDolPGzPuU3eq0/kih0z/0R4l2WAUG8QuP
cbdcVAd0Po+fSfSrgNtl+luH9fpUiE0EtDWcZxviZSerZoObcPcIK68FkgQbBs5T
vlJ7MjuNguYI26tDrBsSAzDZ+FqFGYj9/ykmUNF42aPUqD7GQXFKY6iBgFoTCpze
v9zbPnDoPjShe1Guk72GbmHU+b5xwEXe0bFpWj3O6pM5sXOnQ9jbLNRM70wSvGkX
V17NexWd4HyqVhoSRHy+CnrFUHuzEhBJQARhq1nYoypY6DiYyOSg58uw5EWJC/vH
YXl0LP60sXrhY4ZJ4YUvv86DFpqGGtNftRThNRTq0o7LULXIt3JRyVp9f+fVhxB4
FW8dk2xzbyMijKCLMdbrbLJhbQ7znTfCt8PLnyemVFdc+Ficne/j+D0YOoWCyX+b
vyc7xAQn14ZvywMxS/Ii9SRKZO8jqmjlhnoTa+AiF9Ve9KYbIJ3d9hX/9cw+blmo
3EuUbo0K3DbIWUvNYMFsR+03mY5WyfO9UX5zcI/j5hP+DqxqrHEvNeHo/7/8IYiZ
nI2B0cGwk5JX58+htaaR1rs3FDT7YI3mdoDzmLjVtdnqlRLhUvmHHqraZJQt/QhR
DOx3brrv1loX0DPvr0SM
=Fpx0
-----END PGP SIGNATURE-----

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Edouard Evangelisti]

[-- Attachment #1: Type: text/plain, Size: 3927 bytes --]

Dear all,

Thanks for this inspiring discussion.

The integration of large pieces of code into OCaml standard library is
maybe not the best solution as it will never fulfill everyone's needs. I
like the idea of having a Swiss army knife standard library rather than a
heavy weight set of tools. However, filling existing minor gaps in the
standard library would probably improve programmers day-to-day experience.
Below are some examples:

- Integers, floats and options, for instance, would deserve their own
module for functor application.

- More functions on option type, such as eval, iter, map. Something like
this:
let eval f x = try Some (f x) with _ -> None
let iter f = function None -> () | Some x -> f x
let map f = function None -> None | Some x -> eval f x

- The standard library includes incr and decr on integer references, but
none of the useful C-like operators with assignment such as += that save a
lot of time in situations (there are some) where references are heavily
used.
let ( += ) r n = r := !r + n

- An efficient set of string searching functions would help a lot (no
regexp). Something like Boyer-Moore, KMP or even Rabin-Karp algorithms.
- String.explode / implode to convert to/from char list.
- Trivial functions such as read_file or read_text_file.
- The identity function!
- All fold functions should have a foldi counterpart, and the same set of
iterators should occur in all modules.

Then, for more specialized requirements, I think it may be better to
contribute directly to improve third party libraries.

Kind regards,
Edouard


2015-08-27 9:20 GMT+01:00 Daniil Baturin <daniil@baturin.org>:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> It often provokes exact same feelings as OCaml standard library,
> especially the prelude.  There are also reimplementations of it, e.g.
> https://hackage.haskell.org/package/classy-prelude
>
> GHC authors also keep the old stuff for backwards compatibility.
>
> What I think can serve as a source of inspiration is the Haskell
> Platform or the Python distribution. They are distributed with a set of
> more or less self-contained modules that once were or still are
> third-party libraries. This is especially convenient on platforms not
> friendly to automatic package management.
>
> On 08/27/2015 02:07 PM, Sébastien Hinderer wrote:
> > Hi,
> >
> > Thanks for having started this thread.
> >
> > I am wondering to which extent the standard library of a language such
> > as Haskell could be a source of inspiration for OCaml?
> >
> > Cheers,
> >
> > Sébastien.
> >
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBCAAGBQJV3shSAAoJEEcm35UR4K8fSmwP+wURGWzi/onbTmr0dltA8aRK
> XB6FKEyiwLarEMHKnLPiT3uZhDVbaSvDolPGzPuU3eq0/kih0z/0R4l2WAUG8QuP
> cbdcVAd0Po+fSfSrgNtl+luH9fpUiE0EtDWcZxviZSerZoObcPcIK68FkgQbBs5T
> vlJ7MjuNguYI26tDrBsSAzDZ+FqFGYj9/ykmUNF42aPUqD7GQXFKY6iBgFoTCpze
> v9zbPnDoPjShe1Guk72GbmHU+b5xwEXe0bFpWj3O6pM5sXOnQ9jbLNRM70wSvGkX
> V17NexWd4HyqVhoSRHy+CnrFUHuzEhBJQARhq1nYoypY6DiYyOSg58uw5EWJC/vH
> YXl0LP60sXrhY4ZJ4YUvv86DFpqGGtNftRThNRTq0o7LULXIt3JRyVp9f+fVhxB4
> FW8dk2xzbyMijKCLMdbrbLJhbQ7znTfCt8PLnyemVFdc+Ficne/j+D0YOoWCyX+b
> vyc7xAQn14ZvywMxS/Ii9SRKZO8jqmjlhnoTa+AiF9Ve9KYbIJ3d9hX/9cw+blmo
> 3EuUbo0K3DbIWUvNYMFsR+03mY5WyfO9UX5zcI/j5hP+DqxqrHEvNeHo/7/8IYiZ
> nI2B0cGwk5JX58+htaaR1rs3FDT7YI3mdoDzmLjVtdnqlRLhUvmHHqraZJQt/QhR
> DOx3brrv1loX0DPvr0SM
> =Fpx0
> -----END PGP SIGNATURE-----
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>



-- 
Edouard Evangelisti
Post doctoral Research Associate
Sainsbury Laboratory, Cambridge University (SLCU)
Bateman Street
Cambridge CB2 1LR (United Kingdom)

[-- Attachment #2: Type: text/html, Size: 6658 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[r.3]

[-- Attachment #1: Type: text/plain, Size: 4342 bytes --]


+1 for 80% of it 

and I would add : 
- String.split : char list -> string -> string list 
- Hashtbl.keys 

I also think it is very valuable that inria maintains the stdlib. I prefer to see it kept maintained as inria standard distribution packaged with ocaml, even if that would imply less features on the compiler. 



On 27/08/2015 11:34, Edouard Evangelisti wrote: 




Dear all, 

Thanks for this inspiring discussion. 

The integration of large pieces of code into OCaml standard library is maybe not the best solution as it will never fulfill everyone's needs. I like the idea of having a Swiss army knife standard library rather than a heavy weight set of tools. However, filling existing minor gaps in the standard library would probably improve programmers day-to-day experience. Below are some examples: 

- Integers, floats and options, for instance, would deserve their own module for functor application. 

- More functions on option type, such as eval, iter, map. Something like this: 
let eval f x = try Some (f x) with _ -> None 
let iter f = function None -> () | Some x -> f x 
let map f = function None -> None | Some x -> eval f x 

- The standard library includes incr and decr on integer references, but none of the useful C-like operators with assignment such as += that save a lot of time in situations (there are some) where references are heavily used. 
let ( += ) r n = r := !r + n 

- An efficient set of string searching functions would help a lot (no regexp). Something like Boyer-Moore, KMP or even Rabin-Karp algorithms. 
- String.explode / implode to convert to/from char list. 
- Trivial functions such as read_file or read_text_file . 
- The identity function! 

- All fold functions should have a foldi counterpart, and the same set of iterators should occur in all modules. 



Then, for more specialized requirements, I think it may be better to contribute directly to improve third party libraries. 


Kind regards, 
Edouard 




2015-08-27 9:20 GMT+01:00 Daniil Baturin < daniil@baturin.org > : 

<blockquote>

-----BEGIN PGP SIGNED MESSAGE----- 
Hash: SHA256 

It often provokes exact same feelings as OCaml standard library, 
especially the prelude. There are also reimplementations of it, e.g. 
https://hackage.haskell.org/package/classy-prelude 

GHC authors also keep the old stuff for backwards compatibility. 

What I think can serve as a source of inspiration is the Haskell 
Platform or the Python distribution. They are distributed with a set of 
more or less self-contained modules that once were or still are 
third-party libraries. This is especially convenient on platforms not 
friendly to automatic package management. 

On 08/27/2015 02:07 PM, Sébastien Hinderer wrote: 
> Hi, 
> 
> Thanks for having started this thread. 
> 
> I am wondering to which extent the standard library of a language such 
> as Haskell could be a source of inspiration for OCaml? 
> 
> Cheers, 
> 
> Sébastien. 
> 


-----BEGIN PGP SIGNATURE----- 
Version: GnuPG v2 

iQIcBAEBCAAGBQJV3shSAAoJEEcm35UR4K8fSmwP+wURGWzi/onbTmr0dltA8aRK 
XB6FKEyiwLarEMHKnLPiT3uZhDVbaSvDolPGzPuU3eq0/kih0z/0R4l2WAUG8QuP 
cbdcVAd0Po+fSfSrgNtl+luH9fpUiE0EtDWcZxviZSerZoObcPcIK68FkgQbBs5T 
vlJ7MjuNguYI26tDrBsSAzDZ+FqFGYj9/ykmUNF42aPUqD7GQXFKY6iBgFoTCpze 
v9zbPnDoPjShe1Guk72GbmHU+b5xwEXe0bFpWj3O6pM5sXOnQ9jbLNRM70wSvGkX 
V17NexWd4HyqVhoSRHy+CnrFUHuzEhBJQARhq1nYoypY6DiYyOSg58uw5EWJC/vH 
YXl0LP60sXrhY4ZJ4YUvv86DFpqGGtNftRThNRTq0o7LULXIt3JRyVp9f+fVhxB4 
FW8dk2xzbyMijKCLMdbrbLJhbQ7znTfCt8PLnyemVFdc+Ficne/j+D0YOoWCyX+b 
vyc7xAQn14ZvywMxS/Ii9SRKZO8jqmjlhnoTa+AiF9Ve9KYbIJ3d9hX/9cw+blmo 
3EuUbo0K3DbIWUvNYMFsR+03mY5WyfO9UX5zcI/j5hP+DqxqrHEvNeHo/7/8IYiZ 
nI2B0cGwk5JX58+htaaR1rs3FDT7YI3mdoDzmLjVtdnqlRLhUvmHHqraZJQt/QhR 
DOx3brrv1loX0DPvr0SM 
=Fpx0 
-----END PGP SIGNATURE----- 





-- 
Caml-list mailing list. Subscription management and archives: 
https://sympa.inria.fr/sympa/arc/caml-list 
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners 
Bug reports: http://caml.inria.fr/bin/caml-bugs 




-- 









Edouard Evangelisti 
Post doctoral Research Associate 
Sainsbury Laboratory, Cambridge University (SLCU) 
Bateman Street 
Cambridge CB2 1LR (United Kingdom) 

</blockquote>


[-- Attachment #2: Type: text/html, Size: 9247 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Francois Berenger]

On 08/27/2015 04:52 AM, Hongbo Zhang wrote:
> Dear OCaml developers,
>      I would like to spend one hour in writing down my experience that
> why I had to write some small utilities again and again, since this
> happened so many times that I believe you might come across such issues
> before.
>      I am doing some compiler hacking tonight, I needed a utility
> function “String.split” which split a string into a list of strings by
> whitespace, it is just one liner if you use str library. However, since
> I am doing some low level stuff, I would try to avoid such big
> dependency, and I am pretty sure that I have ever written it  for at
> least three times, I just hoped that I could get it done quickly, so I
> am looking into batteries that if I can steal some code, I have to copy
> some code instead of depend on batteries, batteries is too big for my
> projects. `BatString.nsplit` seems to fit for what I needed, I copied
> the definition of `BatString.nsplit` into REPL, no luck, it depends on
> some previously defined functions, then I copied the whole module
> `BatString` into REPL, still no luck, it depended on another module
> `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
> thrown-away `String.split` function again.
>     OCaml is my favorite programming language, and I am very productive
> at it, however, I was annoyed by such things from time to time. We do
> have four *standard libraries* alternatives: batteries, core, extlib and
> ocaml-containers. In my opinion, none of them matches the beauty of the
> OCaml language itself and probably will never catch up if we don’t do
> anything.
>      Note that I don’t want to be offensive to any of these libraries,
> just my personal feedback that why I think it is not a good standard
> library, */I appreciated a lot to people who contribute their precious
> time in maintaining these libraries, better than nothing/* : )
>      - Batteries(extlib)
>        It’s big with dependencies between different modules

For batteries, there is a very interesting proposal from Simon Cruanes 
(a batteries contributor and also the author of ocaml-containers)
about cutting batteries into small pieces.

https://github.com/ocaml-batteries-team/batteries-included/pull/554

 > (OCaml does
> not have a good story in dead code elimination), some of its modules are
> of low quality, for example, batEnum is used everywhere while its
> implementation is buggy.

If we cut into small pieces, maybe we should adopt list and arrays as 
the "glue" to interchange data between modules.
So you can really depend on a single module, if you want to.

 > batIO makes things even worse since it is not
> compatible with standard library, some type signatures mismatched IIRC.

batIO even introduce a performance regression bug compared to the stdlib.
But, it is also easy to avoid; you can always tap into Legacy.*
to access the stdlib things and avoid the batIO layer (when you have
a 'open Batteries' at the top of your file).

>      - ocaml-containers
>        Mostly one man’s project
>      - core
>        I believe core has high quality, however, it suffers the same
> problem as batteries, a big dependency. There is a blocking issue, its
> development process is opaque, for an open source community, I would
> prefer a standard library developed in an open environment.
>      I am not expecting that we could have a  standard library as rich
> as python, but for some utilities, I do believe that shipped with
> standard library or officially supported is the best solution.
>     Thanks — Hongbo

-- 
Regards,
Francois.
"When in doubt, use more types"

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Drup]

> If we cut into small pieces, maybe we should adopt list and arrays as 
> the "glue" to interchange data between modules.
> So you can really depend on a single module, if you want to.
I would rather have sequence/gen become that. It's much more efficient 
and you don't even need to actually depend on the libraries to 
manipulate them.

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Yaron Minsky]

The core OCaml team is small, and having them focused on building a
great compiler rather than on building a rich stdlib seems right to
me.  The improvements in packaging I think make the question of
whether it's distributed with the compiler mostly a moot issue, I
think.

On the topic of Core: The issue of binary size is real.  It will get
somewhat better when we drop packed modules from the public release,
which should come in the next few months (support for it internally is
mostly in place.)  That said the module-level dependency tracking is
by its nature coarse, so binaries using Core (or the more portable
Core_kernel) are still going to be relatively large.

That said, I think I'd rather have the compiler folk work on improving
dead-code elimination than integrating and maintaining a standard
library.

As to openness: we publish all changes on github, have a mailing list,
and will accept pull requests; but it's true that the development of
Core is focused within Jane Street, and that is unlikely to change.

y

On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com> wrote:
> Dear OCaml developers,
>     I would like to spend one hour in writing down my experience that why I
> had to write some small utilities again and again, since this happened so
> many times that I believe you might come across such issues before.
>     I am doing some compiler hacking tonight, I needed a utility function
> “String.split” which split a string into a list of strings by whitespace, it
> is just one liner if you use str library. However, since I am doing some low
> level stuff, I would try to avoid such big dependency, and I am pretty sure
> that I have ever written it  for at least three times, I just hoped that I
> could get it done quickly, so I am looking into batteries that if I can
> steal some code, I have to copy some code instead of depend on batteries,
> batteries is too big for my projects. `BatString.nsplit` seems to fit for
> what I needed, I copied the definition of `BatString.nsplit` into REPL, no
> luck, it depends on some previously defined functions, then I copied the
> whole module `BatString` into REPL, still no luck, it depended on another
> module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
> thrown-away `String.split` function again.
>    OCaml is my favorite programming language, and I am very productive at
> it, however, I was annoyed by such things from time to time. We do have four
> *standard libraries* alternatives: batteries, core, extlib and
> ocaml-containers. In my opinion, none of them matches the beauty of the
> OCaml language itself and probably will never catch up if we don’t do
> anything.
>     Note that I don’t want to be offensive to any of these libraries, just
> my personal feedback that why I think it is not a good standard library, I
> appreciated a lot to people who contribute their precious time in
> maintaining these libraries, better than nothing : )
>     - Batteries(extlib)
>       It’s big with dependencies between different modules (OCaml does not
> have a good story in dead code elimination), some of its modules are of low
> quality, for example, batEnum is used everywhere while its implementation is
> buggy. batIO makes things even worse since it is not compatible with
> standard library, some type signatures mismatched IIRC.
>     - ocaml-containers
>       Mostly one man’s project
>     - core
>       I believe core has high quality, however, it suffers the same problem
> as batteries, a big dependency. There is a blocking issue, its development
> process is opaque, for an open source community, I would prefer a standard
> library developed in an open environment.
>     I am not expecting that we could have a  standard library as rich as
> python, but for some utilities, I do believe that shipped with standard
> library or officially supported is the best solution.
>    Thanks — Hongbo

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Jesse Haber-Kucharsky]

[-- Attachment #1: Type: text/plain, Size: 7247 bytes --]

To offer the perspective of a relative outsider who has meekly decided to
pipe in:

A standard library with a variety of general-purpose "building block" of
functionality is invaluable. I feel it's missing in OCaml.

Here is some important functionality that I'd like to see based on my own
experience writing a small-to-moderately sized OCaml program.

- Modules for standard types like `Int` and `Float` as previously mentioned
- More standard definitions like `compose`, `const`, `identity`, etc (also
as previously mentioned)
- Comprehensive string and regular expression handling (UTF support can be
relegated to an endorsed third party package)
- More helper functions in the `List` and `Option` modules
- A standard general purpose lazy list (stream). I was able to implement
this comprehensively in about 300 lines, and its enormously useful. It's
absence means that everyone will write their own, or that there will be
hundreds in opam
- More immutable data structures like queues, heaps, stacks, etc
- A standard means of concurrency. Lwt is great, but is almost its own
standard library at this point, and there's still a split between it and
Async.
- Better support for error-handling-by-value with `either` and `option`
through helper functions
- The ppx_deriving package reduces a TONNE of boilerplate for making
defining ordering on data for instance. It should be a standard extension
(or something like it)
- Better documentation/endorsement of build tools. It's possible to get
pretty far with ocamlbuild, _tags, and a simple Makefile, but figuring out
how to get there was not exactly easy
- Better interfaces to the file system with more structured error handling
on failure (`Sys_error` was not the nicest thing to work with).
- Less reliance on exceptions for non-exceptional situations and
retrofitting "pure" functions like `hd` with `option` or `either` result
types.
- Less duplication and or more aggressive deprecation. It's not clear to me
why there should be both "Foo" and "FooLabels" modules, for instance. I
also hear that some modules in the standard library are probably best
avoided.

Finally, about the governance of such a library:

While libraries like Core and and Batteries are undeniably useful, I found
myself somewhat discouraged in practice:

- Batteries has relatively low adoption and I think it does too much
- Core is driven by a private organization. In practise, I've found the
(lack of) documentation to be a major blocker, and I'd feel better about
using it if it was truly community "owned" (or if there was a non-profit
spin-off to support and own it like the F# foundation, for instance).

Best,
--
Jesse


On Thu, Aug 27, 2015 at 10:17 AM, Yaron Minsky <yminsky@janestreet.com>
wrote:

> The core OCaml team is small, and having them focused on building a
> great compiler rather than on building a rich stdlib seems right to
> me.  The improvements in packaging I think make the question of
> whether it's distributed with the compiler mostly a moot issue, I
> think.
>
> On the topic of Core: The issue of binary size is real.  It will get
> somewhat better when we drop packed modules from the public release,
> which should come in the next few months (support for it internally is
> mostly in place.)  That said the module-level dependency tracking is
> by its nature coarse, so binaries using Core (or the more portable
> Core_kernel) are still going to be relatively large.
>
> That said, I think I'd rather have the compiler folk work on improving
> dead-code elimination than integrating and maintaining a standard
> library.
>
> As to openness: we publish all changes on github, have a mailing list,
> and will accept pull requests; but it's true that the development of
> Core is focused within Jane Street, and that is unlikely to change.
>
> y
>
> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com>
> wrote:
> > Dear OCaml developers,
> >     I would like to spend one hour in writing down my experience that
> why I
> > had to write some small utilities again and again, since this happened so
> > many times that I believe you might come across such issues before.
> >     I am doing some compiler hacking tonight, I needed a utility function
> > “String.split” which split a string into a list of strings by
> whitespace, it
> > is just one liner if you use str library. However, since I am doing some
> low
> > level stuff, I would try to avoid such big dependency, and I am pretty
> sure
> > that I have ever written it  for at least three times, I just hoped that
> I
> > could get it done quickly, so I am looking into batteries that if I can
> > steal some code, I have to copy some code instead of depend on batteries,
> > batteries is too big for my projects. `BatString.nsplit` seems to fit for
> > what I needed, I copied the definition of `BatString.nsplit` into REPL,
> no
> > luck, it depends on some previously defined functions, then I copied the
> > whole module `BatString` into REPL, still no luck, it depended on another
> > module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
> > thrown-away `String.split` function again.
> >    OCaml is my favorite programming language, and I am very productive at
> > it, however, I was annoyed by such things from time to time. We do have
> four
> > *standard libraries* alternatives: batteries, core, extlib and
> > ocaml-containers. In my opinion, none of them matches the beauty of the
> > OCaml language itself and probably will never catch up if we don’t do
> > anything.
> >     Note that I don’t want to be offensive to any of these libraries,
> just
> > my personal feedback that why I think it is not a good standard library,
> I
> > appreciated a lot to people who contribute their precious time in
> > maintaining these libraries, better than nothing : )
> >     - Batteries(extlib)
> >       It’s big with dependencies between different modules (OCaml does
> not
> > have a good story in dead code elimination), some of its modules are of
> low
> > quality, for example, batEnum is used everywhere while its
> implementation is
> > buggy. batIO makes things even worse since it is not compatible with
> > standard library, some type signatures mismatched IIRC.
> >     - ocaml-containers
> >       Mostly one man’s project
> >     - core
> >       I believe core has high quality, however, it suffers the same
> problem
> > as batteries, a big dependency. There is a blocking issue, its
> development
> > process is opaque, for an open source community, I would prefer a
> standard
> > library developed in an open environment.
> >     I am not expecting that we could have a  standard library as rich as
> > python, but for some utilities, I do believe that shipped with standard
> > library or officially supported is the best solution.
> >    Thanks — Hongbo
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 8574 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Hongbo Zhang]

[-- Attachment #1: Type: text/plain, Size: 9267 bytes --]

Hi all,
    I did not realize that my email got so many responses, but it does mean that this is a major concern among the community.
    I agree with Yaron that having the core team building a great compiler rather than on building a rich stdlib seems more useful, indeed, my current interest is also focused on the compiler itself.
   But there are some really talented programmers who are also interested in improving the standard library, like Daniel Bunzli (CCed). Building the standard library is technically not more challenging than improving the compiler itself, but it is as important if not even more for a health community. What I proposed is that we need officially support one standard library instead of keep reinventing the wheels or just stand by. This kind of official promotion does play a very important role. For example, ocamlbuild is no better than omake or obuild or maybe ocp-build, but it is mostly widely used build system among the community, the major reason, I believe is that it’s shipped with the core distribution. So, if we promote a standard library, by default that library will be in the included path, like what we currently did for stdlib.   
    Thanks to the attributes, we can introduce [@@ocaml.experimental] for some experimental APIs, another development pattern I found particular useful is that we can keep the standard library in a different repo and merge it back when we make a release for the new compiler, of course, any changes to the standard library should be well understood by the core ocaml developers. Since standard library is expected to be backward compatible in most cases, so I would not expect that this could cause any bootstrapping issues.
     Thanks — Hongbo
> On Aug 27, 2015, at 12:00 PM, Jesse Haber-Kucharsky <jesse@haberkucharsky.com> wrote:
> 
> To offer the perspective of a relative outsider who has meekly decided to pipe in:
> 
> A standard library with a variety of general-purpose "building block" of functionality is invaluable. I feel it's missing in OCaml.
> 
> Here is some important functionality that I'd like to see based on my own experience writing a small-to-moderately sized OCaml program.
> 
> - Modules for standard types like `Int` and `Float` as previously mentioned
> - More standard definitions like `compose`, `const`, `identity`, etc (also as previously mentioned)
> - Comprehensive string and regular expression handling (UTF support can be relegated to an endorsed third party package)
> - More helper functions in the `List` and `Option` modules
> - A standard general purpose lazy list (stream). I was able to implement this comprehensively in about 300 lines, and its enormously useful. It's absence means that everyone will write their own, or that there will be hundreds in opam
> - More immutable data structures like queues, heaps, stacks, etc
> - A standard means of concurrency. Lwt is great, but is almost its own standard library at this point, and there's still a split between it and Async.
> - Better support for error-handling-by-value with `either` and `option` through helper functions
> - The ppx_deriving package reduces a TONNE of boilerplate for making defining ordering on data for instance. It should be a standard extension (or something like it)
> - Better documentation/endorsement of build tools. It's possible to get pretty far with ocamlbuild, _tags, and a simple Makefile, but figuring out how to get there was not exactly easy
> - Better interfaces to the file system with more structured error handling on failure (`Sys_error` was not the nicest thing to work with).
> - Less reliance on exceptions for non-exceptional situations and retrofitting "pure" functions like `hd` with `option` or `either` result types.
> - Less duplication and or more aggressive deprecation. It's not clear to me why there should be both "Foo" and "FooLabels" modules, for instance. I also hear that some modules in the standard library are probably best avoided.
> 
> Finally, about the governance of such a library:
> 
> While libraries like Core and and Batteries are undeniably useful, I found myself somewhat discouraged in practice:
> 
> - Batteries has relatively low adoption and I think it does too much
> - Core is driven by a private organization. In practise, I've found the (lack of) documentation to be a major blocker, and I'd feel better about using it if it was truly community "owned" (or if there was a non-profit spin-off to support and own it like the F# foundation, for instance).
> 
> Best,
> --
> Jesse
> 
> 
> On Thu, Aug 27, 2015 at 10:17 AM, Yaron Minsky <yminsky@janestreet.com <mailto:yminsky@janestreet.com>> wrote:
> The core OCaml team is small, and having them focused on building a
> great compiler rather than on building a rich stdlib seems right to
> me.  The improvements in packaging I think make the question of
> whether it's distributed with the compiler mostly a moot issue, I
> think.
> 
> On the topic of Core: The issue of binary size is real.  It will get
> somewhat better when we drop packed modules from the public release,
> which should come in the next few months (support for it internally is
> mostly in place.)  That said the module-level dependency tracking is
> by its nature coarse, so binaries using Core (or the more portable
> Core_kernel) are still going to be relatively large.
> 
> That said, I think I'd rather have the compiler folk work on improving
> dead-code elimination than integrating and maintaining a standard
> library.
> 
> As to openness: we publish all changes on github, have a mailing list,
> and will accept pull requests; but it's true that the development of
> Core is focused within Jane Street, and that is unlikely to change.
> 
> y
> 
> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com <mailto:bobzhang1988@gmail.com>> wrote:
> > Dear OCaml developers,
> >     I would like to spend one hour in writing down my experience that why I
> > had to write some small utilities again and again, since this happened so
> > many times that I believe you might come across such issues before.
> >     I am doing some compiler hacking tonight, I needed a utility function
> > “String.split” which split a string into a list of strings by whitespace, it
> > is just one liner if you use str library. However, since I am doing some low
> > level stuff, I would try to avoid such big dependency, and I am pretty sure
> > that I have ever written it  for at least three times, I just hoped that I
> > could get it done quickly, so I am looking into batteries that if I can
> > steal some code, I have to copy some code instead of depend on batteries,
> > batteries is too big for my projects. `BatString.nsplit` seems to fit for
> > what I needed, I copied the definition of `BatString.nsplit` into REPL, no
> > luck, it depends on some previously defined functions, then I copied the
> > whole module `BatString` into REPL, still no luck, it depended on another
> > module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
> > thrown-away `String.split` function again.
> >    OCaml is my favorite programming language, and I am very productive at
> > it, however, I was annoyed by such things from time to time. We do have four
> > *standard libraries* alternatives: batteries, core, extlib and
> > ocaml-containers. In my opinion, none of them matches the beauty of the
> > OCaml language itself and probably will never catch up if we don’t do
> > anything.
> >     Note that I don’t want to be offensive to any of these libraries, just
> > my personal feedback that why I think it is not a good standard library, I
> > appreciated a lot to people who contribute their precious time in
> > maintaining these libraries, better than nothing : )
> >     - Batteries(extlib)
> >       It’s big with dependencies between different modules (OCaml does not
> > have a good story in dead code elimination), some of its modules are of low
> > quality, for example, batEnum is used everywhere while its implementation is
> > buggy. batIO makes things even worse since it is not compatible with
> > standard library, some type signatures mismatched IIRC.
> >     - ocaml-containers
> >       Mostly one man’s project
> >     - core
> >       I believe core has high quality, however, it suffers the same problem
> > as batteries, a big dependency. There is a blocking issue, its development
> > process is opaque, for an open source community, I would prefer a standard
> > library developed in an open environment.
> >     I am not expecting that we could have a  standard library as rich as
> > python, but for some utilities, I do believe that shipped with standard
> > library or officially supported is the best solution.
> >    Thanks — Hongbo
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list <https://sympa.inria.fr/sympa/arc/caml-list>
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners <http://groups.yahoo.com/group/ocaml_beginners>
> Bug reports: http://caml.inria.fr/bin/caml-bugs <http://caml.inria.fr/bin/caml-bugs>


[-- Attachment #2: Type: text/html, Size: 12213 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Daniel Bünzli]

Le vendredi, 28 août 2015 à 01:33, Hongbo Zhang a écrit :
> programmers who are also interested in improving the standard library, like Daniel Bunzli (CCed).  

Personally I'm no longer interested in improving the standard library. I take the good parts of it and rewrite the bad parts for myself (and whoever wants to cope with my insane naming schemes). I'm personally not willing to compromise on design with any form of community reviewed process.   

Besides the way "improvements" are being done at the moment is more a curse than a blessing. One or two random functions get added here and there per major version of OCaml without much thinking about the whole design of the given module or the standard library in general which is any way constrained by the strong backward compatibility constraints (not a bad idea in itself, it's just that sometime you need to make transitions). As it stand the current technique only adds the annoyance that you suddenly end up depending on a major version of OCaml just because you used a function that was only recently introduced which makes it a rather painful process when you submit your package to the opam repository.

I do believe in large changes that affect the whole language like the Bytes addition but for the rest the actual title of the thread is no longer relevant to me.  

Best,  

Daniel

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Thomas Braibant]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

>
>
> 2. LICENSES: Every opam package comes with a license which should help
> companies to choose which ones to use.  For the problem Hongbo mentioned,
> maybe one could develop a tool that does the following: given a white-list
> of licenses that the company has agreed are OK (e.g. ISC) and a list of
> opam packages, the tool would warn if any of the (recursive) dependencies
> does not have a “good” license.


Here is an example of a script that provides (almost) such a tool

```
PACKAGES=irmin
for p in $(opam list --recursive --short --sort --required-by $PACKAGES); do
   echo "$p $(opam show $p -f license)"
done
```

The name of the licenses could probably be standardized a bit to make it
easier to come up with white-lists.

[-- Attachment #2: Type: text/html, Size: 1143 bytes --]

[Caml-list] opam license field (was Re: We need a rich standard library distributed with OCaml, really)

[Daniel Bünzli]

Le vendredi, 28 août 2015 à 13:38, Thomas Braibant a écrit :

> The name of the licenses could probably be standardized a bit to make it easier to come up with white-lists.

Can't find the issue anymore but I think that Louis planned for opam lint to report non-conformance to debian's dep5 [1] format. My packages don't conform but I'm gradually moving them to this as I see them — of course having opam lint do the check will only speed up the process.

Best,

Daniel

[1] http://dep.debian.net/deps/dep5/#license-specification

Re: [Caml-list] opam license field (was Re: We need a rich standard library distributed with OCaml, really)

[David Sheets]

On Fri, Aug 28, 2015 at 2:00 PM, Daniel Bünzli
<daniel.buenzli@erratique.ch> wrote:
>
>
> Le vendredi, 28 août 2015 à 13:38, Thomas Braibant a écrit :
>
>> The name of the licenses could probably be standardized a bit to make it easier to come up with white-lists.
>
> Can't find the issue anymore but I think that Louis planned for opam lint to report non-conformance to debian's dep5 [1] format. My packages don't conform but I'm gradually moving them to this as I see them — of course having opam lint do the check will only speed up the process.

Maybe <https://github.com/ocaml/opam/pull/2224> about SPDX adherence?

> Best,
>
> Daniel
>
> [1] http://dep.debian.net/deps/dep5/#license-specification
>
>
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Oliver Bandel]

Zitat von Christophe Troestler <Christophe.Troestler@umons.ac.be>  
(Fri, 28 Aug 2015 14:08:26 +0200)

[...]
> 1. INTEROPERABILITY: While many, possibly overlapping, libraries may
[...]

[...]
> 2. LICENSES: Every opam package comes with a license which should  
> help companies to choose which ones to use.  For the problem Hongbo  
> mentioned, maybe one could develop a tool that does the following:
[...]


Possibly that's not a questio of a tool,
but rather a question of lawyers.

Are there already schemes / tables, that suggest what combinations
of licenses do fit, and which not?
And: the result may also be different, depending on the question
(which aspects of the problem of licenses is looked at).

So, I think a tool would be the step that comes after classification of the
problem.

[...]
> given a white-list of licenses that the company has agreed are OK  
> (e.g. ISC) and a list of opam packages, the tool would warn if any  
> of the (recursive) dependencies does not have a “good” license.
[...]



Also a "which license do I choose", or "what do I have to add"
(e.g. linking exception for OCaml and GPL-like licenses) would pop up.

=> which license can be used / derived for the project that someone  
wants to program,
    from those licenses that are available in the libs (which might  
have different licensing schemes)

So, thats the reverse question of "what libs can I chose to fit in license X".


So possible questions are:

  - what lib(s) can I use, which to fit into "I need license X for my program"
  - what license(s) can I chose for my program, if I use library Y




>
> 3. APPLICATION DOMAINS: A newcomer has to use resources like  
> https://github.com/rizo/awesome-ocaml/blob/master/sotu.md

Ah, didn't knew that.

More of "we have a lot of documents, cluttered through the web".

Also seems like some (a lot of?) entries are missing there...


Ciao,
    Oliver

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Hendrik Boom]

On Fri, Aug 28, 2015 at 04:01:31PM +0200, Oliver Bandel wrote:
> 
> Zitat von Christophe Troestler <Christophe.Troestler@umons.ac.be>
> (Fri, 28 Aug 2015 14:08:26 +0200)
...
> >2. LICENSES: Every opam package comes with a license which should
> >help companies to choose which ones to use.  For the problem
> >Hongbo mentioned, maybe one could develop a tool that does the
> >following:
...
> Possibly that's not a questio of a tool,
> but rather a question of lawyers.

Which licenses are on the approved list? That's a question for 
lawyers.

Whether there any used ppackages have licenses off the approved list?
That's a question for a tool.

-- hendrik

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Alain Frisch]

Hi,

I'd like to add to this discussion that (at least some part of) the core 
team considers that the standard library is an important component which 
deserves some (more) attention.  FWIW, Damien started recently some 
discussion on caml-devel on how to best organize such efforts.

The key property of the standard library is that it is distributed with 
the compiler, not that it must necessarily follow the same development 
model as the rest of the compiler.  Being distributed with the compiler 
has a number of consequences, such as guarantees in terms of stability, 
portability, and of course, ease of access and use (both technically and 
legally).

I don't think that the stdlib will/should be extended with many new 
functional perimeters (e.g. support for file formats, network protocols, 
or bindings to external systems are better left to external libraries). 
  But filling missing holes in existing modules, revisiting some design 
decisions (use of exceptions, labeled/optional arguments, etc), easing 
interactions between external libraries by agreeing on shared 
definitions, and perhaps adding a few widely useful new features are 
certainly desirable (and hopefully achievable) goals.


Alain

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[David MENTRÉ]

Hello,

Le 2015-08-28 16:35, Alain Frisch a écrit :
> I don't think that the stdlib will/should be extended with many new
> functional perimeters (e.g. support for file formats, network protocols,
> or bindings to external systems are better left to external libraries).

One advantage of Python is its wide standard library. So it is at least 
a meaningful question to ask.

That's said, an interesting step might not be to include other libraries 
but the mechanism to install them, i.e. opam.

Best regards,
david

RE: [Caml-list] We need a rich standard library distributed with OCaml, really

[Jon Harrop]

David MENTRÉ:
> That's said, an interesting step might not be to include other libraries but the mechanism to install them, i.e. opam.

I think that is exactly the right thing to do. This is a packaging issue, not a language issue.

Cheers,
Jon.

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Emmanuel Surleau]

[-- Attachment #1: Type: text/plain, Size: 514 bytes --]

On Mon, Aug 31, 2015 at 2:37 PM, Jon Harrop <jon@ffconsultancy.com> wrote:

> David MENTRÉ:
> > That's said, an interesting step might not be to include other libraries
> but the mechanism to install them, i.e. opam.
>
> I think that is exactly the right thing to do. This is a packaging issue,
> not a language issue.
>

Including OPAM does not address the issue of standardized data types for
things that everybody needs. You still have many different ways of
streaming data, of storing time, etc.

[-- Attachment #2: Type: text/html, Size: 867 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Oliver Bandel]

Zitat von Emmanuel Surleau <emmanuel.surleau@gmail.com> (Mon, 31 Aug  
2015 17:05:30 +0200)

> On Mon, Aug 31, 2015 at 2:37 PM, Jon Harrop <jon@ffconsultancy.com> wrote:
>
>> David MENTRÉ:
>> > That's said, an interesting step might not be to include other libraries
>> but the mechanism to install them, i.e. opam.
>>
>> I think that is exactly the right thing to do. This is a packaging issue,
>> not a language issue.
>>
>
> Including OPAM does not address the issue of standardized data types for
> things that everybody needs. You still have many different ways of
> streaming data, of storing time, etc.
[...]

So, this could mean including signatures (not structures/implementations)
for these application domains to be included in the OCaml-distribution.
So that the signatures are equal for all, but the implementation is up  
to (can be done by)
external libs (but need to match the signatures from (the future)  
OCaml-distribution)?

Something like signature-proposals from INRIA.

This would have the advantage of central (coordinated) types in OCaml  
projects,
without eating up the time of the OCaml-core-developers to implement  
the structures too.
IMHO this also could show the power of OCaml's module language...


Ciao,
    Oliver

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Simon Cruanes]

[-- Attachment #1: Type: text/plain, Size: 3762 bytes --]

I strongly support the view that interoperability is important. Defining common, compatible types is necessary for a peaceful cohabitation of multiple stdlibs (types for json, xml, result, unicode, iterators, etc.) Some types should be provided by OCaml (result and Uchar imho); some other we should agree on. I tried to propose the structural type 'a sequence = ('a -> unit) -> unit as a glue between data structures (efficient, easy to provide, plays well with flambda) but so far Core and batteries do not use it.

About the licensing, we have a diverse set of licenses, afaict. 

Cheers, 

Le 28 août 2015 14:08:26 UTC+02:00, Christophe Troestler <Christophe.Troestler@umons.ac.be> a écrit :
>Hi,
>
>The topic of a richer standard library was discussed extensively in the
>past (before the advent of opam).  I believe Yaron's opinion is the one
>generally agreed upon: the small core team should focus on the compiler
>and it is up to the community to take care of libraries.
>
>In the last 10 years, the community has not agreed on what the
>“improved” standard library should be, there always have been competing
>proposals and I do not see that stopping in the near future.  I believe
>this is not a bad thing, after all some applications have special
>requirements — such as being able to be compiled to javascript — and it
>is hard for a single library to satisfy everybody without being
>bloated.
>
>However, as this discussion shows, there are a few places where we
>could do a better job.
>
>1. INTEROPERABILITY: While many, possibly overlapping, libraries may be
>seen as sign of liveliness of the community, they become a problem if a
>user has to write boiler plate code to use them together.  Thus I would
>propose that we sit down together and define a minimal set of modules
>for interoperability purposes.  Since these modules would in general
>only define some types, I propose to reserve the names type_* for that,
>possibly with a version number at the end — so newer versions can
>coexist with older ones and provide functions for backward
>compatibility.  Examples of such modules are:
>    - type_time: define date, time, and calendar types;
>- type_html: define a common representation for HTML documents (a
>library can of course provide its own but should also have a function
>to export to type_html);
>    - type_xml
>- type_linear: a common, linear, exchange format between various
>containers (e.g. a lazy list);
>    - etc.
>
>2. LICENSES: Every opam package comes with a license which should help
>companies to choose which ones to use.  For the problem Hongbo
>mentioned, maybe one could develop a tool that does the following:
>given a white-list of licenses that the company has agreed are OK (e.g.
>ISC) and a list of opam packages, the tool would warn if any of the
>(recursive) dependencies does not have a “good” license.
>
>3. APPLICATION DOMAINS: A newcomer has to use resources like
>https://github.com/rizo/awesome-ocaml/blob/master/sotu.md to understand
>what packages are interesting for which domain.  These resources are
>maintained by hand.  We should agree on a list of tags for important
>application domains and add them to the appropriate opam packages.  An
>up-to-date list (linked to each package, homepage,...) can then be
>generated automatically — and posted, say, to opam.ocaml.org
>
>One could also provide meta-packages to install sets of libraries with
>a certain tag — for example all of batteries packages (once split).
>
>My 0.02€,
>C.
>
>-- 
>Caml-list mailing list.  Subscription management and archives:
>https://sympa.inria.fr/sympa/arc/caml-list
>Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>Bug reports: http://caml.inria.fr/bin/caml-bugs

-- 
Simon

[-- Attachment #2: Type: text/html, Size: 4000 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Gabriel Scherer]

On Fri, Aug 28, 2015 at 5:02 PM, Simon Cruanes
<simon.cruanes.2007@m4x.org> wrote:
> I tried to propose the structural type 'a sequence = ('a -> unit) -> unit
> as a glue between data structures [...] but so far Core and batteries
> do not use it.

This is a golden occasion to point out to everyone that contributions
adding `to_seq`, `of_seq` conversion functions to Batteries data
structures are warmly welcome. (If you are wondering what's a
practical step to help improving the "base library" ecosystem...)

You may find out that it is not in general trivial to implement these
(for the reason explained in this blog post:
http://gallium.inria.fr/blog/generators-iterators-control-and-continuations/),
but luckily all the hard lifting has already been done to support
functions to and from the Enum type, and generalizing these to also
provide sequence providers or consumers should not be too hard.

On Fri, Aug 28, 2015 at 5:02 PM, Simon Cruanes
<simon.cruanes.2007@m4x.org> wrote:
> I strongly support the view that interoperability is important. Defining
> common, compatible types is necessary for a peaceful cohabitation of
> multiple stdlibs (types for json, xml, result, unicode, iterators, etc.)
> Some types should be provided by OCaml (result and Uchar imho); some other
> we should agree on. I tried to propose the structural type 'a sequence = ('a
> -> unit) -> unit as a glue between data structures (efficient, easy to
> provide, plays well with flambda) but so far Core and batteries do not use
> it.
>
> About the licensing, we have a diverse set of licenses, afaict.
>
> Cheers,
>
> Le 28 août 2015 14:08:26 UTC+02:00, Christophe Troestler
> <Christophe.Troestler@umons.ac.be> a écrit :
>>
>> Hi,
>>
>> The topic of a richer standard library was discussed extensively in the
>> past (before the advent of opam).  I believe Yaron's opinion is the one
>> generally agreed upon: the small core team should focus on the compiler and
>> it is up to the community to take care of libraries.
>>
>> In the last 10 years, the community has not agreed on what the “improved”
>> standard library should be, there always have been competing proposals and I
>> do not see that stopping in the near future.  I believe this is not a bad
>> thing, after all some applications have special requirements — such as being
>> able to be compiled to javascript — and it is hard for a single library to
>> satisfy everybody without being bloated.
>>
>> However, as this discussion shows, there are a few places where we could
>> do a better job.
>>
>> 1. INTEROPERABILITY: While many, possibly overlapping, libraries may be
>> seen as sign of liveliness of the community, they become a problem
>> if a user has to write boiler plate code to use them together.  Thus I
>> would propose that we sit down together and define a minimal set of modules
>> for interoperability purposes.  Since these modules would in general only
>> define some types, I propose to reserve the names type_* for that, possibly
>> with a version number at the end — so newer versions can coexist with older
>> ones and provide functions for backward compatibility.  Examples of such
>> modules are:
>>     - type_time: define date, time, and calendar types;
>>     - type_html: define a common representation for HTML documents (a
>> library can of course provide its own but should also have a function to
>> export to type_html);
>>     - type_xml
>>     - type_linear: a common, linear, exchange format between various
>> containers (e.g. a lazy list);
>>     - etc.
>>
>> 2. LICENSES: Every opam package comes with a license which should help
>> companies to choose which ones to use.  For the problem Hongbo mentioned,
>> maybe
>> one could develop a tool that does the following: given a white-list of
>> licenses that the company has agreed are OK (e.g. ISC) and a list of opam
>> packages, the tool would warn if any of the (recursive) dependencies does
>> not have a “good” license.
>>
>> 3. APPLICATION DOMAINS: A newcomer has to use resources like
>> https://github.com/rizo/awesome-ocaml/blob/master/sotu.md to understand what
>> packages are interesting for which domain.  These resources are maintained
>> by hand.  We should agree on a list of tags for important application
>> domains and add them to the appropriate opam packages.  An up-to-date list
>> (linked to each package, homepage,...) can then be generated automatically —
>> and posted, say, to opam.ocaml.org
>>
>>    One could also provide meta-packages to install sets of libraries with
>> a certain tag — for example all of batteries packages (once
>> split).
>>
>> My 0.02€,
>> C.
>
>
> --
> Simon

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Oliver Bandel]

Zitat von Christophe Troestler <Christophe.Troestler@umons.ac.be>  
(Fri, 28 Aug 2015 14:08:26 +0200)

[...]
> 1. INTEROPERABILITY: While many, possibly overlapping, libraries may  
> be seen as sign of liveliness of the community, they become a  
> problem if a user has to write boiler plate code to use them  
> together.  Thus I would propose that we sit down together and define  
> a minimal set of modules for interoperability purposes.  Since these  
> modules would in general only define some types, I propose to  
> reserve the names type_* for that, possibly with a version number at  
> the end — so newer versions can coexist with older ones and provide  
> functions for backward compatibility.  Examples of such modules are:
>     - type_time: define date, time, and calendar types;

Example of existing type:

Ocamlnet:
   Netdate.t

...also used by ocaml-rss.


OCaml-Calendar may also make sense.
It offers a lot of functions to do calculations with dates.

=> http://calendar.forge.ocamlcore.org/

So there already is something available...



>     - type_html: define a common representation for HTML documents (a

Example of existing type:

Ocamlnet:
   Nethtml.document

( works for me;-) )


> library can of course provide its own but should also have a  
> function to export to type_html);
>     - type_xml

Example of existing type:
   Xmlm has some types.


Not sure if I'm happy with them.
For what I did so far, maybe too much.
But for more sophisticated xml-parsing that maybe is fine.

XMLM is recommended for use by other OCaml'ers, so it seems to make  
sense to other people too.



IMHO, the idea to look at types first, is a good starting point.

For some categories there is already stuff, that maybe can be picked up.

But I'm not sure if that all should go into a library shipped with OCaml.

Regarding the standard-lib discussion: I assume a standard-library to  
be shipped with
OCaml itself.

Should Date-/Time and so on really be part of std.-lib?
Maybe yes, maybe no. (I tend into direction of "No".)

If yes: only the signatures, and let implementations/structures be outside?
(This is what some peole seem to have proposed??)
So that different implementations fit into the (extended) stdlib-types  
(which are
there for interoperability only?)?


Ciao,
    Oliver

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Ashish Agarwal]

[-- Attachment #1: Type: text/plain, Size: 381 bytes --]

On Thu, Aug 27, 2015 at 7:33 PM, Hongbo Zhang <bobzhang1988@gmail.com>
wrote:

For example, ocamlbuild is no better than omake or obuild or maybe
> ocp-build, but it is mostly widely used build system among the community,
> the major reason, I believe is that it’s shipped with the core distribution.
>

Why do you think ocamlbuild is the most widely used build system?

[-- Attachment #2: Type: text/html, Size: 761 bytes --]

RE: [Caml-list] We need a rich standard library distributed with OCaml, really

[Jon Harrop]

Jesse Haber-Kucharsky wrote:
> I've found the (lack of) documentation to be a major blocker

You need Intellisense.

Cheers,
Jon.

From: hakuch@gmail.com [mailto:hakuch@gmail.com] On Behalf Of Jesse Haber-Kucharsky
Sent: 27 August 2015 17:01
To: Yaron Minsky
Cc: Hongbo Zhang; caml-list@inria.fr
Subject: Re: [Caml-list] We need a rich standard library distributed with OCaml, really

To offer the perspective of a relative outsider who has meekly decided to pipe in:

A standard library with a variety of general-purpose "building block" of functionality is invaluable. I feel it's missing in OCaml.

Here is some important functionality that I'd like to see based on my own experience writing a small-to-moderately sized OCaml program.

- Modules for standard types like `Int` and `Float` as previously mentioned
- More standard definitions like `compose`, `const`, `identity`, etc (also as previously mentioned)
- Comprehensive string and regular expression handling (UTF support can be relegated to an endorsed third party package)
- More helper functions in the `List` and `Option` modules
- A standard general purpose lazy list (stream). I was able to implement this comprehensively in about 300 lines, and its enormously useful. It's absence means that everyone will write their own, or that there will be hundreds in opam
- More immutable data structures like queues, heaps, stacks, etc
- A standard means of concurrency. Lwt is great, but is almost its own standard library at this point, and there's still a split between it and Async.
- Better support for error-handling-by-value with `either` and `option` through helper functions
- The ppx_deriving package reduces a TONNE of boilerplate for making defining ordering on data for instance. It should be a standard extension (or something like it)
- Better documentation/endorsement of build tools. It's possible to get pretty far with ocamlbuild, _tags, and a simple Makefile, but figuring out how to get there was not exactly easy
- Better interfaces to the file system with more structured error handling on failure (`Sys_error` was not the nicest thing to work with).
- Less reliance on exceptions for non-exceptional situations and retrofitting "pure" functions like `hd` with `option` or `either` result types.
- Less duplication and or more aggressive deprecation. It's not clear to me why there should be both "Foo" and "FooLabels" modules, for instance. I also hear that some modules in the standard library are probably best avoided.

Finally, about the governance of such a library:

While libraries like Core and and Batteries are undeniably useful, I found myself somewhat discouraged in practice:

- Batteries has relatively low adoption and I think it does too much
- Core is driven by a private organization. In practise, I've found the (lack of) documentation to be a major blocker, and I'd feel better about using it if it was truly community "owned" (or if there was a non-profit spin-off to support and own it like the F# foundation, for instance).

Best,
--
Jesse


On Thu, Aug 27, 2015 at 10:17 AM, Yaron Minsky <yminsky@janestreet.com> wrote:
The core OCaml team is small, and having them focused on building a
great compiler rather than on building a rich stdlib seems right to
me.  The improvements in packaging I think make the question of
whether it's distributed with the compiler mostly a moot issue, I
think.

On the topic of Core: The issue of binary size is real.  It will get
somewhat better when we drop packed modules from the public release,
which should come in the next few months (support for it internally is
mostly in place.)  That said the module-level dependency tracking is
by its nature coarse, so binaries using Core (or the more portable
Core_kernel) are still going to be relatively large.

That said, I think I'd rather have the compiler folk work on improving
dead-code elimination than integrating and maintaining a standard
library.

As to openness: we publish all changes on github, have a mailing list,
and will accept pull requests; but it's true that the development of
Core is focused within Jane Street, and that is unlikely to change.

y

On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com> wrote:
> Dear OCaml developers,
>     I would like to spend one hour in writing down my experience that why I
> had to write some small utilities again and again, since this happened so
> many times that I believe you might come across such issues before.
>     I am doing some compiler hacking tonight, I needed a utility function
> “String.split” which split a string into a list of strings by whitespace, it
> is just one liner if you use str library. However, since I am doing some low
> level stuff, I would try to avoid such big dependency, and I am pretty sure
> that I have ever written it  for at least three times, I just hoped that I
> could get it done quickly, so I am looking into batteries that if I can
> steal some code, I have to copy some code instead of depend on batteries,
> batteries is too big for my projects. `BatString.nsplit` seems to fit for
> what I needed, I copied the definition of `BatString.nsplit` into REPL, no
> luck, it depends on some previously defined functions, then I copied the
> whole module `BatString` into REPL, still no luck, it depended on another
> module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
> thrown-away `String.split` function again.
>    OCaml is my favorite programming language, and I am very productive at
> it, however, I was annoyed by such things from time to time. We do have four
> *standard libraries* alternatives: batteries, core, extlib and
> ocaml-containers. In my opinion, none of them matches the beauty of the
> OCaml language itself and probably will never catch up if we don’t do
> anything.
>     Note that I don’t want to be offensive to any of these libraries, just
> my personal feedback that why I think it is not a good standard library, I
> appreciated a lot to people who contribute their precious time in
> maintaining these libraries, better than nothing : )
>     - Batteries(extlib)
>       It’s big with dependencies between different modules (OCaml does not
> have a good story in dead code elimination), some of its modules are of low
> quality, for example, batEnum is used everywhere while its implementation is
> buggy. batIO makes things even worse since it is not compatible with
> standard library, some type signatures mismatched IIRC.
>     - ocaml-containers
>       Mostly one man’s project
>     - core
>       I believe core has high quality, however, it suffers the same problem
> as batteries, a big dependency. There is a blocking issue, its development
> process is opaque, for an open source community, I would prefer a standard
> library developed in an open environment.
>     I am not expecting that we could have a  standard library as rich as
> python, but for some utilities, I do believe that shipped with standard
> library or officially supported is the best solution.
>    Thanks — Hongbo
--
Caml-list mailing list.  Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Simon Cruanes]

[-- Attachment #1: Type: text/plain, Size: 8566 bytes --]

Le Sun, 27 Mar 2016, Jon Harrop a écrit :
> > I've found the (lack of) documentation to be a major blocker
> 
> You need Intellisense.

Thanks to your intervention, I missed this thread last autumn. Indeed,
containers is a one-man project, born of the same frustrations several
people have expressed here. Unlike batteries or Core users, I kept
using the standard library, which is why containers is a complement, not
a replacement, to it.

What I would *really* like, though, is being able to contribute some of
the most central parts of containers to the standard library. I don't
think maintaining a extended Option or List module takes a lot of effort
to maintain (if the implementation is kept reasonably sane, and tests
have been written) — at least it doesn't take me a lot of time.

Since I'm already in easter-level fantasy, I'd also like the standard
library to provide more type definitions (similar to UChar or result) so that
external, featurefull libraries can agree on them. In no particular
order: datetime types, S-expressions, lazy lists, iterators (which
should also be pervasive among existing modules)…

Cheers,


> From: hakuch@gmail.com [mailto:hakuch@gmail.com] On Behalf Of Jesse Haber-Kucharsky
> Sent: 27 August 2015 17:01
> To: Yaron Minsky
> Cc: Hongbo Zhang; caml-list@inria.fr
> Subject: Re: [Caml-list] We need a rich standard library distributed with OCaml, really
> 
> To offer the perspective of a relative outsider who has meekly decided to pipe in:
> 
> A standard library with a variety of general-purpose "building block" of functionality is invaluable. I feel it's missing in OCaml.
> 
> Here is some important functionality that I'd like to see based on my own experience writing a small-to-moderately sized OCaml program.
> 
> - Modules for standard types like `Int` and `Float` as previously mentioned
> - More standard definitions like `compose`, `const`, `identity`, etc (also as previously mentioned)
> - Comprehensive string and regular expression handling (UTF support can be relegated to an endorsed third party package)
> - More helper functions in the `List` and `Option` modules
> - A standard general purpose lazy list (stream). I was able to implement this comprehensively in about 300 lines, and its enormously useful. It's absence means that everyone will write their own, or that there will be hundreds in opam
> - More immutable data structures like queues, heaps, stacks, etc
> - A standard means of concurrency. Lwt is great, but is almost its own standard library at this point, and there's still a split between it and Async.
> - Better support for error-handling-by-value with `either` and `option` through helper functions
> - The ppx_deriving package reduces a TONNE of boilerplate for making defining ordering on data for instance. It should be a standard extension (or something like it)
> - Better documentation/endorsement of build tools. It's possible to get pretty far with ocamlbuild, _tags, and a simple Makefile, but figuring out how to get there was not exactly easy
> - Better interfaces to the file system with more structured error handling on failure (`Sys_error` was not the nicest thing to work with).
> - Less reliance on exceptions for non-exceptional situations and retrofitting "pure" functions like `hd` with `option` or `either` result types.
> - Less duplication and or more aggressive deprecation. It's not clear to me why there should be both "Foo" and "FooLabels" modules, for instance. I also hear that some modules in the standard library are probably best avoided.
> 
> Finally, about the governance of such a library:
> 
> While libraries like Core and and Batteries are undeniably useful, I found myself somewhat discouraged in practice:
> 
> - Batteries has relatively low adoption and I think it does too much
> - Core is driven by a private organization. In practise, I've found the (lack of) documentation to be a major blocker, and I'd feel better about using it if it was truly community "owned" (or if there was a non-profit spin-off to support and own it like the F# foundation, for instance).
> 
> Best,
> --
> Jesse
> 
> 
> On Thu, Aug 27, 2015 at 10:17 AM, Yaron Minsky <yminsky@janestreet.com> wrote:
> The core OCaml team is small, and having them focused on building a
> great compiler rather than on building a rich stdlib seems right to
> me.  The improvements in packaging I think make the question of
> whether it's distributed with the compiler mostly a moot issue, I
> think.
> 
> On the topic of Core: The issue of binary size is real.  It will get
> somewhat better when we drop packed modules from the public release,
> which should come in the next few months (support for it internally is
> mostly in place.)  That said the module-level dependency tracking is
> by its nature coarse, so binaries using Core (or the more portable
> Core_kernel) are still going to be relatively large.
> 
> That said, I think I'd rather have the compiler folk work on improving
> dead-code elimination than integrating and maintaining a standard
> library.
> 
> As to openness: we publish all changes on github, have a mailing list,
> and will accept pull requests; but it's true that the development of
> Core is focused within Jane Street, and that is unlikely to change.
> 
> y
> 
> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com> wrote:
> > Dear OCaml developers,
> >     I would like to spend one hour in writing down my experience that why I
> > had to write some small utilities again and again, since this happened so
> > many times that I believe you might come across such issues before.
> >     I am doing some compiler hacking tonight, I needed a utility function
> > “String.split” which split a string into a list of strings by whitespace, it
> > is just one liner if you use str library. However, since I am doing some low
> > level stuff, I would try to avoid such big dependency, and I am pretty sure
> > that I have ever written it  for at least three times, I just hoped that I
> > could get it done quickly, so I am looking into batteries that if I can
> > steal some code, I have to copy some code instead of depend on batteries,
> > batteries is too big for my projects. `BatString.nsplit` seems to fit for
> > what I needed, I copied the definition of `BatString.nsplit` into REPL, no
> > luck, it depends on some previously defined functions, then I copied the
> > whole module `BatString` into REPL, still no luck, it depended on another
> > module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
> > thrown-away `String.split` function again.
> >    OCaml is my favorite programming language, and I am very productive at
> > it, however, I was annoyed by such things from time to time. We do have four
> > *standard libraries* alternatives: batteries, core, extlib and
> > ocaml-containers. In my opinion, none of them matches the beauty of the
> > OCaml language itself and probably will never catch up if we don’t do
> > anything.
> >     Note that I don’t want to be offensive to any of these libraries, just
> > my personal feedback that why I think it is not a good standard library, I
> > appreciated a lot to people who contribute their precious time in
> > maintaining these libraries, better than nothing : )
> >     - Batteries(extlib)
> >       It’s big with dependencies between different modules (OCaml does not
> > have a good story in dead code elimination), some of its modules are of low
> > quality, for example, batEnum is used everywhere while its implementation is
> > buggy. batIO makes things even worse since it is not compatible with
> > standard library, some type signatures mismatched IIRC.
> >     - ocaml-containers
> >       Mostly one man’s project
> >     - core
> >       I believe core has high quality, however, it suffers the same problem
> > as batteries, a big dependency. There is a blocking issue, its development
> > process is opaque, for an open source community, I would prefer a standard
> > library developed in an open environment.
> >     I am not expecting that we could have a  standard library as rich as
> > python, but for some utilities, I do believe that shipped with standard
> > library or officially supported is the best solution.
> >    Thanks — Hongbo


-- 
Simon Cruanes

http://weusepgp.info/
key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3  7D8D 4AC0 1D08 49AA 62B6

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Caml-list] We need a rich standard library distributed with OCaml, really

[Yaron Minsky]

Merlin is actually pretty great for this! It provides auto-completion,
type throwback, documentation throwback, go-to-definition, and a few
other nice features.  It's worth checking out.

y

On Sun, Mar 27, 2016 at 4:54 PM, Jon Harrop <jon@ffconsultancy.com> wrote:
> Jesse Haber-Kucharsky wrote:
>> I've found the (lack of) documentation to be a major blocker
>
> You need Intellisense.
>
> Cheers,
> Jon.
>
> From: hakuch@gmail.com [mailto:hakuch@gmail.com] On Behalf Of Jesse Haber-Kucharsky
> Sent: 27 August 2015 17:01
> To: Yaron Minsky
> Cc: Hongbo Zhang; caml-list@inria.fr
> Subject: Re: [Caml-list] We need a rich standard library distributed with OCaml, really
>
> To offer the perspective of a relative outsider who has meekly decided to pipe in:
>
> A standard library with a variety of general-purpose "building block" of functionality is invaluable. I feel it's missing in OCaml.
>
> Here is some important functionality that I'd like to see based on my own experience writing a small-to-moderately sized OCaml program.
>
> - Modules for standard types like `Int` and `Float` as previously mentioned
> - More standard definitions like `compose`, `const`, `identity`, etc (also as previously mentioned)
> - Comprehensive string and regular expression handling (UTF support can be relegated to an endorsed third party package)
> - More helper functions in the `List` and `Option` modules
> - A standard general purpose lazy list (stream). I was able to implement this comprehensively in about 300 lines, and its enormously useful. It's absence means that everyone will write their own, or that there will be hundreds in opam
> - More immutable data structures like queues, heaps, stacks, etc
> - A standard means of concurrency. Lwt is great, but is almost its own standard library at this point, and there's still a split between it and Async.
> - Better support for error-handling-by-value with `either` and `option` through helper functions
> - The ppx_deriving package reduces a TONNE of boilerplate for making defining ordering on data for instance. It should be a standard extension (or something like it)
> - Better documentation/endorsement of build tools. It's possible to get pretty far with ocamlbuild, _tags, and a simple Makefile, but figuring out how to get there was not exactly easy
> - Better interfaces to the file system with more structured error handling on failure (`Sys_error` was not the nicest thing to work with).
> - Less reliance on exceptions for non-exceptional situations and retrofitting "pure" functions like `hd` with `option` or `either` result types.
> - Less duplication and or more aggressive deprecation. It's not clear to me why there should be both "Foo" and "FooLabels" modules, for instance. I also hear that some modules in the standard library are probably best avoided.
>
> Finally, about the governance of such a library:
>
> While libraries like Core and and Batteries are undeniably useful, I found myself somewhat discouraged in practice:
>
> - Batteries has relatively low adoption and I think it does too much
> - Core is driven by a private organization. In practise, I've found the (lack of) documentation to be a major blocker, and I'd feel better about using it if it was truly community "owned" (or if there was a non-profit spin-off to support and own it like the F# foundation, for instance).
>
> Best,
> --
> Jesse
>
>
> On Thu, Aug 27, 2015 at 10:17 AM, Yaron Minsky <yminsky@janestreet.com> wrote:
> The core OCaml team is small, and having them focused on building a
> great compiler rather than on building a rich stdlib seems right to
> me.  The improvements in packaging I think make the question of
> whether it's distributed with the compiler mostly a moot issue, I
> think.
>
> On the topic of Core: The issue of binary size is real.  It will get
> somewhat better when we drop packed modules from the public release,
> which should come in the next few months (support for it internally is
> mostly in place.)  That said the module-level dependency tracking is
> by its nature coarse, so binaries using Core (or the more portable
> Core_kernel) are still going to be relatively large.
>
> That said, I think I'd rather have the compiler folk work on improving
> dead-code elimination than integrating and maintaining a standard
> library.
>
> As to openness: we publish all changes on github, have a mailing list,
> and will accept pull requests; but it's true that the development of
> Core is focused within Jane Street, and that is unlikely to change.
>
> y
>
> On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com> wrote:
>> Dear OCaml developers,
>>     I would like to spend one hour in writing down my experience that why I
>> had to write some small utilities again and again, since this happened so
>> many times that I believe you might come across such issues before.
>>     I am doing some compiler hacking tonight, I needed a utility function
>> “String.split” which split a string into a list of strings by whitespace, it
>> is just one liner if you use str library. However, since I am doing some low
>> level stuff, I would try to avoid such big dependency, and I am pretty sure
>> that I have ever written it  for at least three times, I just hoped that I
>> could get it done quickly, so I am looking into batteries that if I can
>> steal some code, I have to copy some code instead of depend on batteries,
>> batteries is too big for my projects. `BatString.nsplit` seems to fit for
>> what I needed, I copied the definition of `BatString.nsplit` into REPL, no
>> luck, it depends on some previously defined functions, then I copied the
>> whole module `BatString` into REPL, still no luck, it depended on another
>> module `BatReturn`, then I stopped here, it’s time to write my own ad-hoc
>> thrown-away `String.split` function again.
>>    OCaml is my favorite programming language, and I am very productive at
>> it, however, I was annoyed by such things from time to time. We do have four
>> *standard libraries* alternatives: batteries, core, extlib and
>> ocaml-containers. In my opinion, none of them matches the beauty of the
>> OCaml language itself and probably will never catch up if we don’t do
>> anything.
>>     Note that I don’t want to be offensive to any of these libraries, just
>> my personal feedback that why I think it is not a good standard library, I
>> appreciated a lot to people who contribute their precious time in
>> maintaining these libraries, better than nothing : )
>>     - Batteries(extlib)
>>       It’s big with dependencies between different modules (OCaml does not
>> have a good story in dead code elimination), some of its modules are of low
>> quality, for example, batEnum is used everywhere while its implementation is
>> buggy. batIO makes things even worse since it is not compatible with
>> standard library, some type signatures mismatched IIRC.
>>     - ocaml-containers
>>       Mostly one man’s project
>>     - core
>>       I believe core has high quality, however, it suffers the same problem
>> as batteries, a big dependency. There is a blocking issue, its development
>> process is opaque, for an open source community, I would prefer a standard
>> library developed in an open environment.
>>     I am not expecting that we could have a  standard library as rich as
>> python, but for some utilities, I do believe that shipped with standard
>> library or officially supported is the best solution.
>>    Thanks — Hongbo
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs