From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 889547F168 for ; Fri, 28 Aug 2015 02:31:59 +0200 (CEST) IronPort-PHdr: 9a23:RiMjIhJg7ZmHSYLld9mcpTZWNBhigK39O0sv0rFitYgVK/zxwZ3uMQTl6Ol3ixeRBMOAu6kC1bed6PGocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC1ILpiqvpqtX6WEZhunmUWftKNhK4rAHc5IE9oLBJDeIP8CbPuWZCYO9MxGlldhq5lhf44dqsrtY4q3wD89pozcNLUL37cqIkVvQYSW1+ayFm0vb2rgHORhej4X4VU2Ne0kYZQluN0BavfJrqqSayluN31TPfHtD9TbkuQjezp/NpRQTzhQ8HPjQ06mLKgcx5lrlYsVSqoBkpkKDOZ4TAFP14canaNeEaTGxOFpJbUCZTAoq6YNJeX7opMuNRro27rFwL+0jtTTKwDf/in2cbzkT92rc3hqF8SAw= Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=bobzhang1988@gmail.com; spf=Pass smtp.mailfrom=bobzhang1988@gmail.com; spf=None smtp.helo=postmaster@mail-qk0-f170.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of bobzhang1988@gmail.com) identity=pra; client-ip=209.85.220.170; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="bobzhang1988@gmail.com"; x-sender="bobzhang1988@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of bobzhang1988@gmail.com designates 209.85.220.170 as permitted sender) identity=mailfrom; client-ip=209.85.220.170; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="bobzhang1988@gmail.com"; x-sender="bobzhang1988@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-qk0-f170.google.com) identity=helo; client-ip=209.85.220.170; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="bobzhang1988@gmail.com"; x-sender="postmaster@mail-qk0-f170.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0DRAwAZq99VlKrcVdFVCYNvLAE8rTCQIYIshXsCgT88EAEBAQEBAQEBEAEBAQEHCwsJHzCCHYIGAQEBAwESER0BGxILAQMBCwYFCw0NGgMCAiECEQEFAQoBEQYTEgIHB4d2AQMKCAUIo0iBLz4xi0CBbIJ5ihsKGScDClaERwEBAQEBAQQBAQEBAQEBARQBBQ6IaoFmgQOCT4FTDVgEBgGCaS+BFAWFdAyPADYHhQaGAIFtghCGWA+KL4VvNYEXEQaCHhyBcFWCTQEBAQ X-IPAS-Result: A0DRAwAZq99VlKrcVdFVCYNvLAE8rTCQIYIshXsCgT88EAEBAQEBAQEBEAEBAQEHCwsJHzCCHYIGAQEBAwESER0BGxILAQMBCwYFCw0NGgMCAiECEQEFAQoBEQYTEgIHB4d2AQMKCAUIo0iBLz4xi0CBbIJ5ihsKGScDClaERwEBAQEBAQQBAQEBAQEBARQBBQ6IaoFmgQOCT4FTDVgEBgGCaS+BFAWFdAyPADYHhQaGAIFtghCGWA+KL4VvNYEXEQaCHhyBcFWCTQEBAQ X-IronPort-AV: E=Sophos;i="5.17,423,1437429600"; d="scan'208,217";a="144014772" Received: from mail-qk0-f170.google.com ([209.85.220.170]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 28 Aug 2015 02:31:57 +0200 Received: by qkda128 with SMTP id a128so20375035qkd.3 for ; Thu, 27 Aug 2015 17:31:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=vT4du+ix6yBtfRyFxUafkgJgbwEHgoE7JlcIKe6x1lg=; b=Br9zmMpZNpP1SVPBOVdf6+spoYzqSObWGVHpBIC6C2TnI2SIgmodnzyRncHCoQg5qP LXlXi8S0G/65Eq14cM+Z2zr3kCVkwO+944XOauiccyWqTOiZ0apJjoa/WtM9hsalnJ9V gmCY3JTSlaJYMdeI89zgjEIzD1cdMFIoxBIP8IzgovbSIve1hFvRF51p5qUX6MWzFLm/ sKgnQ20ir2zzC23hpR8egQsEV4W78lcwe+AmbxRa8h7XLw5QFXX07vJ9M135Ap/wPqah 41t4d2wlNlbI3XeJShCiu5bzIDmhEPsMRK1vPLYNcYIKPOupLa52n7Kee4AOxVCA5kYz h/Mg== X-Received: by 10.55.192.130 with SMTP id v2mr11220393qkv.82.1440721915891; Thu, 27 Aug 2015 17:31:55 -0700 (PDT) Received: from [192.168.5.10] (ip-64-134-70-81.public.wayport.net. [64.134.70.81]) by smtp.gmail.com with ESMTPSA id 68sm2252048qhf.37.2015.08.27.17.31.55 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Aug 2015 17:31:55 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_38A7B3E2-F203-42FE-9E5F-EA95D6D80F79" Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) From: Hongbo Zhang In-Reply-To: Date: Thu, 27 Aug 2015 20:33:02 -0400 Cc: "caml-list@inria.fr" , gabriel.scherer@gmail.com, yminsky@janestreet.com, daniel.buenzli@erratique.ch Message-Id: References: <1C02B1E2-D17D-4008-998E-B17048C62DFA@gmail.com> To: Jesse Haber-Kucharsky X-Mailer: Apple Mail (2.2098) Subject: Re: [Caml-list] We need a rich standard library distributed with OCaml, really --Apple-Mail=_38A7B3E2-F203-42FE-9E5F-EA95D6D80F79 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi all, I did not realize that my email got so many responses, but it does mean= that this is a major concern among the community. I agree with Yaron that having the core team building a great compiler = rather than on building a rich stdlib seems more useful, indeed, my current= interest is also focused on the compiler itself. But there are some really talented programmers who are also interested i= n improving the standard library, like Daniel Bunzli (CCed). Building the s= tandard library is technically not more challenging than improving the comp= iler itself, but it is as important if not even more for a health community= . What I proposed is that we need officially support one standard library i= nstead of keep reinventing the wheels or just stand by. This kind of offici= al promotion does play a very important role. For example, ocamlbuild is no= better than omake or obuild or maybe ocp-build, but it is mostly widely us= ed build system among the community, the major reason, I believe is that it= =E2=80=99s shipped with the core distribution. So, if we promote a standard= library, by default that library will be in the included path, like what w= e currently did for stdlib.=20=20=20 Thanks to the attributes, we can introduce [@@ocaml.experimental] for s= ome experimental APIs, another development pattern I found particular usefu= l is that we can keep the standard library in a different repo and merge it= back when we make a release for the new compiler, of course, any changes t= o the standard library should be well understood by the core ocaml develope= rs. Since standard library is expected to be backward compatible in most ca= ses, so I would not expect that this could cause any bootstrapping issues. Thanks =E2=80=94 Hongbo > On Aug 27, 2015, at 12:00 PM, Jesse Haber-Kucharsky wrote: >=20 > To offer the perspective of a relative outsider who has meekly decided to= pipe in: >=20 > A standard library with a variety of general-purpose "building block" of = functionality is invaluable. I feel it's missing in OCaml. >=20 > Here is some important functionality that I'd like to see based on my own= experience writing a small-to-moderately sized OCaml program. >=20 > - Modules for standard types like `Int` and `Float` as previously mention= ed > - More standard definitions like `compose`, `const`, `identity`, etc (als= o as previously mentioned) > - Comprehensive string and regular expression handling (UTF support can b= e relegated to an endorsed third party package) > - More helper functions in the `List` and `Option` modules > - A standard general purpose lazy list (stream). I was able to implement = this comprehensively in about 300 lines, and its enormously useful. It's ab= sence means that everyone will write their own, or that there will be hundr= eds in opam > - More immutable data structures like queues, heaps, stacks, etc > - A standard means of concurrency. Lwt is great, but is almost its own st= andard library at this point, and there's still a split between it and Asyn= c. > - Better support for error-handling-by-value with `either` and `option` t= hrough helper functions > - The ppx_deriving package reduces a TONNE of boilerplate for making defi= ning ordering on data for instance. It should be a standard extension (or s= omething like it) > - Better documentation/endorsement of build tools. It's possible to get p= retty far with ocamlbuild, _tags, and a simple Makefile, but figuring out h= ow to get there was not exactly easy > - Better interfaces to the file system with more structured error handlin= g on failure (`Sys_error` was not the nicest thing to work with). > - Less reliance on exceptions for non-exceptional situations and retrofit= ting "pure" functions like `hd` with `option` or `either` result types. > - Less duplication and or more aggressive deprecation. It's not clear to = me why there should be both "Foo" and "FooLabels" modules, for instance. I = also hear that some modules in the standard library are probably best avoid= ed. >=20 > Finally, about the governance of such a library: >=20 > While libraries like Core and and Batteries are undeniably useful, I foun= d myself somewhat discouraged in practice: >=20 > - Batteries has relatively low adoption and I think it does too much > - Core is driven by a private organization. In practise, I've found the (= lack of) documentation to be a major blocker, and I'd feel better about usi= ng it if it was truly community "owned" (or if there was a non-profit spin-= off to support and own it like the F# foundation, for instance). >=20 > Best, > -- > Jesse >=20 >=20 > On Thu, Aug 27, 2015 at 10:17 AM, Yaron Minsky > wrote: > The core OCaml team is small, and having them focused on building a > great compiler rather than on building a rich stdlib seems right to > me. The improvements in packaging I think make the question of > whether it's distributed with the compiler mostly a moot issue, I > think. >=20 > On the topic of Core: The issue of binary size is real. It will get > somewhat better when we drop packed modules from the public release, > which should come in the next few months (support for it internally is > mostly in place.) That said the module-level dependency tracking is > by its nature coarse, so binaries using Core (or the more portable > Core_kernel) are still going to be relatively large. >=20 > That said, I think I'd rather have the compiler folk work on improving > dead-code elimination than integrating and maintaining a standard > library. >=20 > As to openness: we publish all changes on github, have a mailing list, > and will accept pull requests; but it's true that the development of > Core is focused within Jane Street, and that is unlikely to change. >=20 > y >=20 > On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang > wrote: > > Dear OCaml developers, > > I would like to spend one hour in writing down my experience that w= hy I > > had to write some small utilities again and again, since this happened = so > > many times that I believe you might come across such issues before. > > I am doing some compiler hacking tonight, I needed a utility functi= on > > =E2=80=9CString.split=E2=80=9D which split a string into a list of stri= ngs by whitespace, it > > is just one liner if you use str library. However, since I am doing som= e low > > level stuff, I would try to avoid such big dependency, and I am pretty = sure > > that I have ever written it for at least three times, I just hoped tha= t I > > could get it done quickly, so I am looking into batteries that if I can > > steal some code, I have to copy some code instead of depend on batterie= s, > > batteries is too big for my projects. `BatString.nsplit` seems to fit f= or > > what I needed, I copied the definition of `BatString.nsplit` into REPL,= no > > luck, it depends on some previously defined functions, then I copied the > > whole module `BatString` into REPL, still no luck, it depended on anoth= er > > module `BatReturn`, then I stopped here, it=E2=80=99s time to write my = own ad-hoc > > thrown-away `String.split` function again. > > OCaml is my favorite programming language, and I am very productive = at > > it, however, I was annoyed by such things from time to time. We do have= four > > *standard libraries* alternatives: batteries, core, extlib and > > ocaml-containers. In my opinion, none of them matches the beauty of the > > OCaml language itself and probably will never catch up if we don=E2=80= =99t do > > anything. > > Note that I don=E2=80=99t want to be offensive to any of these libr= aries, just > > my personal feedback that why I think it is not a good standard library= , I > > appreciated a lot to people who contribute their precious time in > > maintaining these libraries, better than nothing : ) > > - Batteries(extlib) > > It=E2=80=99s big with dependencies between different modules (OCa= ml does not > > have a good story in dead code elimination), some of its modules are of= low > > quality, for example, batEnum is used everywhere while its implementati= on is > > buggy. batIO makes things even worse since it is not compatible with > > standard library, some type signatures mismatched IIRC. > > - ocaml-containers > > Mostly one man=E2=80=99s project > > - core > > I believe core has high quality, however, it suffers the same pro= blem > > as batteries, a big dependency. There is a blocking issue, its developm= ent > > process is opaque, for an open source community, I would prefer a stand= ard > > library developed in an open environment. > > I am not expecting that we could have a standard library as rich as > > python, but for some utilities, I do believe that shipped with standard > > library or officially supported is the best solution. > > Thanks =E2=80=94 Hongbo >=20 > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs --Apple-Mail=_38A7B3E2-F203-42FE-9E5F-EA95D6D80F79 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi all,
    I did not realize that my email got so many responses, = but it does mean that this is a major concern among the community.
    I agree with Yaron that having the core team bui= lding a great compiler rather than on building a rich stdlib seems more use= ful, indeed, my current interest is also focused on the compiler itself.
   But there are some really talented programm= ers who are also interested in improving the standard library, like Daniel = Bunzli (CCed). Building the standard library is technically not more challe= nging than improving the compiler itself, but it is as important if not eve= n more for a health community. What I proposed is that we need officially s= upport one standard library instead of keep reinventing the wheels or just = stand by. This kind of official promotion does play a very important role. = For example, ocamlbuild is no better than omake or obuild or maybe ocp-buil= d, but it is mostly widely used build system among the community, the major= reason, I believe is that it=E2=80=99s shipped with the core distribution.= So, if we promote a standard library, by default that librar= y will be in the included path, like what we currently did for stdlib. =   
    Thanks to the attributes, w= e can introduce [@@ocaml.experimental] for some experimental APIs, another = development pattern I found particular useful is that we can keep the stand= ard library in a different repo and merge it back when we make a release fo= r the new compiler, of course, any changes to the standard library should b= e well understood by the core ocaml developers. Since standard library is e= xpected to be backward compatible in most cases, so I would not expect that= this could cause any bootstrapping issues.
  &nb= sp;  Thanks =E2=80=94 Hongbo
On Aug 27, 2015, at 12:00 PM, Jesse H= aber-Kucharsky <j= esse@haberkucharsky.com> wrote:

To offer the perspecti= ve of a relative outsider who has meekly decided to pipe in:

A standard library with a variety of = general-purpose "building block" of functionality is invaluable. I feel it'= s missing in OCaml.

Here is some important functionality that I'd like to see based on my ow= n experience writing a small-to-moderately sized OCaml program.

- Modules for standard types= like `Int` and `Float` as previously mentioned
- More= standard definitions like `compose`, `const`, `identity`, etc (also as pre= viously mentioned)
- Comprehensive string and regular = expression handling (UTF support can be relegated to an endorsed third part= y package)
- More helper functions in the `List` and `= Option` modules
- A standard general purpose lazy list= (stream). I was able to implement this comprehensively in about 300 lines,= and its enormously useful. It's absence means that everyone will write the= ir own, or that there will be hundreds in opam
- More = immutable data structures like queues, heaps, stacks, etc
- A standard means of concurrency. Lwt is great, but is almost its ow= n standard library at this point, and there's still a split between it and = Async.
- Better support for error-handling-by-value wi= th `either` and `option` through helper functions
- Th= e ppx_deriving package reduces a TONNE of boilerplate for making defining o= rdering on data for instance. It should be a standard extension (or somethi= ng like it)
- Better documentation/endorsement of buil= d tools. It's possible to get pretty far with ocamlbuild, _tags, and a simp= le Makefile, but figuring out how to get there was not exactly easy
- Better interfaces to the file system with more structured e= rror handling on failure (`Sys_error` was not the nicest thing to work with= ).
- Less reliance on exceptions for non-exceptional s= ituations and retrofitting "pure" functions like `hd` with `option` or `eit= her` result types.
- Less duplication and or more aggr= essive deprecation. It's not clear to me why there should be both "Foo" and= "FooLabels" modules, for instance. I also hear that some modules in the st= andard library are probably best avoided.

Finally, about the governance of such a library:

While libraries li= ke Core and and Batteries are undeniably useful, I found myself somewhat di= scouraged in practice:

- Batteries has relatively low adoption and I think it does too much<= /div>
- Core is driven by a private organization. In practis= e, I've found the (lack of) documentation to be a major blocker, and I'd fe= el better about using it if it was truly community "owned" (or if there was= a non-profit spin-off to support and own it like the F# foundation, for in= stance).

Best,
--
Jesse


On Thu, Aug 27, 2015 at 10:17 AM, Yaron Minsky <yminsky@janestreet.com> wrote:
The core OCaml team is small, and having them foc= used on building a
great compiler rather than on building a rich stdlib seems right to
me.  The improvements in packaging I think make the question of
whether it's distributed with the compiler mostly a moot issue, I
think.

On the topic of Core: The issue of binary size is real.  It will get somewhat better when we drop packed modules from the public release,
which should come in the next few months (support for it internally is
mostly in place.)  That said the module-level dependency tracking is by its nature coarse, so binaries using Core (or the more portable
Core_kernel) are still going to be relatively large.

That said, I think I'd rather have the compiler folk work on improving
dead-code elimination than integrating and maintaining a standard
library.

As to openness: we publish all changes on github, have a mailing list,
and will accept pull requests; but it's true that the development of
Core is focused within Jane Street, and that is unlikely to change.

y

On Wed, Aug 26, 2015 at 8:52 PM, Hongbo Zhang <bobzhang1988@gmail.com>= ; wrote:
> Dear OCaml developers,
>     I would like to spend one hour in writing down my e= xperience that why I
> had to write some small utilities again and again, since this happened= so
> many times that I believe you might come across such issues before. >     I am doing some compiler hacking tonight, I needed = a utility function
> =E2=80=9CString.split=E2=80=9D which split a string into a list of str= ings by whitespace, it
> is just one liner if you use str library. However, since I am doing so= me low
> level stuff, I would try to avoid such big dependency, and I am pretty= sure
> that I have ever written it  for at least three times, I just hop= ed that I
> could get it done quickly, so I am looking into batteries that if I ca= n
> steal some code, I have to copy some code instead of depend on batteri= es,
> batteries is too big for my projects. `BatString.nsplit` seems to fit = for
> what I needed, I copied the definition of `BatString.nsplit` into REPL= , no
> luck, it depends on some previously defined functions, then I copied t= he
> whole module `BatString` into REPL, still no luck, it depended on anot= her
> module `BatReturn`, then I stopped here, it=E2=80=99s time to write my= own ad-hoc
> thrown-away `String.split` function again.
>    OCaml is my favorite programming language, and I am very = productive at
> it, however, I was annoyed by such things from time to time. We do hav= e four
> *standard libraries* alternatives: batteries, core, extlib and
> ocaml-containers. In my opinion, none of them matches the beauty of th= e
> OCaml language itself and probably will never catch up if we don=E2=80= =99t do
> anything.
>     Note that I don=E2=80=99t want to be offensive to a= ny of these libraries, just
> my personal feedback that why I think it is not a good standard librar= y, I
> appreciated a lot to people who contribute their precious time in
> maintaining these libraries, better than nothing : )
>     - Batteries(extlib)
>       It=E2=80=99s big with dependencies between d= ifferent modules (OCaml does not
> have a good story in dead code elimination), some of its modules are o= f low
> quality, for example, batEnum is used everywhere while its implementat= ion is
> buggy. batIO makes things even worse since it is not compatible with > standard library, some type signatures mismatched IIRC.
>     - ocaml-containers
>       Mostly one man=E2=80=99s project
>     - core
>       I believe core has high quality, however, it= suffers the same problem
> as batteries, a big dependency. There is a blocking issue, its develop= ment
> process is opaque, for an open source community, I would prefer a stan= dard
> library developed in an open environment.
>     I am not expecting that we could have a  stand= ard library as rich as
> python, but for some utilities, I do believe that shipped with standar= d
> library or officially supported is the best solution.
>    Thanks =E2=80=94 Hongbo

--
Caml-list mailing list.  Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list<= br class=3D""> Beginner's list: http://groups.yahoo.com/gro= up/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs


= --Apple-Mail=_38A7B3E2-F203-42FE-9E5F-EA95D6D80F79--