Re: OCaml's Cathedral & Bazaar (was Re: [Caml-list] Completeness of "Unix" run-time library)

[Xavier Leroy <xavier.leroy@inria.fr>]

This discussion is heating up, so allow me to make a few points.

One should carefully distinguish between the core OCaml distribution
(the one that comes out of INRIA) and the whole OCaml programming
environment, which includes a lot of third-party libraries and tools.

The core OCaml distribution should and will remain just that: the
core, i.e. the compilers, run-time system and the tools and libraries
that are closely intertwined with the first two.  We at INRIA do not
have the manpower to maintain, document and make distributions of a
much larger software set.  (Witness the problems with the CDK.)  We are
commited to developing and maintaining that core.  I agree we do this
in a "cathedral" style, but this is intended and unlikely to change.

For everything else, bazaar-style developments from members of this
community are most welcome, and indeed the preferred way to enrich the
OCaml programming environment.  A developer has an itch to scratch,
develops and releases a library or tool, gets it listed on the Hump,
users pick it up if it's good, discuss bugs, features and enhancements
with the developer, etc.  There is absolutely no reason we at INRIA
should interfere with this process: in general, we don't have the
manpower to play a significant role, and we don't have the competences
either (many libraries and tools require expertise in application
domains that we're not familiar with, e.g. database interfaces).

There remains a problem of how to make it easy for everyone to install
and use these third-party contributions.  CPAN managed to do it
through standardization on naming conventions, configuration and
installation procedures, and a *lot* of discipline from the
contributors.  We aren't quite at this point with OCaml, although Gerd
Stolpmann's GODI is an impressive first step in this direction.
Again, it's up to this community to tell whether this is a good
approach that should be pursued, e.g. by providing GODI packaging from
your own libraries.  One cannot just wish there would be a CPAN for
OCaml and just wait for us INRIA folks to come up with it overnight.

> The problem is not simply that INRIA is too cautious, it's that there is 
> no visible process for accepting enhancements to Caml or its libraries 
> from outside INRIA.  INRIA very rarely responds at all, either 
> positively or negatively, to proposed modifications from outsiders (the 
> sole exception seems to be bug fixes).

Don't attribute to malice what is generally a lack of time.  What do
you prefer: that I pontificate on every idea proposed on this mailing
list, or that I fix bugs?

As I said above, the preferred way to contribute to Caml is through
independent libraries and tools, not by aiming at getting your stuff
in the core OCaml distribution.  There are good reasons why we are
very careful indeed with what goes in it:

- As Diego said, it's extremely painful to roll back a change or
  addition that turns out to be a bad idea, because of backward
  compatibility issues.

- Maintenance and documentation takes a lot of time.  Often, it looks
  like contributors expect us to maintain their contributed code.

- Copyright issues are not trivial.  It's important for INRIA and the
  Caml consortium to own the copyright on everything in the core
  distribution.  Significant contributions by others would therefore
  require copyright transfers, whose legality in the French copyright
  law is unclear.

Moreover, a *lot* of the suggested enhancements can be done equally
well, if not better, without touching the core OCaml distribution.
A typical example is syntactic sugar (for regexps, for hashtables, etc):
all this can easily be done as Camlp4 syntax extensions, so don't
expect it to end up in the (already way too rich) core language syntax.

> Recently there has been a long discussion on this list about enhancing 
> the Unix module, and no one from INRIA has said a word about it; this is 
> very discouraging.

Again, this is essentially by lack of time.  If you want my opinion on
this discussion:

- Changing the organization and naming of the Unix library is out of
the question.  Yes, it could be organized a bit more nicely, but that
doesn't deserve breaking all the existing code that uses it.  Still,
the Caml module system makes it easy to wrap existing code in a
different interface, so everyone is welcome to come up with a
differently-structued OS interface.

- IPv6 support is on my to do list.  Missing POSIX syscalls can be
added on a case by case basis if there is clear need.  Having a full
POSIX interface just for the sake of it is low on my priorities.

- Extending the Unix library is a lot harder than what most
contributors realize, because of portability and autoconfiguration
issues.  The world isn't just the latest Linux release.  Writing and
testing the autoconf code for an extension (e.g. IPv6) is often harder
than writing the C-Caml wrapper code for it.

> Has ocaml-lib.sourceforge.net been rejected?

By whom?  It seems like ExtLib is progressing, and if it's good it
will be widely adopted by OCaml users (just like, say, Markus Mottl's
PCRE library was widely adopted).  I don't have anything to say on
this matter.

> INRIA silently working on its own library enhancements which will be 
> incompatibly replace some of the enhancements developed by the 
> community?

As a matter of fact, no, we're not.  But even if we were, these would
not "replace" the work done by others, but at most compete with it.
Users get to choose.

> Is there a plan for the future development of Caml?

The short-term plans are stabilizing the core distribution, preserve
compatibility, and refrain from major user-visible changes.  We are
discussing some internal changes e.g. on the run-time representation
of objects, but these should not change the user's view of the system.
If GODI doesn't take up, maybe we'll invest more efforts into library
packaging and installation frameworks.

> We are like the man in Kafka's novel _The Trial_, who stands for
> years at the door of the Law, and is never told whether he will be
> seen, or when, or if not, why not.

Aren't you overdoing it a little bit? :-)

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners