Re: new+old Camlp4 (was Re: [Caml-list] Bug in ocamlyacc)

["Markus Mottl" <markus.mottl@gmail.com>]

On 4/24/07, Martin Jambon <martin.jambon@ens-lyon.org> wrote:
> I wanted to start this as dedicated thread, but let's do it now.
> The current situation with camlp4 3.10-beta is terrible. Not
> because the new camlp4 is not good or anything, but because it is not
> compatible with the old one and yet replaces it. Replacing the old one by
> the new one in the next release of OCaml is practically unmanageable
> because:
>
> 1) Source duplication is unavoidable
> 2) Upgrading to the new camlp4 is not automatic

And:

  3) Documentation for the new camlp4 is almost non-existent.

> The direct consequences are that it will take time until all syntax
> extensions support camlp4/ocaml 3.10, and probably anything new will work
> with either ocaml 3.09 or 3.10, but not both.

I am not so concerned about the 3.09 compiler not working with the new
macro syntax, because presumably people will upgrade somewhat quickly
anyway.  Unless... they depend on old syntax extensions that have not
been rewritten yet.

Very substantial parts of our code base depend on non-trivial Camlp4
macros, and it is a lot of tedious work to rewrite those.  Even though
it's a huge PITA, we could live with the lack of backwards
compatibility as long as there were at least sufficient documentation
to support a quick transition.  I am also currently working on a new
non-trivial extension, and haven't started writing the macro part yet
in the hope that documentation will soon be available to improve my
productivity.

It seems reasonable to predict that many people won't (be able to)
upgrade to 3.10 for the reasons explained by Martin.  I can't imagine
that INRIA wants to make a release that has a hard time of being
accepted by the community.  This would be a waste of INRIA's
resources, too.

I'd therefore strongly suggest that INRIA plan more carefully how to
make the next release.  From my point of view, the best way would be
to provide sufficient documentation in advance to allow Camlp4
developers to rewrite their macros in time.  It would be a great
advantage, too, if the old Camlp4 supported any new syntax (not
extensions, just the base language; there are probably very few
changes anyway) so that people have at least a short reprieve during
which they can just use an old camlp4 installation before they are
forced to upgrade their macros for future OCaml-releases.

> I started upgrading some of my non-trivial syntax extensions already. It's
> ok to translate a file once, but I'll be adding new features and I don't
> know if I should continue to support ocaml 3.09 or ocaml 3.10. Even though
> I want to support both, I can't because it takes twice the time and it's
> pretty boring.

Indeed...

> It could go like "the old camlp4 is deprecated, and support will be
> dropped completely [one year after the first stable-and-documented release
> of camlp4 3.10]"

Sounds reasonable.

> * It means two versions of camlp4 would be distributed and installed with
>   ocaml 3.10.

This would surely be helpful, but I don't think that this is
absolutely necessary.  It isn't horribly hard to have two
OCaml-installations, and refer to the old camlp4 where necessary.

> * Upgraded programs could add new features only to the 3.10 version, while
>   maintaining minimal support (bug fixes) for the 3.09 version, without
>   forcing the camlp4 programmer to use ocaml 3.09 for testing.

Maintaining a second OCaml-installation would certainly be a bigger
problem if the macro code itself refers to a lot of external packages
(this may be rare), which may then need to be maintained in a
separation installation, too.

It would surely be most helpful for the community if both good
documentation were available, and a backwards compatible camlp4
version were provided.  If there are limited resources at INRIA that
prevent solving both problems, my preference is that they be invested
in the former rather than the latter...

Regards,
Markus

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com