Re: [Caml-list] GODI available for download

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Sam, 2003-08-02 um 05.28 schrieb Jacques Garrigue:
> Hi Gerd,
> 
> I attempted to install godi on FreeBSD 5.1, but my first attempt
> lamentably failed.
> The error I got was
> [...]
> I join a patch.

Thank you.

> The fix was easy, but getting it to work difficult, as you have to
> erase the godi directory, and use the cleanup script in the
> distribution before recompiling everything (strange for such a small
> detail).

The bootstrap script uses the binaries it creates; installations and
compilations are not separated, so you cannot restart at the failed
step. I am not really happy about that, but I hope that fixes will no
longer be necessary when the bootstrap process stabilizes.

> Also, godi_build and godi_info were not installed, and I had to
> install them by hand.

I guess because the script did run until its end (it is a "set -e"
script, so it stops after the first error).

> Next, building godi-ocaml and godi-ocaml-labltk. godi-ocaml went fine,
> but godi-ocaml-labltk failed, because it needs some configuration
> options (I see no way to pass them). 

The idea is to put all configuration options into godi.conf, and that
every driver Makefile pulls the options it needs from this file.

> Since I'm used to BSD packages, I
> go to the godi-ocaml-labltk/work directory, and reconfigure by hand.
> Build goes fine, but it uses ocamlopt rather than the faster
> ocamlopt.opt.
> 
> I was also a bit wondering why it installs so many utilities, as they
> are all there already on FreeBSD :-) But compatibility can be a pain.

I have simply copied them from the NetBSD bootstrap script. I do not
know whether we need all them (for the uninformed reader, we are talking
about BSD make, lukemftp, pax, digest, mtree, and the tools for the
binary packages). I guess mtree is not used at all because it requires
root privileges, so it should be possible to omit it. Regarding bmake, I
do not know how identical the NetBSD and FreeBSD versions are; it looks
like that they have been independently developed after the fork (the
FreeBSD man page lists fewer syntax elements than the NetBSD version).
You can try to make the godi_make symlink pointing to the FreeBSD
version. pax (which is a tool implementing the tar and cpio commands) is
standardized, but the danger is too big that non-POSIX versions like
GNU-tar are unintentionally used. The differences are small, but I do
not want to get bug reports because of deviating tar formats.

Regarding ftp we can try to let the script find out the native version
and use it.

> I have two questions concerning godi itself:
>  - is it a good idea to use a small tgz by package?
>    we might en up with lots of them!
>    personnally I would rather go for updating directly from a CVS
>    repository, and eventually distribute a big tgz containing all
>    instruction corresponding to a specific version of ocaml.
>    but you may have your reasons.

CVS is a good tool to get the current development version, i.e. for the
unstable branch (when it exists). For the casual user, it is too random
which version is the current one. There are further disadvantages, e.g.
you cannot use CVS behind firewalls (at least like firewalls that are
typically used in company LANs).

On the other hand, a big tgz has the disadvantage that you cannot "pin"
certain versions of libraries, so that they are excluded from updates.

The current solution wants to be a compromise that works for everybody.
That the checks are so slow is caused by bmake calls that find out the
version on the disk; this can be improved, e.g. by comparing the $Id$
tags once they are in place.

By the way, I am not using CVS to manage the sources but subversion.
There are several reasons: (1) I want to check it out, (2) it has
features that can be very helpful to track foreign sources, and (3) it
works even over HTTP.

>  - what about windows? Do you think it is reasonnable to have a binary
>    distribution of godi with all the necessary commands (not depending
>    on cygwin)? Or a distribution on top of cygwin.
>    Not depending on cygwin would be a big plus for occasional users.

For a binary distribution we basically need the package tools, and some
scripts around them. I do not know whether there are Windows ports. Of
course, these tools are typical Unix programs, i.e. lots of pipes,
forks, etc., so that a MinGW port would be some work (but certainly
doable).

The scripts around these tools could be written in O'Caml itself, as we
do not have the bootstrap problem for a binary distribution. (As a last
resort, we could also rewrite the package tools in O'Caml, which is not
too hard.)

Am Sam, 2003-08-02 um 11.52 schrieb Jacques Garrigue: 
> Some more considerations on my experience.
> 
> > Next, building godi-ocaml and godi-ocaml-labltk. godi-ocaml went fine,
> > but godi-ocaml-labltk failed, because it needs some configuration
> > options (I see no way to pass them). Since I'm used to BSD packages, I
> > go to the godi-ocaml-labltk/work directory, and reconfigure by hand.
> 
> I realize now that this configuration problem is going to be very
> complicated. BSD packages are OS-oriented: they assume a closed world,
> where everything is controlled by the packaging system. So you don't
> need to tell them where something is on the system, they already know
> it.
> 
> With godi, the situation is different. For instance, tcl/tk headers
> may be in many places, maybe even several versions of them, and the
> user should be able to indicate where to find them. Same thing for
> camlimages, lablgl, lablgtk, etc...
> My feeling is that there has to be a way to give this information
> by hand the first time you install a package, and these parameters
> would be reused in following installations. When the package is
> installed as a dependency, this may require some interaction from the
> user.

In the current version of GODI, the godi.conf file contains these
configuration options. Of course, this is a non-interactive solution,
but I have added now hints that are printed on the screen just before
the build starts (and the user must confirm). So the user is now
reminded that there are such options, and where they can be found.

In the future, I can imagine that we have a labltk program that assists
the user in configuring the system. This GUI would be the better
solution for "camlimages, lablgl, lablgtk, etc...", but of course not
for labltk itself.

I see only one way for labltk: implanting more knowledge about the
various operating systems, and more intelligence to find even unusual
locations for tcl/tk.

There is not only the question of configuration options to find external
installations, but also the question which libraries one should better
include in GODI. Currently, I have only included GDBM (because an ndbm
implementation is necessary for a full O'Caml installation, and I wanted
to have something predictable), and PCRE (because PCRE4 is not yet
widely available, but the O'Caml bindings require it). Of course, the
question is where to draw the line. If we had a GUI for GODI, it could
even be justified to include tcl/tk if the user wishes so.

> Also, the possibility of finishing the work by hand must be left,
> otherwise recovering from simple configurations bugs can become
> unwieldly.

This is also necessary for the developers. Currently, the whole make
infrastructure works like NetBSD's, and I am not going to change this.

> I'm afraid this is just one example of mismatch between an
> OS-oriented packager and a language-oriented multi-platform packager.

I hope there will not be more. I have already resolved several other
mismatches: 

- The original scripts require root privileges for installation. Of course,
  we do not want that.

- The original package tools store the version conditions like
  "libraryXY>=3.5" in the binary packages. I have changed that to
  "libraryXY=3.5" because of OCaml's strict interface checksums.

- Some of the PLIST issues are resolved. There is a special
  PLIST.godi format that recognizes additional directives, so you
  can package whole directories, with filename globbing, and under
  a number of O'Caml-specific conditions.

- There can be several GODI installations on the same system (no
  conflict because there is only one /etc/mk.conf as in NetBSD)

Up to now, it was quite easy to change the build and packaging scripts
as needed (I guess there would be more trouble if I had used a closed
system like rpm).

> Another one is the use of makefiles internally and in packages. While
> makefiles are not intrinsically bad, they have one big default: they
> depend heavily on the behaviour of external commands on the system. If
> you have only platform this is not a problem: you just have to check
> once that this works, but if you must handle multiple platforms, it is
> very hard to be sure that the Makefile is portable.
> I don't know how CPAN handles that, but this is going to be painful.
> Using an ocaml only tool like ocamake may be an interesting option, at
> least for simple packages.

It is painful. I have already run into problems, e.g. /bin/sh on Solaris
is not even POSIX-compliant, and it starts subshells under much more
conditions than a standard shell. Fortunately, most of the tools are not
that problematic; an mkdir is an mkdir. And most of the O'Caml sources
come with Makefiles that have the same portability problems; in the long
run, I guess most of the bugs will occur in the upstream Makefiles.

(There is already such a problem in godi-wlex. The "patch" utility of
Solaris does not grok the wlex patch.)

In my opinion, the portability problem can only be _finally_ solved if
the upstream Makefiles are replaced by something better. Perl has its
Makefile.PL, which is a Perl script that writes the system-dependent
Makefile. However, Perl is not compiled, and the possible cases are
simpler to manage.

> Just some thoughts.
> I certainly believe that godi is a big step in the right direction.
> 
> Jacques Garrigue
> 
> P.S. A small detail, but would it be hard to replace godi_ftp by
> fetch? On my system godi_ftp is 351K whereas fetch is only 18K. I
> suppose this is also reflected by the size of the sources.

You can set FETCH_CMD in godi.conf; I don't know if the options are
compatible enough.

> I wonder also if there is a packaging system based on gmake rather
> than bmake. It feels a bit strange to install bmake only for internal
> purposes.

I consider bmake as just another scripting language. Of course, we could
port the whole bsd.pkg.mk to gmake, but does this make sense? It has
more than 4600 lines, and it makes heavy use of the BSD extensions.

If we are going to switch to another tool, it will be more attractive to
write a small make utility in O'Caml that can be scripted in O'Caml.
There is very ugly code in bsd.pkg.mk that would have a trivial and
elegant expression in O'Caml.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners