Re: [Caml-list] [OSR] Ports-like package management system

[Alain Frisch <alain@frisch.fr>]

Hello,

It's great to see some discussion and thoughts about package management 
for OCaml software. Of course, various kind of users have different 
expectations. I'm quite happy with GODI as a package developer or 
maintainer or when I need to write small applications that uses external 
OCaml libraries. But I'd like to describe our current approach here at 
LexiFi concerning the use of external libraries. This is the point of 
view a company with a few "big" projects, several developers, and 
several source languages. I guess that many companies that use OCaml 
would face a similar situation.

We develop end-user applications and our main target and development 
platform is Windows (using the native Win32 and Mingw ports of OCaml), 
although we also compile everything under Linux. This unfortunately 
rules out any existing package management system for OCaml. We use a 
number of third-party libraries in OCaml, C, Fortran that come with 
source code and some .Net components that come in binary form.

We don't want to depend on too many external packages to be downloaded 
in order to build our projects. Currently, we only rely on a few tools 
like MS Visual Studio, omake, a Subversion client to be installed on the 
development machines. Everything else is considered part of our project 
tree and kept under the control of Subversion. In particular, we have 
our locally-patched version of OCaml and all the third-party libraries 
(either in source or binary form) in the repository. (Note that our 
membership to the Caml Consortium even allow use to redistribute 
directly this modified version of OCaml.)

Putting everything under the control of SVN has several advantages:

- the developers don't need to do anything special on their computer 
when a new library is installed or updated; we know they all have a 
uniform environment;

- it is trivial to add local patches;

- we have control over the precise version of the libraries we use (we 
cannot afford to depend on any external decision to upgrade a library);

- in particular, various development branches can have different 
versions, and it is trivial to go back in time and build an old version 
with exactly the same libraries as it used to be.

The drawback, of course, is that we need to take care manually of 
upgrading to new versions of external libraries. The OCaml libraries we 
use are quite stable and we rarely find a serious problem that cannot be 
resolved by a small local patch. The only project that we follow 
regularly is OCaml itself. We have a vendor branch in our SVN on which 
we simply copy a current version from INRIA's public CVS; then we merge 
the diffs from this vendor branch into our project.

Another point worth mentioning is that all our project tree, except the 
OCaml part, is managed as a single OMake project. Creating OMakefiles 
for external OCaml/C/Fortan libraries is usually very easy (something 
like 2-3 lines for each library). It is quite comfortable to be able to 
patch, add debug statements, etc in 3rd party components and see the 
changes quickly (without recompiling everything). Another motivation for 
writing our own OMakefiles is that the Makefiles provided in the 
upstream packages are often broken under Windows (which is not 
surprising given how hard it is to deal with Windows path names within a 
Makefile).

We don't use findlib but have recreated a very minimal version of it 
within omake. Although we haven't done it yet, I can imagine something 
more serious: omake functions to read findlib's META files (or something 
similar, maybe in OMake syntax so that they can be simply include'd) so 
that various parts of the project can declare in a simple way which 
OCaml libraries they use and that correct flags can be automatically 
computed and passed to the compilers.

We also use omake to do some local configuration (finding where some 
external tools are, deciding what to compile or not depending on the 
host system, ...), manage local installation, prepare directory 
structures to help installer builders and help manage the list of files 
that should be distributed. We still have some bash/Perl scripts around 
(using Cygwin under Windows), but we should be able to get rid of them 
quite easily (omake has its own scripting language). I believe that 
ocamlbuild is really a great tool but in its current form, it could not 
replace our use of omake. Since we need something like omake anyway, we 
don't see a compelling reason to switch to ocamlbuild and to write 
plugins to compile C#/Fortan/...

We are quite satisfied with the current situation. Things could be 
streamlined if upstream packages came with OMakefiles that could be 
integrated directly in such configurations (that would require some 
conventions to be created and followed). It is really a minor point for 
us, but it could allow to transpose our approach to smaller projects 
where writing OMakefiles by hand is not desirable. Also, I can imagine 
that some scripts could automate the task of downloading, integrating 
and tracking external libraries in a SVN repository, maybe with a nice 
GUI and a central list of  packages (a la GODI). Note that almost 
nothing here is really OCaml-specific (except maybe some omake functions 
/ conventions to emulate findlib).

-- Alain