From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 456417FE3A for ; Mon, 3 Jun 2013 12:46:59 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of berenger@riken.jp) identity=pra; client-ip=134.160.33.161; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="berenger@riken.jp"; x-sender="berenger@riken.jp"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of berenger@riken.jp designates 134.160.33.161 as permitted sender) identity=mailfrom; client-ip=134.160.33.161; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="berenger@riken.jp"; x-sender="berenger@riken.jp"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of postmaster@postman.riken.jp designates 134.160.33.161 as permitted sender) identity=helo; client-ip=134.160.33.161; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="berenger@riken.jp"; x-sender="postmaster@postman.riken.jp"; x-conformance=sidf_compatible; x-record-type="v=spf1" X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ak0AABHtq1GGoCGhhWdsb2JhbABagzmsX5IigRUOAQEBCgsLBRYogiMBAQQBODYKBgsLGAkEEggHCQMCAQIBDyQBERMGAgEBDgmHYAMJBgyxUQMKiQOMTYE1gSyDWAOJHogQhCqBZoEphHWFfwOIPoFb X-IPAS-Result: Ak0AABHtq1GGoCGhhWdsb2JhbABagzmsX5IigRUOAQEBCgsLBRYogiMBAQQBODYKBgsLGAkEEggHCQMCAQIBDyQBERMGAgEBDgmHYAMJBgyxUQMKiQOMTYE1gSyDWAOJHogQhCqBZoEphHWFfwOIPoFb X-IronPort-AV: E=Sophos;i="4.87,790,1363129200"; d="scan'208";a="19976980" Received: from postman1.riken.jp (HELO postman.riken.jp) ([134.160.33.161]) by mail2-smtp-roc.national.inria.fr with ESMTP; 03 Jun 2013 03:15:38 +0200 Received: from postman.riken.jp (postman1.riken.jp [127.0.0.1]) by postman.riken.jp (Postfix) with SMTP id EB4102588002 for ; Mon, 3 Jun 2013 10:15:34 +0900 (JST) Received: from [172.27.98.103] (rikad98.riken.jp [134.160.214.98]) by postman.riken.jp (Postfix) with ESMTPA id 2A47132A0085 for ; Mon, 3 Jun 2013 10:15:32 +0900 (JST) Message-ID: <51ABEE34.7080009@riken.jp> Date: Mon, 03 Jun 2013 10:15:32 +0900 From: Francois Berenger User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130510 Thunderbird/17.0.6 MIME-Version: 1.0 To: caml-list@inria.fr References: <20130523235355.GI6510@siouxsie> <1629005.lH05byJ9SH@groupon> <1804446.xtBoISCFl2@groupon> <87a9ndovip.fsf@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 6.0.0.2142326, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2013.6.3.10631 Subject: Re: Problems to get larger user base ... (Re: [Caml-list] OCaml's variables) On 05/31/2013 10:32 PM, Mike Lin wrote: > It sure would be nice if OPAM happened to provide install-able OCaml > binaries, but as real production systems are an amalgam of many > different moving parts, I think it would solve only one small part of a > larger problem. I think an increasingly popular solution - which is > taking Malcom's idea to its extreme logical conclusion - is to deploy > virtual machine images. In grid5000, you even deploy (not virtual) images. But this is in an HPC context. > In my company, for example, we build such images > frequently as part of continuous integration and, during a deploy, just > slowly cycle thousands of cloud instances over to a new image. LXC > (chroot on steroids) is another popular way to instantiate this strategy > with less overhead, and we've also experimented with CDE > (http://www.pgbovine.net/cde.html). > I understand Facebook somehow manages to compile one ginormous binary, > though: > http://arstechnica.com/business/2012/04/exclusive-a-behind-the-scenes-look-at-facebook-release-engineering/ > > > On Wed, May 29, 2013 at 11:02 PM, Malcolm Matalka > wrote: > > No I'm not suggesting building on every machine. I am suggesting using > a Build Server, such as Jenkins or Bamboo or Hydra, to build your system > and package it, then you can deploy it through another mechanism. > > Opam generating RPMs (or packages for you) I believe is wrong because: > > 1 - There are a lot of options when it comes to building a package. > What should the init scripts look like? Where should the various files > go? What about configs, etc. These are issues opam should not concern > it self with. On top of that, there are already tools for building a > RPM packages from a directory structure, such as fpm. > > 2 - Which package systems should Opam support? RPM, Deb? I use Nix, > why don't you support me? Some other guy uses something else, what > about him? I would predict a lot of heated and ultimately fruitless ML > discussions about people wanting their package support in a > particular way. > > 3 - *Maybe* Opam should support exporting the closure of a package to an > external directory structure, then you can package it in any way you > want. > > However, IME, and someone with more knowledge in Ocaml please correct > me, if you are deploying to production on a lot of machines I don't > really see why you would want to install Ocaml libraries to those > machines. Ocaml libraries are statically linked so copying the produced > binary should be sufficient. This is how I deploy Ocaml apps to the > clusters I have worked on, with success. > > Perhaps I am missing something? > > If you want a 'deploy' command to be added to Opam that takes a > configuration and pushes builds to machines, I am vehemently opposed to > that. If you want a mechanism for exporting a package closure someplace > else that you can package, I am less opposed to that. But I don't > develop Opam so my opinion doesn't really matter :) > > /M > > Chet Murthy > > writes: > > > I'm glad we're having this discussion. > > > > On Thursday, May 30, 2013 06:52:25 AM Malcolm Matalka wrote: > >> I think out would be wrong for opam to try to solve this > problem. There > >> are already many tools available for deploying (Ansible, Puppet, > Chef, > >> Fabric, Capistrano). Such a later can be build on top of opam > of need be. > > > > I think this is incorrect. Let me explain. > > > > (1) when we look at deploying complex collections of code/libs/data > > onto multiple machines, usually we assume that the code has already > > been built. > > > > (2) but let's first dispatch the case where the code has -not- been > > built. In such a case, I presume you're proposing that the code be > > built on each machine, yes? > > > > (a) this drastically increases the CPU required to perform upgrades > > and deploys > > > > (b) but far, far, far more importantly, it means that on each > > machine, a nontrivially complex script runs that builds the actual > > installed binaries. If that script contains -any- > nondeterminism or > > environmental sensitivity, it could produce different results on > > different machines. The technical term is "version skew". > > > > In scale-out systems, this sort of "skew" is absolutely fatal, > because > > it means that machines/nodes are not a priori interchangeable. And > > all of fast-fail fault-tolerance depends on nodes being > > interchangeable. > > > > (3) But let's say that what you really mean is that we should use > > tools like puppet/chef/capistrano to copy collections of > > binaries/libs/data to target machines and install them. These > > scripts/recipes are written by some person. You could have equally > > well suggested that that person build Debian packages (or RPMs) of > > each OPAM package, writing out all the descriptions and manifests. > > > > And manually specifying all dependencies and requiremeents. > > > > Either way, that person is doing a job that OPAM already does a lot > > of, and does quite well. Gosh, wouldn't it be nice if OPAM could > > generate those RPMs? Well, it's a little more complicated than that, > > but really, not much more. The complexity comes in that you -might- > > (I'm not saying I have this part figured out yet) want ways to > > -generalize- (say) the camlp5 package so that it could be > installed on > > many different base OPAM installations. > > > > But setting aside that nice-to-have, imagine that OPAM knew how to > > generate RPMs from each package it installed, and from the ocaml+opam > > base itself. You combine those, and you can: > > > > (i) install ocaml, opam, and a bunch of packages > > > > (ii) push a button, and out come a pile of RPMs, along with > > dependencies amongst them (and hopefully on the relevant > > environmental RPMs (e.g., libpcre-dev for pcre-ocaml, etc) so that > > you can just stuff those RPMs into a YUM repo, go to a second box, > > and say > > > > "yum install opam ocaml pcre-ocaml" > > > > and get everything slurped down and installed, just as if OPAM had > > installed it all, package-by-package. > > > > --chet-- > > > > P.S. And this doesn't even get into the unsuitability of chef/puppet > > for managing software package installation. There's a reason that no > > distro uses such schemes to install the large and complex sets of > > packages needed to run amodern Linux box. And why there is no Linux > > version of Microsorft's "DLL Hell". Linux distros by and large (and > > esp Debian and Ubuntu) have worked hard to make package installation > > foolproof -- and chef/puppet etc are anything but. > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > >