From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 5733B8239C for ; Mon, 2 Apr 2018 20:18:10 +0200 (CEST) Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=hrafaelsen@gmail.com; spf=Pass smtp.mailfrom=hrafaelsen@gmail.com; spf=None smtp.helo=postmaster@mail-oi0-f54.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of hrafaelsen@gmail.com) identity=pra; client-ip=209.85.218.54; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="hrafaelsen@gmail.com"; x-sender="hrafaelsen@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of hrafaelsen@gmail.com designates 209.85.218.54 as permitted sender) identity=mailfrom; client-ip=209.85.218.54; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="hrafaelsen@gmail.com"; x-sender="hrafaelsen@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-oi0-f54.google.com) identity=helo; client-ip=209.85.218.54; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="hrafaelsen@gmail.com"; x-sender="postmaster@mail-oi0-f54.google.com"; x-conformance=sidf_compatible IronPort-PHdr: =?us-ascii?q?9a23=3AsnzpDh3IOzs41bDzsmDT+DRfVm0co7zxezQtwd8Z?= =?us-ascii?q?se0TKvad9pjvdHbS+e9qxAeQG9mDsLQc06L/iOPJYSQ4+5GPsXQPItRndiQuro?= =?us-ascii?q?EopTEmG9OPEkbhLfTnPGQQFcVGU0J5rTngaRAGUMnxaEfPrXKs8DUcBgvwNRZv?= =?us-ascii?q?JuTyB4Xek9m72/q99pHPbQhEniaxba9vJxiqsAvdsdUbj5F/Iagr0BvJpXVIe+?= =?us-ascii?q?VSxWx2IF+Yggjx6MSt8pN96ipco/0u+dJOXqX8ZKQ4UKdXDC86PGAv5c3krgfM?= =?us-ascii?q?QA2S7XYBSGoWkx5IAw/Y7BHmW5r6ryX3uvZh1CScIMb7S60/Vza/4KdxUBLnii?= =?us-ascii?q?kHOT43/m/Ul8J+kr5UrQm7qBBj2YPZep2ZOOZ8c67bYNgURXBBXsFUVyFZAI2z?= =?us-ascii?q?dY0PD/EcNuhBq4nyuUYFowekCgmiHuzg1DtIhmTo0q08zu8sFgHG3BY6ENIAqn?= =?us-ascii?q?vbstH1NLwMXuCz16TIwjDDYOlX2Tf58oTHbhchofSVUL92bMHfx04vFwbfgVWR?= =?us-ascii?q?r4zoJzyV1uURs2ib8upvTvijhHIgqwF0pDWk28QiipHRi44L1lzJ8T91zYU1KN?= =?us-ascii?q?GiVkJ3fN2pHIFQuiyVMYZ9X9ksTHtyuCkgz70LoZ67czYOyJQg3xPfbuaIc4mM?= =?us-ascii?q?4h76U+aRICt0iGtreL+/iBu+60egyur7Vsm71FZFsDBJncXLtnAIzxDT686HRe?= =?us-ascii?q?Vh/kq5xzqDywTe5vtHLE00j6bXNYMtz70qmpcTvknPBir2l1/3jK+SeEUk4O+o?= =?us-ascii?q?6+H/b7XmvJ+TLYF0hRvgPaQqgcywH+I4PxMBX2ie4+u81bnj8VflT7VNi/06ir?= =?us-ascii?q?PZv4zCJcQHuq65BBdY3Zo55Ba6CzeqydAYnXgcLFJZYx+HlIjoO1TWIP/iF/u/?= =?us-ascii?q?glKskC1qx//cJLHhDI/NfTD/l+LqdLN5rkpd0xYbzNZF5psSBKtSDuj0XxrRvd?= =?us-ascii?q?rDRjg9PkSOyujuBdFm24VWDWKVC6GQPafJvFmg6ecmIu3Kb4gQ7mWuY8M57uLj?= =?us-ascii?q?2Cdq0WQWerOkiMNOOSKIW89+KkDcWkLCx9IIEGMEpA07FbW4h1iLUDoVbHG3Df?= =?us-ascii?q?tlumMLTbm+BIKGfbiDxaSb1X7iTJJTb2FCTFuLFCWwLtjWa7I3cCuXZ/RZvHkE?= =?us-ascii?q?WLymEdJz0BivsEr9yuIiILeIvCIfsp3n2Z5+4OiBzRw=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0CbAABxcsJahjbaVdFdHgEGDINmPW8oC?= =?us-ascii?q?oNSgR2TZYF0dRqHRYYqhGSBegslhF8ChCsHGQcBBDAYAQIBAQEBAQEBAQESAQE?= =?us-ascii?q?BCAsLCCgvgjcigkwBAgIBIx0BGxIMAwELBgMCCzcCAiIBEQEFARwZG4RZAQMNC?= =?us-ascii?q?A+SJ495PIsFgX8FAReCbwWDSgoZJg1VV4IdAgYSh0+BVD+BDIIISAYugxECAoI?= =?us-ascii?q?HglOCVAKLcYRThnYIhVGIXYw3iRWGWQ8DHoEEHIIKMxojTzGCEgkKgV0kDA4Jj?= =?us-ascii?q?hg9MI4JAQE?= X-IPAS-Result: =?us-ascii?q?A0CbAABxcsJahjbaVdFdHgEGDINmPW8oCoNSgR2TZYF0dRq?= =?us-ascii?q?HRYYqhGSBegslhF8ChCsHGQcBBDAYAQIBAQEBAQEBAQESAQEBCAsLCCgvgjcig?= =?us-ascii?q?kwBAgIBIx0BGxIMAwELBgMCCzcCAiIBEQEFARwZG4RZAQMNCA+SJ495PIsFgX8?= =?us-ascii?q?FAReCbwWDSgoZJg1VV4IdAgYSh0+BVD+BDIIISAYugxECAoIHglOCVAKLcYRTh?= =?us-ascii?q?nYIhVGIXYw3iRWGWQ8DHoEEHIIKMxojTzGCEgkKgV0kDA4Jjhg9MI4JAQE?= X-IronPort-AV: E=Sophos;i="5.48,396,1517871600"; d="scan'208,217";a="260616976" Received: from mail-oi0-f54.google.com ([209.85.218.54]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 02 Apr 2018 20:18:08 +0200 Received: by mail-oi0-f54.google.com with SMTP id 188-v6so13627246oih.8 for ; Mon, 02 Apr 2018 11:18:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Mmz3+hOeyIVGtL56wYrpCe6bDtn6UQY4xrIn9guKGAE=; b=AiqKINk8ahc69JbctP/xNR+73UKfWPN0VgZpueXEV7YsJPThzq7DWJzDgAtYgpFGRe InqBzo25nh2FwOFVBtY665QxQ1SCCBjOt2xKfnqvChVF5TsQWk12TcalH1x73xXF4LUQ qfIhO69/QsuvtJ6Gp85ZjVce8pJsTNmV1MgTTAOOH+eSzHz5yGMixN57tuRrqc8ubVhk y/MaSwZSgNkvjiSVkSJonsLzs/Il/ysn7A1obb33y714ZsGI1ikpmseRn3GX7qlE5sAu jiBd3urLLr00TSrWBteFFHVfTuxJCHiRAj0O08SsCwIre60HsW1uIltUzDhLZMii4SCo VVSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Mmz3+hOeyIVGtL56wYrpCe6bDtn6UQY4xrIn9guKGAE=; b=XK3DYRN+ZjUeqXkgm9TMIuDepcaFyLmLxYZf4YGX6wdhiwS9Mb9E43RjWEeP5SJlBY h61pzVWAiK8weO2k03kIUQY6nk2XizCONSdzaO/zieiXqRGsBl1XJevcLIfWE6zBYMaf 8xgaGCPDz93VrM+wnwu6ILjMIXIuF08LV4fsQP/iMpt68u5PhoOvliRHNWh/BEz3Awdi fICwck47xAjF3jJ2veoNkKgDoqcEwY11Dqd8sVA9Ij1/HnGRboujiO3Zn6TBcmyO6gnV shzB6v7P9bhkGQcrqR33aJCWX/qjPPr9Trg9V6v1z9hw+upTFtmlxeJbmKPV8pz0Erm9 Md7A== X-Gm-Message-State: ALQs6tC2ShzJPzl/EMxXhvD3UO/hUFU9iXufhvxpD2hWaLsR7TC9bFLp i0tj9ZqXYciZK6zeu1Jy8xJXRS4Bv5U81tveip9a0SQx X-Google-Smtp-Source: AIpwx48izbpNuPDdDMeqFz5meUQX2rZQYzTdEm9vfw8mqgq5FZHGk/yorDA890tTN3BNSsh1mHvCwAw4zNum40zTaCc= X-Received: by 2002:aca:39c2:: with SMTP id g185-v6mr5363930oia.355.1522693086694; Mon, 02 Apr 2018 11:18:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.203.2 with HTTP; Mon, 2 Apr 2018 11:17:46 -0700 (PDT) In-Reply-To: <4945274.q1Lr0Fu7B2@agaric> References: <4945274.q1Lr0Fu7B2@agaric> From: Hans Ole Rafaelsen Date: Mon, 2 Apr 2018 20:17:46 +0200 Message-ID: To: caml-list@inria.fr Content-Type: multipart/alternative; boundary="00000000000088d3240568e19c58" Subject: Re: [Caml-list] Create a deployable application unit from OPAM? --00000000000088d3240568e19c58 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I managed to get it working. First I tried to copy files from the lib folder (.cmxs etc), but did not manage to solve all dependencies. It would probably have worked if I had spent some more time figuring out what was still missing. So I installed opam2 and installed the packages using the destdir folder (/opt/myapp) for p in `opam2 list | cut -d ' ' -f 1`; do opam2 install --destdir /opt/myapp/ $p ; done (This is all, so I still have a job to do to figure out which I can do without) Also installed the Osigen app under /opt/myapp/ The whole folder was then copied a target node under /opt/myapp/ To solve paths I had to crate a new findlib.conf file and put it under /opt/myapp/etc/ $ cat /opt/myapp/etc/findlib.conf destdir=3D"/opt/myapp/lib" path=3D"/opt/myapp/lib" ocamlc=3D"ocamlc.opt" ocamlopt=3D"ocamlopt.opt" ocamldep=3D"ocamldep.opt" ocamldoc=3D"ocamldoc.opt" To run it, have have this script setting the environment $ cat /opt/myapp/run.sh #!/bin/bash MYTOP=3D/opt/myapp/ CAML_LD_LIBRARY_PATH=3D$MYTOP/lib/stublibs MANPATH=3D:$MYTOP/man PERL5LIB=3D$MYTOP/lib/perl5: OCAML_TOPLEVEL_PATH=3D/lib/toplevel PATH=3D$MYTOP/bin:$PATH OCAMLPATH=3D$MYTOP export OCAMLFIND_CONF=3D$MYTOP/etc/findlib.conf ocsigenserver.opt -c $MYTOP/etc/myapp/myapp.conf Also, oscigenserver seems to use the ~/.opam2/4.06.1/ paths form extensions loaded, so these have to be configured. Got this kind of log messages ocsigenserver.opt: ocsipersist:dbm: Launching a new Ocsidbm process: /home/hans/.opam2/4.06.1/lib/ocsigenserver/extensions/ocsidbm.opt on directory /opt/myapp/var/data/myapp/ocsipersist. Needed to change the configuration of the module to use the right path for the binary vi /opt/myapp/etc/myapp/myapp.conf ... ... Still have a problem with mime-types, but it's probably a similar config change. So it's not totally smooth, but it seems to be doable. Regards, Hans Ole Rafaelsen On Fri, Mar 30, 2018 at 4:11 PM, Louis Gesbert wrote: > There is no perfect solution at the moment, as there are still some > hiccups with relocation of quite a few opam packages, but there are a few > options: > > 1. Not precisely what you were asking for, since it involves re-compiling, > but opam-bundle=C2=B9 can generate an all-in-one package that can easily = be > deployed to a different system and will bootstrap OCaml, opam, the packag= es > and their dependencies, and can put wrappers for the binaries to your PAT= H. > This takes a while, but avoids any relocation problems and is quite > reliable. > The bundle could also be used as base for a container, or even copied > across similar hosts (to the same location), and just the wrappers > reinstalled. > > 2. opam 2.0 has an `opam install --destdir` option, that you could use to > copy the installed artefacts of the required packages to some prefix (be > sure to reinstall all the dependencies, or start from an empty switch). Y= ou > could then tar/gz that prefix, and copy it across hosts. > > 3. in a simpler way, you could just copy the opam switch across hosts. > With opam 2, all metadata is gathered in /.opam-switch/, so > omit that subdirectory and you just get a "normal" Linux prefix (i.e. with > bin/, lib/, etc. subdirectories). > > Of course, 2. and 3. can hit relocation issues, depending on what the > packages you are interested in use (e.g. ocsigenserver seems to have > specific needs). As Gerd mentionned, specifying `CAML_LD_LIBRARY_PATH` and > `OCAMLPATH` (and maybe a few others=C2=B2) may help. > And none of these reduce the end result to "just the binaries needed", > although 2. might help a little by letting you choose which packages get > into the dest-prefix. I am not aware of a way to do this automatically. A > vast super-appoximation could be to just ignore opam build-dependencies: > `opam reinstall --destdir SOME_TEMP_PREFIX $(opam list --installed --rec > --required-by PACKAGES --nobuild --short)`, on the origin host, then > package and ship SOME_TEMP_PREFIX, possibly with wrappers to set the > required PATH variables. > > These are just a few clues, but hope this helps! > > Louis Gesbert =E2=80=94 OCamlPro > > > > =C2=B9 https://github.com/AltGr/opam-bundle > =C2=B2 https://github.com/ocaml/opam-repository/issues/10863# > issuecomment-347174662 > > > - Hans Ole Rafaelsen, 29/03/2018 18:31 - > > Hi, > > > > I have small application written using Ocsigen. Ocaml and the rest of t= he > > development environment is installed using OPAM. > > > > Now I want to install this application on a few other nodes, without > having > > to bring bring along the whole development process for deployment on ea= ch > > node. What I want to end up with, is a single unit, e.g a tar-file, a > > docker container or similar, that can be copied and deployed to the > target > > nodes. This unit should only contain the binaries needed to run the > > application. > > > > I have tried to ask on the Ocsigen mailing-list how do do this for > Ocsigen > > applications, but without success. But I guess the problem is more > related > > to how to deploy applications built using OPAM. I did not find any way = to > > do this with OPAM, but I might have missed something. I have tried to > copy > > all the binary files (.so, .cmxa .cmxs, etc) from the OPAM directory and > > setting the environment variables to point the this new location, but > > without much success. When starting ocsigenserver it was not able to > solve > > all symbols. Looking more into the problem, it looks like binaries such > as > > ocsigenserver has hard coded the location of the OPAM development > > environment. Might be possible to override this using environment > > variables, but I'm not sure how. So I gave up on this track. > > > > > > Is there a way to generate such a deployable unit from OPAM development > > environment? > > > > Do anyone have some tips / best practice on how to deploy applications > > built using OPAM? > > > > (I'm developing and want to deploy on Ubuntu 16.04.) > > > > Regards, > > > > Hans Ole Rafaelsen > > > > > --00000000000088d3240568e19c58 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I managed to get it working.

First I tried to = copy files from the lib folder (.cmxs etc), but did not manage to solve all= dependencies. It would probably have worked if I had spent some more time = figuring out what was still missing. So I installed opam2 and installed the= packages using the destdir folder (/opt/myapp)

for p in `opam2 list= | cut -d ' ' -f 1`; do opam2 install --destdir /opt/myapp/ $p ; do= ne
(This is all, so I still have a job to do to figure out which = I can do without)

Also installed the Osigen app unde= r /opt/myapp/

The whole folder was then copied a target n= ode under /opt/myapp/
To solve paths I had to crate a new fin= dlib.conf file and put it under /opt/myapp/etc/
$ cat /opt/myapp/etc/fin= dlib.conf
destdir=3D"/opt/myapp/lib"
path=3D"/opt/mya= pp/lib"
ocamlc=3D"ocamlc.opt"
ocamlopt=3D"ocamlop= t.opt"
ocamldep=3D"ocamldep.opt"
ocamldoc=3D"ocam= ldoc.opt"

To run it, have have this script setting the environm= ent
$ cat /opt/myapp/run.sh
#!/bin/bash

MYTOP=3D/opt/myapp/
CAML_LD_LIBRARY_PATH=3D$MYTOP/lib/stublibs
MANPATH=3D:$MYTOP/manPERL5LIB=3D$MYTOP/lib/perl5:
OCAML_TOPLEVEL_PATH=3D/lib/toplevel
PA= TH=3D$MYTOP/bin:$PATH
OCAMLPATH=3D$MYTOP

export OCAMLFIND_CONF=3D= $MYTOP/etc/findlib.conf

ocsigenserver.opt -c $MYTOP/etc/myapp/myapp.= conf

Also, oscigenserver seems to use the ~/.opam2/4.06.1= / paths form extensions loaded, so these have to be configured.
Got this kind of log messages
ocsigenserver.opt: ocsipersist:dbm: Lau= nching a new Ocsidbm process: /home/hans/.opam2/4.06.1/lib/ocsigenserver/ex= tensions/ocsidbm.opt on directory /opt/myapp/var/data/myapp/ocsipersist.
Needed to change the configuration of the module to use the right path= for the binary
vi /opt/myapp/etc/myapp/myapp.conf
...
=C2=A0=C2= =A0=C2=A0 <extension findlib-package=3D"ocsigenserver.ext.ocsipersi= st-dbm">
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 <ocsidbm name=3D"= ;/opt/myapp/lib/ocsigenserver/extensions/ocsidbm.opt"/>
=C2=A0= =C2=A0=C2=A0 </extension>
...
Still have = a problem with mime-types, but it's probably a similar config change.

So it's not totally smooth, but it seems to= be doable.

Regards,

Hans Ol= e Rafaelsen




On Fri, Mar 30, 2018 at 4:11 PM= , Louis Gesbert <louis.gesbert@ocamlpro.com> wrote:=
There is no perfect solution at the mome= nt, as there are still some hiccups with relocation of quite a few opam pac= kages, but there are a few options:

1. Not precisely what you were asking for, since it involves re-compiling, = but opam-bundle=C2=B9 can generate an all-in-one package that can easily be= deployed to a different system and will bootstrap OCaml, opam, the package= s and their dependencies, and can put wrappers for the binaries to your PAT= H. This takes a while, but avoids any relocation problems and is quite reli= able.
The bundle could also be used as base for a container, or even copied acros= s similar hosts (to the same location), and just the wrappers reinstalled.<= br>
2. opam 2.0 has an `opam install --destdir` option, that you could use to c= opy the installed artefacts of the required packages to some prefix (be sur= e to reinstall all the dependencies, or start from an empty switch). You co= uld then tar/gz that prefix, and copy it across hosts.

3. in a simpler way, you could just copy the opam switch across hosts. With= opam 2, all metadata is gathered in <switch-prefix>/.opam-switch/, s= o omit that subdirectory and you just get a "normal" Linux prefix= (i.e. with bin/, lib/, etc. subdirectories).

Of course, 2. and 3. can hit relocation issues, depending on what the packa= ges you are interested in use (e.g. ocsigenserver seems to have specific ne= eds). As Gerd mentionned, specifying `CAML_LD_LIBRARY_PATH` and `OCAMLPATH`= (and maybe a few others=C2=B2) may help.
And none of these reduce the end result to "just the binaries needed&q= uot;, although 2. might help a little by letting you choose which packages = get into the dest-prefix. I am not aware of a way to do this automatically.= A vast super-appoximation could be to just ignore opam build-dependencies:= `opam reinstall --destdir SOME_TEMP_PREFIX $(opam list --installed --rec -= -required-by PACKAGES --nobuild --short)`, on the origin host, then package= and ship SOME_TEMP_PREFIX, possibly with wrappers to set the required PATH= variables.

These are just a few clues, but hope this helps!

Louis Gesbert =E2=80=94 OCamlPro



=C2=B9 https://github.com/AltGr/opam-bundle
=C2=B2 https://github.co= m/ocaml/opam-repository/issues/10863#issuecomment-347174662
> - Hans Ole Rafaelsen, 29/03/2018 18:31 -
> Hi,
>
> I have small application written using Ocsigen. Ocaml and the rest of = the
> development environment is installed using OPAM.
>
> Now I want to install this application on a few other nodes, without h= aving
> to bring bring along the whole development process for deployment on e= ach
> node. What I want to end up with, is a single unit, e.g a tar-file, a<= br> > docker container or similar, that can be copied and deployed to the ta= rget
> nodes. This unit should only contain the binaries needed to run the
> application.
>
> I have tried to ask on the Ocsigen mailing-list how do do this for Ocs= igen
> applications, but without success. But I guess the problem is more rel= ated
> to how to deploy applications built using OPAM. I did not find any way= to
> do this with OPAM, but I might have missed something. I have tried to = copy
> all the binary files (.so, .cmxa .cmxs, etc) from the OPAM directory a= nd
> setting the environment variables to point the this new location, but<= br> > without much success. When starting ocsigenserver it was not able to s= olve
> all symbols. Looking more into the problem, it looks like binaries suc= h as
> ocsigenserver has hard coded the location of the OPAM development
> environment. Might be possible to override this using environment
> variables, but I'm not sure how. So I gave up on this track.
>
>
> Is there a way to generate such a deployable unit from OPAM developmen= t
> environment?
>
> Do anyone have some tips / best practice on how to deploy applications=
> built using OPAM?
>
> (I'm developing and want to deploy on Ubuntu 16.04.)
>
> Regards,
>
> Hans Ole Rafaelsen
>
>

--00000000000088d3240568e19c58--