[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

Alan Schmitt
2021-02-16T13:53:47Z

[-- Attachment #1: Type: text/plain, Size: 18679 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of February 09 to 16,
2021.

Table of Contents
─────────────────

opam 2.0.8 release
opam 2.1.0~beta4
Set up OCaml 1.1.6
Set up OCaml 1.1.7
Old CWN

opam 2.0.8 release
══════════════════

Archive: <https://discuss.ocaml.org/t/ann-opam-2-0-8-release/7242/1>

R. Boujbel announced
────────────────────

We are pleased to announce the minor release of [opam 2.0.8].

This new version contains some fixes, mainly for sandbox and fish
scripts. You can find more information in this [blog post], and more
detailed in the [release note].

/opam is a source-based package manager for OCaml. It supports
multiple simultaneous compiler installations, flexible package
constraints, and a Git-friendly development workflow./

[opam 2.0.8] <https://github.com/ocaml/opam/releases/tag/2.0.8>

[blog post] <https://opam.ocaml.org/blog/opam-2-0-8>

[release note] <https://github.com/ocaml/opam/releases/tag/2.0.8>

opam 2.1.0~beta4
════════════════

Archive: <https://discuss.ocaml.org/t/ann-opam-2-1-0-beta4/7252/1>

David Allsopp announced
───────────────────────

On behalf of the opam team, it gives me great pleasure to announce the
third beta release of opam 2.1. Don’t worry, you didn’t miss beta3 -
we had an issue with a configure script that caused beta2 to report as
beta3 in some instances, so we skipped to beta4 to avoid any further
confusion!

We encourage you to try out this new beta release: there are
instructions for doing so in [our wiki]. The instructions include
taking a backup of your `~/.opam' root as part of the process, which
can be restored in order to wind back. _Please note that local
switches which are written to by opam 2.1 are upgraded and will need
to be rebuilt if you go back to opam 2.0_. This can either be done by
removing `_opam' and repeating whatever you use in your build process
to create the switch, or you can use `opam switch export
switch.export' to backup the switch to a file before installing new
packages. Note that opam 2.1 _shouldn’t_ upgrade a local switch unless
you upgrade the base packages (i.e. the compiler).

[our wiki]
<https://github.com/ocaml/opam/wiki/How-to-test-an-opam-feature>

What’s new in opam 2.1?
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌

• Switch invariants
• Improved options configuration (see the new `option' and expanded
`var' sub-commands)
• Integration of system dependencies (formerly the opam-depext
plugin), increasing their reliability as it integrates the solving
step
• Creation of lock files for reproducible installations (formerly the
opam-lock plugin)
• CLI versioning, allowing cleaner deprecations for opam now and also
improvements to semantics in future without breaking
backwards-compatibility
• Performance improvements to opam-update, conflict messages, and many
other areas
• New plugins: opam-compiler and opam-monorepo

Switch invariants
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

In opam 2.0, when a switch is created the packages selected are put
into the “base” of the switch. These packages are not normally
considered for upgrade, in order to ease pressure on opam’s
solver. This was a much bigger concern early on in opam 2.0’s
development, but is less of a problem with the default mccs solver.

However, it’s a problem for system compilers. opam would detect that
your system compiler version had changed, but be unable to upgrade the
ocaml-system package unless you went through a slightly convoluted
process with `--unlock-base'.

In opam 2.1, base packages have been replaced by switch
invariants. The switch invariant is a package formula which must be
satisfied on every upgrade and install. All existing switches’ base
packages could just be expressed as `package1 & package2 & package3'
etc. but opam 2.1 recognises many existing patterns and simplifies
them, so in most cases the invariant will be `"ocaml-base-compiler" {=
4.11.1}', etc. This means that `opam switch create my_switch
ocaml-system' now creates a _switch invariant_ of `"ocaml-system"'
rather than a specific version of the `ocaml-system' package. If your
system OCaml package is updated, `opam upgrade' will seamlessly switch
to the new package.

This also allows you to have switches which automatically install new
point releases of OCaml. For example:

┌────
│ opam switch create ocaml-4.11 --formula='"ocaml-base-compiler" {>= "4.11.0" & < "4.12.0~"}'
│ --repos=old=git+https://github.com/ocaml/opam-repository#a11299d81591
│ opam install utop
└────

Creates a switch with OCaml 4.11.0 (the `--repos=' was just to select
a version of opam-repository from before 4.11.1 was released). Now
issue:

┌────
│ opam repo set-url old git+https://github.com/ocaml/opam-repository
│ opam upgrade
└────

and opam 2.1 will automatically offer to upgrade OCaml 4.11.1 along
with a rebuild of the switch. There’s not yet a clean CLI for
specifying the formula, but we intend to iterate further on this with
future opam releases so that there is an easier way of saying “install
OCaml 4.11.x”.

opam depext integration
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

opam has long included the ability to install system dependencies
automatically via the [depext plugin]. This plugin has been promoted
to a native feature of opam 2.1.0 onwards, giving the following
benefits:

• You no longer have to remember to run `opam depext', opam always
checks depexts (there are options to disable this or automate it for
CI use). Installation of an opam package in a CI system is now as
easy as `opam install .', without having to do the dance of `opam
pin add -n/depext/install'. Just one command now for the common
case!
• The solver is only called once, which both saves time and also
stabilises the behaviour of opam in cases where the solver result is
not stable. It was possible to get one package solution for the
`opam depext' stage and a different solution for the `opam install'
stage, resulting in some depexts missing.
• opam now has full knowledge of depexts, which means that packages
can be automatically selected based on whether a system package is
already installed. For example, if you have *neither* MariaDB nor
MySQL dev libraries installed, `opam install mysql' will offer to
install `conf-mysql' and `mysql', but if you have the MariaDB dev
libraries installed, opam will offer to install `conf-mariadb' and
`mysql'.

[depext plugin] <https://github.com/ocaml-opam/opam-depext>

opam lock files and reproducibility
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

When opam was first released, it had the mission of gathering together
scattered OCaml source code to build a [community repository]. As time
marches on, the size of the opam repository has grown tremendously, to
over 3000 unique packages with over 18000 unique versions. opam looks
at all these packages and is designed to solve for the best
constraints for a given package, so that your project can keep up with
releases of your dependencies.

While this works well for libraries, we need a different strategy for
projects that need to test and ship using a fixed set of
dependencies. To satisfy this use-case, opam 2.0.0 shipped with
support for _using_ `project.opam.locked' files. These are normal opam
files but with exact versions of dependencies. The lock file can be
used as simply as `opam install . --locked' to have a reproducible
package installation.

With opam 2.1.0, the creation of lock files is also now integrated
into the client:
• `opam lock' will create a `.locked' file for your current switch and
project, that you can check into the repository.
• `opam switch create . --locked' can be used by users to reproduce
your dependencies in a fresh switch.

This lets a project simultaneously keep up with the latest
dependencies (without lock files) while providing a stricter set for
projects that need it (with lock files).

[community repository] <https://github.com/ocaml/opam-repository>

CLI Versioning
┄┄┄┄┄┄┄┄┄┄┄┄┄┄

A new `--cli' switch was added to the first beta release, but it’s
only now that it’s being widely used. opam is a complex enough system
that sometimes bug fixes need to change the semantics of some
commands. For example:

• `opam show --file' needed to change behaviour
• The addition of new controls for setting global variables means that
the `opam config' was becoming cluttered and some things want to
move to `opam var'
• `opam switch create 4.11.1' still works in opam 2.0, but it’s really
an OPAM 1.2.2 syntax.

Changing the CLI is exceptionally painful since it can break scripts
and tools which themselves need to drive `opam'. CLI versioning is
our attempt to solve this. The feature is inspired by the `(lang dune
...)' stanza in `dune-project' files which has allowed the Dune
project to rename variables and alter semantics without requiring
every single package using Dune to upgrade their `dune' files on each
release.

Now you can specify which version of opam you expected the command to
be run against. In day-to-day use of opam at the terminal, you
wouldn’t specify it, and you’ll get the latest version of the CLI. For
example: `opam var --global' is the same as `opam var --cli=2.1
--global'. However, if you issue `opam var --cli=2.0 --global', you
will told that `--global' was added in 2.1 and so is not available to
you. You can see similar things with the renaming of `opam upgrade
--unlock-base' to `opam upgrade --update-invariant'.

The intention is that `--cli' should be used in scripts, user guides
(e.g. blog posts), and in software which calls opam. The only decision
you have to take is the _oldest_ version of opam which you need to
support. If your script is using a new opam 2.1 feature (for example
`opam switch create --formula=') then you simply don’t support opam
2.0. If you need to support opam 2.0, then you can’t use `--formula'
and should use `--packages' instead. opam 2.0 does not have the
`--cli' option, so for opam 2.0 instead of `--cli=2.0' you should set
the environment variable `OPAMCLI' to `2.0'. As with _all_ opam
command line switches, `OPAMCLI' is simply the equivalent of `--cli'
which opam 2.1 will pick-up but opam 2.0 will quietly ignore (and, as
with other options, the command line takes precedence over the
environment).

Note that opam 2.1 sets `OPAMCLI=2.0' when building packages, so on
the rare instances where you need to use the `opam' command in a
_package_ `build:' command (or in your build system), you _must_
specify `--cli=2.1' if you’re using new features.

There’s even more detail on this feature [in our wiki]. We’re still
finalising some details on exactly how `opam' behaves when `--cli' is
not given, but we’re hoping that this feature will make it much easier
in future releases for opam to make required changes and improvements
to the CLI without breaking existing set-ups and tools.

[in our wiki]
<https://github.com/ocaml/opam/wiki/Spec-for-opam-CLI-versioning>

What’s new since the last beta?
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌

• opam now uses CLI versioning ([#4385])
• opam now exits with code 31 if all failures were during fetch
operations ([#4214])
• `opam install' now has a `--download-only' flag ([#4036]), allowing
opam’s caches to be primed
• `opam init' now advises the correct shell-specific command for `eval
$(opam env)' ([#4427])
• `post-install' hooks are now allowed to modify or remove installed
files ([#4388])
• New package variable `opamfile-loc' with the location of the
installed package opam file ([#4402])
• `opam update' now has `--depexts' flag ([#4355]), allowing the
system package manager to update too
• depext support NetBSD and DragonFlyBSD added ([#4396])
• The format-preserving opam file printer has been overhauled
([#3993], [#4298] and [#4302])
• pins are now fetched in parallel ([#4315])
• `os-family=ubuntu' is now treated as `os-family=debian' ([#4441])
• `opam lint' now checks that strings in filtered package formulae are
booleans or variables ([#4439])

and many other bug fixes as listed [on the release page].

[#4385] <https://github.com/ocaml/opam/pull/4385>

[#4214] <https://github.com/ocaml/opam/issues/4214>

[#4036] <https://github.com/ocaml/opam/issues/4036>

[#4427] <https://github.com/ocaml/opam/pull/4427>

[#4388] <https://github.com/ocaml/opam/pull/4388>

[#4402] <https://github.com/ocaml/opam/pull/4402>

[#4355] <https://github.com/ocaml/opam/issues/4355>

[#4396] <https://github.com/ocaml/opam/pull/4396>

[#3993] <https://github.com/ocaml/opam/issues/3993>

[#4298] <https://github.com/ocaml/opam/pull/4298>

[#4302] <https://github.com/ocaml/opam/pull/4302>

[#4315] <https://github.com/ocaml/opam/issues/4315>

[#4441] <https://github.com/ocaml/opam/pull/4441>

[#4439] <https://github.com/ocaml/opam/issues/4439>

[on the release page]
<https://github.com/ocaml/opam/releases/tag/2.1.0-beta4>

New Plugins
╌╌╌╌╌╌╌╌╌╌╌

Several features that were formerly plugins have been integrated into
opam 2.1.0. We have also developed some _new_ plugins that satisfy
emerging workflows from the community and the core OCaml team. They
are available for use with the opam 2.1 beta as well, and feedback on
them should be directed to the respective GitHub trackers for those
plugins.

opam compiler
┄┄┄┄┄┄┄┄┄┄┄┄┄

The [`opam compiler'] plugin can be used to create switches from
various sources such as the main opam repository, the ocaml-multicore
fork, or a local development directory. It can use Git tag names,
branch names, or PR numbers to specify what to install.

Once installed, these are normal opam switches, and one can install
packages in them. To iterate on a compiler feature and try opam
packages at the same time, it supports two ways to reinstall the
compiler: either a safe and slow technique that will reinstall all
packages, or a quick way that will just overwrite the compiler in
place.

[`opam compiler'] <https://github.com/ocaml-opam/opam-compiler>

opam monorepo
┄┄┄┄┄┄┄┄┄┄┄┄┄

The [`opam monorepo'] plugin lets you assemble standalone dune
workspaces with your projects and all of their opam dependencies,
letting you build it all from scratch using only Dune and OCaml. This
satisfies the “monorepo” workflow which is commonly requested by large
projects that need all of their dependencies in one place. It is also
being used by projects that need global cross-compilation for all
aspects of a codebase (including C stubs in packages), such as the
MirageOS unikernel framework.

[`opam monorepo'] <https://github.com/ocamllabs/opam-monorepo>

Next Steps
╌╌╌╌╌╌╌╌╌╌

This is anticipated to be the final beta in the 2.1 series, and we
will be moving to release candidate status after this. We could really
use your help with testing this release in your infrastructure and
projects and let us know if you run into any blockers. If you have
feature requests, please also report them on [our issue tracker] – we
will be planning the next release cycle once we ship opam 2.1.0
shortly.

[our issue tracker] <https://github.com/ocaml/opam/issues>

Set up OCaml 1.1.6
══════════════════

Archive: <https://discuss.ocaml.org/t/ann-set-up-ocaml-1-1-6/7276/1>

Sora Morimoto announced
───────────────────────

This release includes a change to make the OCaml CI workflow on
Windows faster!

I tested this on one of my repos where the build itself is
mere seconds. Before this change, setup-ocaml needed an
average of 5:39 to install OCaml+opam and 1:53 to build
the dependencies of the library. After this change, it
needs an average of 3:15 for the installation and 1:27 for
the deps.

Changed
╌╌╌╌╌╌╌

• Windows installs Cygwin to `D:\cygwin', using faster Azure temporary
storage.

<https://github.com/avsm/setup-ocaml/releases/tag/v1.1.6>

Set up OCaml 1.1.7
══════════════════

Archive: <https://discuss.ocaml.org/t/ann-set-up-ocaml-1-1-7/7279/1>

Sora Morimoto announced
───────────────────────

Changed
╌╌╌╌╌╌╌

• Ubuntu and macOS runners no longer display "No switch is currently
installed." before building the compiler.
• Ubuntu no longer installs the system ocaml packages.
• macOS no longer builds two compilers on every run.
• Upgrade opam to 2.0.8 for Linux VMs.

<https://github.com/avsm/setup-ocaml/releases/tag/v1.1.7>

Old CWN
═══════

If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].

If you also wish to receive it every week by mail, you may subscribe
[online].

[Alan Schmitt]

[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <https://alan.petitepomme.net/cwn/>

[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>

[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>

[Alan Schmitt] <https://alan.petitepomme.net/>

[-- Attachment #2: Type: text/html, Size: 32262 bytes --]