OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of July 04 to 11, 2023.

Table of Contents

opam 2.2.0~alpha release!

R. Boujbel announced

On behalf of the opam dev team, I’m happy to announce you the alpha release of opam 2.2.0. :tada:

What’s new in this alpha ?

  • 🔗 Native Windows compatibility, you can now launch opam in your favorite Windows terminal! For the moment, it needs an preexisting Cygwin installation, a limitation that will be lifted for the alpha2.
  • 🔗 Recursive pinning to have opam look for .opam files in subdirectories.
  • 🔗 Tree view of your installed packages dependencies (and reverse dependencies).
  • 🔗 Recommended development tools variable {with-dev-setup} and option --with-dev-setup. It is used the same way as with-doc and with-test. E.g. depends: [ "ocamlformat" {with-dev-setup & = "0.23.0"} ].
  • 🔗 Software Heritage fallback for archive retrieval.
  • 🔗 opam install with formulas, like for for switch creation.
  • 🔗 Several new options for pin, exec, source, clean, switch, and admin.
  • And of course numerous enhancements, fixes and updates.

You’ll find these features presented in the blog post, and for a more detailed view you can take a look at the release note or changelog.

We encourage you to try out this alpha release, instructions are detailed in the blog post, in particular for Windows.

Initial Emissions Monitoring of the OCaml.org Infrastructure

Patrick Ferris announced

The library used for accessing emission information from electricity grids called “carbon” should now be available on opam. This is an initial release and might change a bit (e.g. it depends on an alpha release of cohttp-eio).

opam update && opam install carbon

Here’s a simple example to get the emission numbers for Great Britain.

Eio_main.run @@ fun env ->
Mirage_crypto_rng_eio.run (module Mirage_crypto_rng.Fortuna) env @@ fun _ ->
Carbon.Gb.get_intensity env#net
|> Eio.traceln "%a" Carbon.Gb.Intensity.pp

Thanks to @emillon for providing a backend for French data and to @reynir for suggesting important changes to the CO2-signal backend before the release.

Autofonce, a modern runner for GNU Autotests suites

Fabrice Le Fessant announced

A recent blog article on the subject:

https://ocamlpro.com/blog/2023_03_18_autofonce/

First beta release of OCaml 5.1.0

octachron announced

After two alpha releases, the release of OCaml 5.1.0 is drawing near.

We have thus released a first beta version of OCaml 5.1.0 to help you update your softwares and libraries ahead of the release (see below for the installation instructions).

Compared to the last alpha release, this beta contains two subtle internal runtime fixes, and one Windows fix. Overall, the opam ecosystem looks in a good shape for the first beta release.

If you find any bugs, please report them on OCaml’s issue tracker.

Nearly all core development tools supports OCaml 5.1.0, and you can follow the last remaining wrinkles on the opam readiness for 5.1.0 meta-issue.

Currently, the release is planned for the end of July or the beginning of August.

If you are interested in full list of features and bug fixes of the new OCaml version, the updated change log for OCaml 5.1.0 is available on GitHub.

Installation Instructions

The base compiler can be installed as an opam switch with the following commands on opam 2.1:

opam update
opam switch create 5.1.0~beta1

The source code for the alpha is also available at these addresses:

  • Fine-Tuned Compiler Configuration

    If you want to tweak the configuration of the compiler, you can switch to the option variant with:

    opam update
    opam switch create <switch_name> ocaml-variants.5.1.0~beta1+options <option_list>
    

    where option_list is a space-separated list of ocaml-option-* packages. For instance, for a flambda and no-flat-float-array switch:

    opam switch create 5.1.0~beta1+flambda+nffa ocaml-variants.5.1.0~beta1+options ocaml-option-flambda
    ocaml-option-no-flat-float-array
    

    All available options can be listed with opam search ocaml-option.

Changes since the last alpha release:

  • Runtime System Bugfix
    • #12037: Fix some data races by using volatile when necessary (Fabrice Buoro and Olivier Nicole, review by Guillaume Munch-Maccagnoni, Gabriel Scherer and Luc Maranget)
    • #12253, #12342: Fix infinite loop in signal handling. (Guillaume Munch-Maccagnoni, report by Thomas Leonard, review by KC Sivaramakrishnan and Sadiq Jaffer)
  • Windows Bugfix
    • #12184, #12320: Sys.rename Windows fixes on directory corner cases. (Jan Midtgaard, review by Anil Madhavapeddy)

octachron added

After spending some time trying to map the current package build failures with the current beta release on the opam repository, I have written down a short topography of such build failures at http://gallium.inria.fr/blog/florian-compiler-weekly-2023-07-05 .

Dream-html - DSL to build HTML, integrated with Dream

Yawar Amin announced

Just released 0.1.0 with a couple of small helpers:

  • Hx.__ can be used to render the _ attribute, which is used for Hyperscript
  • attr "attr-name" can be used to create a new attribute which does not carry a payload (what is usually called a boolean attribute)

ppx_subliner 0.2.0

Boning announced

I am very happy to announce the second release (v0.2.0) of ppx_subliner ! The new version is available through OPAM.

The extension had been greatly enhanced since the first release.

Now, the user is able to generate a multi-subcommand application from a single variant type with inline records. And the new version does not need to rely on other extensions.

type params = Foo of { my_arg : string } | Bar
[@@deriving subliner]

let handle = function
  | Foo { my_arg } -> print_endline ("Foo " ^ my_arg)
  | Bar -> print_endline "Bar"

[%%subliner.cmds
eval.params <- handle]
(** Some docs *)

The user is also able to generate Cmdliner.Term.t from a record type, and the signature is fully compatible with the generated value of ppx_deriving_cmdliner . The deriver support various attributes, such as [@pos], [@pos_all], [@pos_left], [@non_empty], [@conv], [@term], [@file], to support different kinds of positional arguments and types, and give the user great flexibility.

type foo = { my_arg : string } [@@deriving subliner]

This release should cover majority of Cmdliner. Please see the documentation for the full feature set.

Happy hacking!

happy-eyeballs, dns, http-{lwt,mirage}-client ++ writeup of happy-eyeballs issue and fix

Hannes Mehnert announced

Dear OCaml people,

some time ago we encountered issues in the connection establishment of http-lwt-client in a GitHub action (using a pre-commit.com) hook. It was quite some journey from surfacing issue “HTTP connection timeout” to the actual issue, so we wrote it up at https://semgrep.dev/blog/2023/http-request-failed-timeout-issue-in-ocaml – it reminds me of an issue surfacing as “bad record mac” in the TLS stack some years ago (see https://hannes.robur.coop/Posts/BadRecordMac).

This lead to the releases of happy-eyeballs{,-lwt,-mirage} 0.6.0 and dns* 7.0.3. In related news, we just release http-lwt-client 0.2.5 with further fixes (related to HTTP2) and http-mirage-client 0.0.5.

This debugging and fixing work was sponsored by semgrep.com – a great open source utility to statically analyse your codebase :D

All the best, and please report issues and feedback either here, via eMail, or in the issue tracker(s). Enjoy your day!

Getting started with ReasonReact and Melange

psb announced

A short blog post on getting started with ReasonReact and Melange: https://dev.to/psb/getting-started-with-reasonreact-and-melange-13hd

Making OCaml 5 Succeed for Developers and Organisations

KC Sivaramakrishnan announced, spawning a big thread

I wrote up a summary of the various efforts at Tarides aimed at making OCaml 5 succeed for developers: https://tarides.com/blog/2023-07-07-making-ocaml-5-succeed-for-developers-and-organisations/.

The post gives an idea of how we’ve been approaching this challenge. It would be great to hear what the community thinks about our approach and whether we are missing something impactful that we should be focussing on.

Also, I hope you give OCaml 5 a try if you haven’t already :-)

Llama, an OCaml library for declaratively building software-defined modular synthesizers

Steve Sherratt announced

llama is a library for building audio synthesizers using a declarative EDSL. It contains a collection of combinators which consume and produce streams of values (usually ~float~s for audio signals and ~bool~s for control signals) which can be composed much in the same way as one would patch a modular synthesizer. The library also contains a player which can play a stream of ~float~s (treated as audio samples) through your sound card.

An additional library llama_interactive can be used to connect synthesizers to input events and to render oscilloscope visualizations:

8a5c6d13b4d812cb798a8e27255ff1bb0b342f91.png

Some demos (from the examples directory):

There are more demo videos linked from the readme.

Get llama with opam install llama or opam install llama_interactive. A third package llama_core contains just the core type definitions and combinators but not the player. Use this if you just want to make more effects, filters, etc without depending on additional packages needed to play audio.

Note that llama (and llama_interactive) depends on conf-rust-2021 as interacting with the sound card is done using the cpal rust library. If you don’t want to install rust system-wide (e.g. because you use rustup) then run opam install conf-rust-2021 --assume-depexts before installing llama.

Code Example

This will play repeating pulses of a 440Hz sine wave.

open Llama
open Dsl

(* [osc] represents a signal whose value varies between -1 and 1 according
   to a 440Hz sine wave. *)
let osc : float Signal.t = oscillator (const Sine) (const 440.0)

(* [note_clock] represents a signal whose value is either [true] or [false]
   which changes from [false] to [true] twice per second, and spends 30% of the
   time on. This is often used to communicate the fact that a key is pressed to
   a module that responds to such events. *)
let note_clock : bool Signal.t =
  pulse ~frequency_hz:(const 2.0) ~duty_01:(const 0.3)

(* [envelope] is a signal which is 0 while its [gate] argument is producing
   [false] values, but which raises to 1 over the course of [attack_s] seconds
   when [gate] transitions to [true], and transitions back to [false] when
   [gate] transitions to [false]. Note that even though it is also a [float
   Signal.t] like [osc] is, it doesn't contain audio data. Instead an envelope
   is typically used to modulate a signal in response to a key press, which we
   are simulating here with [note_clock]. *)
let envelope : float Signal.t =
  asr_linear ~gate:note_clock ~attack_s:(const 0.01) ~release_s:(const 0.2)

(* Multiply the oscillator with the envelope to produce a repeating
   burst of volume which gradually tapers off twice per second. *)
let output : float Signal.t = osc *.. envelope

(* Play the sound! *)
let () = play_signal output

dune 3.9.0

Continuing this thread, Tim McGilchrist said and Etienne Millon announced

Thank you to the Dune team for all your fantastic work.

For anyone using macOS there is a know bug with this version https://github.com/ocaml/dune/issues/8083 that you might want to hold off until it is fixed before upgrading.

To expand on that:

In Dune 3.9.0, we added a feature that offloads some computations to background threads. Unfortunately, this has a bad interaction on macOS, where we fork processes to implement the RPC server and watch mode.

We marked Dune 3.9.0 unavailable on macOS, and released 3.9.1 with some mitigations: we don’t offload these computations on macOS, and we only fork when necessary.

The plan for the next release is to stop forking processes on macOS.

The first alpha for Dune 3.10.0 is planned to happen around 2023-07-25.

Here’s the changelog for 3.9.1:

Fixes

  • Disable background operations and threaded console on MacOS and other Unixes where we rely on fork. (#8100, #8121, fixes #8083, @rgrinberg, @emillon)
  • Initialize async IO thread lazily. (#8122, @emillon)

Depending on non-OCaml languages from the opam repository

Anil Madhavapeddy announced

I’m currently reviewing feature requests for the opam repository, and one of the most common ones is for us to support non-OCaml toolchains as dependencies from OCaml packages submitted to our package repository. In recent years, there have been a number of OCaml libraries that depend on Rust, Python or Node, and cannot be easily tested in our automated infrastructure (which currently uses a fixed base image per distribution).

I’ve put together a prototype way we might solve this easily, without taking on the burden of maintaining non-OCaml toolchains ourselves with limited maintainer resources. Opinions and ideas welcome on this thread, and the repository is at: https://github.com/avsm/opam-lang-repo

A multi-language devcontainer package repository

Devcontainers are an emerging mechanism to use container runtimes as a full-fledged development environment. They can support multiple programming languages in one filesystem by means of features, which allow for the activation of a given toolchain alongside others. For example, using features allows for the simultaneous use of Python, Rust and OCaml within one container image, whereas with traditional devcontainers there would be a separate container for each toolchain.

Using the opam solver to manage feature selection

The opam package manager integrates a builtin constraint solver that allows for the selection of a compatible set of dependencies from a package repository that contains all released versions of all packages.

This repository translates published devcontainers into opam packages, such that devcontainer features can be selected by simply adding dependencies to an opam package. Additionally, version constraints on the desired tooling can be added to pick the required versions. For example:

depends: [ "dev-rust" {>="1.68"}
           "dev-ocaml" {>="4.12" & < "5.0"}
           "dev-python"
           "dev-python-optimize" ]

This picks a version of Rust greater than 1.68, and any OCaml compiler between 4.12-4.14, and any Python compiler with the optimize flag activated for more efficient code generation.

The Good News

This solution frees the opam-repo maintainers from having to support the myriad other toolchains, and lets us depend on them freely from opam. By adding explicit dependencies like this, we can continue to run automated end-to-end tests for new and existing packages in the OCaml ecosystem, even when they do not exclusively use OCaml.

The Bad News

There are still some limitations to figure out before this is production worthy:

  • The devcontainer installation busts the opam security sandbox, and so cannot be installed simultaneously with normal packages. It would be ok in a CI system where sandboxing is normally disabled. Another option is for these packages to not actually perform the installation, but generate a single install.sh with all the right environment variables. An image generator could then run that script to generate a base image.
  • opam doesn’t currently support composing remote repositories, so some strategy is needed for how to keep this generated repo in sync with anything included in the central repository.
  • Need to support devcontainer boolean defaults correctly (e.g. Python feature), and figure out what to do about arbitrary string options. Env variables could be used to pass in values, but opam can’t recompile if these variables change. Dune does support systematic env variable tracking and recompile if it changes, so this would work in a monorepo.
  • Need to extract feature dependencies into the opam formula as well.
  • Something, something, Nix, instead?

Status of DkML Windows

jbeckford announced

I’m preparing to release the next semi-annual version of DkML (the MSVC/Windows distribution for OCaml). Among other things new in this version, it has been completely decoupled from the deprecated fdopen repository (thanks @fdopen for doing it for so long!). And from my perspective Windows looks good for OCaml going forward:

  • opam 2.2 will be able to stand up a Cygwin / OCaml compiler from the main opam repository
  • opam CI + repository are getting mechanisms so that the number of safe-for-Windows packages should grow over time
  • OCaml 5 will be getting full MinGW (GCC) and then MSVC support
  • Many many packages have accepted MSVC patches

There is one structural problem with DkML … it is tightly coupled to my company (Diskuv) and that is not healthy. In particular, seemingly minor decisions on my part (ex. sticking with OCaml 4 for the foreseeable future) have a large blast radius on the OCaml community.

Are there any Windows users who want to use MSVC + OCaml 5 on a daily basis, and who are also willing to maintain the OCaml 5 part of the distribution? The DkML distribution will still be actively developed (ie. the repository, installer, ease-of-use shims, hosting, testing, and release system), so your maintenance responsibilities should not be excessive. If so, please contact me in the next few days so you can see what goes into the release process.

Old CWN

If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.