OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of September 05 to 12, 2023.

Table of Contents

Miou, a simple scheduler for OCaml 5

Archive: https://discuss.ocaml.org/t/ann-miou-a-simple-scheduler-for-ocaml-5/12963/1

Calascibetta Romain announced

I’m pleased to announce the experimental release of Miou (0.0.1~beta1): a round-robing scheduler for OCaml 5. Miou is a small library that focuses on implementing system & network applications.

You can now find the project on GitHub, Robur’s repository, as well as its documentation and a few tutorials. This release was marked by the implementation of a happy-eyeballs/dns [1] client combining concurrency and parallelism. The aim is to be able to continue implementing services using this library and to guide the user/developer towards good system practices.

The project is still in an iteration phase with the implementation of services such as dns and email in order to validate our design. This release invites you to iterate with us to benefit from everyone’s experience.

We will complete this thread with articles specifying the implementation of Miou. The documentation gives a good overview of our objectives and the Miou framework. If you are interested, we invite you to read it.

We would like to thank everyone who has been involved in any way in the project for their experience and their contributions. And we hope to consider other experiences and feedback in order to develop Miou in the same way as Robur has already done for the other projects.

[1]: Implementing happy-eyeballs via the excellent ocaml-dns project has enabled us to:

  1. check that it is possible to launch a background task managing connections without being able to detach a task
  2. satisfy the notions of socket ownership
  3. implement a real application using one of the protocols we maintain

Eio Digest #1 (September 2023)

Archive: https://discuss.ocaml.org/t/eio-digest-1-september-2023/12968/1

Sudha Parimala announced

As we’re bracing towards an update to Eio 1.0 after ICFP (famous last words, just kidding :D), the team is happy to present updates on Eio and friends. We hope to carry this out on monthly or bi-monthly basis, in line with the precedent set by Multicore Monthlies, the Platform Newsletter and others.

For the uninitiated, Eio is a direct-style concurrency library for OCaml 5 and beyond. Eio leverages effect handlers for non-blocking IO, and at the same time offers advantages such as enhanced support for backtraces and being able to write concurrent code in direct-style.

Additionally, Eio offers fast native backends for various operating systems when available, such as io_uring for Linux. Eio is conceived to be a replacement for existing monadic concurrency libraries such as Lwt.

Eio 0.12 was released not long ago. The pending tasks for Eio 1.0 are being tracked at Eio#338.

Before going on with detailed updates, I’d like to take a moment to thank all contributors and users for spending their time on Eio. Without their collective efforts, there is no way we could be as far in the development of Eio as we are. If you’re interested in contributing or testing we’d be more than happy to welcome you to the community!

Updates

Eio has undergone significant developments and improvements in the past few months. Notably, it replaces the OCaml object system with polymorphic variants to represent OS resources, thereby making code more accessible. Eio also boasts various backends, including a fast native IO backend for Linux’s io_uring and a generic POSIX backend, while also exploring possibilities for macOS and Windows support.

Moreover, Eio is making strides in JavaScript interoperability, aiming to target browser applications. The latest release introduces a resource pool, akin to Lwt_pool, and anticipates support for custom events with the meio CLI tool. Quite a few applications have started migrating to Eio, including Irmin, Wayland, ocaml-grpc, OCaml CI’s solver service and more, expanding Eio’s utility across diverse domains.

  • Replacing objects with variants

    PR: https://github.com/ocaml-multicore/eio/pull/553

    Originally, Eio’s capabilities were encoded using the OCaml object system. However, as objects in OCaml are not a widely used feature, there was concern from the community that reliance on the object system would hinder wider adoption of Eio. As a result of this feedback, Eio now uses polymorphic variants to serve the same function as objects did previously.

    Polymorphic variants are better understood by most OCaml practitioners and we hope that this change will enable more users to adopt Eio. Very little code is expected to break as a result, and this change will be made available in other libraries built with Eio soon.

  • Backends

    One of the goals of Eio is to provide fast native IO backends for major operating systems. It is for this reason that we started with a backend for io_uring, Linux’s state-of-the-art, high-performance I/O API. We eventually added a libuv backend (through Luv), which allowed portability across multiple operating systems, vis-à-vis Linux, MacOS, and Windows. Unfortunatley it also caused several problems, and we ended up replacing it with eio_posix.

    Eio_posix is a generic POSIX backend, using ocaml-iomux. The Windows backend is functional (pun intended), with more improvements being added. At some point, we may rethink a Grand Central Dispatch backend and IO Completion ports backend for MacOS and Windows respectively.

  • Javascript Interop

    PR: https://github.com/ocaml-multicore/eio/pull/405

    Having talked about backends for major operating systems, one cannot ignore browsers, which are getting increasingly common. We have a WIP browser backend for enabling Eio to target Javascript (and Jsoo) applications. This is a promising avenue that opens up possibilities for Lwt based libraries that use Jsoo to migrate to Eio.

    Furthermore, eiojsoo shows some interesting examples of Eio

    • Js_of_ocaml in action.
  • Pool and Lazy

    The latest release of Eio provides a pool. This is similar to Lwt_pool, helpful for managing a collection of resources. For instance, it’s possible that maintaining a pool of expensive resources such as domains or database connections is more memory and resource efficient than the alternative.

    The latest release also includes a fiber-safe lazy module (Eio.Lazy) that allows lazy values to be forced concurrently from multiple fibers.

  • Custom Events (meio)

    OCaml 5.1 will have support for user-defined custom events, extending the ring-buffer based event tracing system. meio is a tokio-console style CLI tool for real time monitoring of Eio programs. The hope is that it will make debugging Eio programs a more pleasant experience. The tool is currently experimental. We hope to have an initial release scheduled for after OCaml 5.1 is out.

  • Splitting Eio Core

    Issue: https://github.com/ocaml-multicore/eio/issues/544

    Something we discussed in our developer meeting is potentially splitting out the core Fiber as a separate library. A potential use-case of this might be formally verifying Eio Fibers. While no decisions have been made on this yet, we’re interested to know if this could be useful for anything not mentioned above.

  • Applications

    Several applications have started harnessing the power of Eio. Below are some we know of:

    Irmin

    In a first for the library, Irmin has a shiny new Eio backend. Irmin, the distributed database (which is used as storage system for Tezos, amongst others), has started migrating to Eio with the eventual goal of supporting multicore Irmin. Stay tuned to hear more about this from the Irmin team.

    Wayland

    Eio 0.12 added support for SCM_RIGHTS. This allowed conversion of the ocaml-wayland library to Eio. That, in turn, allowed an Eio port of wayland-proxy-virtwl. These ports should be merged and released soon.

    OCaml-grpc

    Repository: https://github.com/dialohq/ocaml-grpc

    ocaml-grpc now includes an Eio mode. @quernd reported that it has been deployed in their production systems and is working quite well.

    Solver Service

    OCaml CI’s solver service is now powered by Eio. This has also made the solver service multicore. The Lwt version is still active because we are still debugging some performance issues with the Eio version.

    Eio-solo5

    Available at: https://github.com/TheLortex/eio-solo5

    We’re committed to preserving Eio’s compatibility with the larger Mirage ecosystem. With that in mind, eio-solo5 is a small experiment that demonstrates Dream running over Eio and Mirage libraries.

    Nixpackages

    PR: https://github.com/NixOS/nixpkgs/pull/230270

    Thanks to @toastal, Eio and sub-packages are now available at Nix packages.

  • Call to Action

    Windows

    The good news is that we have a functional Windows backend for Eio (shout-out to @patricoferris). Not so good news is that we haven’t done an awful lot of stress testing on it. If you are running Eio on Windows, we’re eager to hear what works, and more importantly what doesn’t. We intend for the Windows backend to be a first class citizen, in alignment with the overarching objective of upholding Windows as a primary platform within the OCaml ecosystem. That means resolving bugs and improving workflows, so please let us know about your experience using Eio on Windows.

  • Learn More

    Our README offers a comprehensive getting-started guide for Eio. For the more curious readers wanting additional information:

    • Lwt to Eio tutorial at ICFP 2023: Thomas Leonard (@talex5) and Jon Ludlam (@jonludlam) are running a tutorial at ICFP on migrating Lwt applications to Eio. Materials are available here.
    • Awesome multicore OCaml: awesome-multicore-ocaml is a collection of all resources related to Multicore and effect handlers.

Help revamping the getting started tutorials in ocaml.org

Archive: https://discuss.ocaml.org/t/help-revamping-the-getting-started-tutorials-in-ocaml-org/12749/4

Cuihtlauac Alvarado announced

The pull request corresponding to these tutorials has been updated with the feedback we’ve received.

We are considering publishing soon but are still accepting feedback.

pcre2-ocaml.7.5.1 released

Archive: https://discuss.ocaml.org/t/ann-pcre2-ocaml-7-5-1-released/12972/1

Chet Murthy announced

Hi, I’ve just released pcre2-ocaml, a wrapper around the Debian package pcre2 (for PCRE new version #2, PCRE2). pcre-ocaml, the long-standing version by Markus Mottl will become obsolete b/c Debian is removing support for the underlying Debian package pcre3 (PCRE old version #1, PCRE1).

[yes, this is confusing.]

This is a port by @tobil4sk (don’t know their ID on this forum): I’m (for now) just maintaining it.

https://github.com/camlp5/pcre2-ocaml

It should be available in opam by now.

I’ve already found a bug (courtesy of @viritrilbia ) and have a fix with the beginning of unit-tests ready to release.

If anybody has any interest in joining in to maintain this package, please do contact me.

Third (and hopefully last) release candidate for OCaml 5.1.0

Archive: https://discuss.ocaml.org/t/third-and-hopefully-last-release-candidate-for-ocaml-5-1-0/12975/1

octachron announced

Since last week, there were two significant bugs fixed in the OCaml 5.1.0 runtime (one overflow bug, and a stack corruption bug in the s390x port). Since those bug fixes are as small as they are subtle, they were deemed worthy of a release of a third release candidate for OCaml 5.1.0.

If there are no more surprises this week, the release of OCaml 5.1.0 shall happen next week.

If you find any bugs, please report them on OCaml’s issue tracker.

The full changelog for OCaml 5.1.0 is available on GitHub

A short summary of the two fixed bugs in this release candidate is also available below.

Installation Instructions

The base compiler can be installed as an opam switch with the following commands on opam 2.1 and later: 

opam update
opam switch create 5.1.0~rc3

The source code for the release candidate is also directly available on:

  • Fine-Tuned Compiler Configuration

    If you want to tweak the configuration of the compiler, you can switch to the option variant with: 

    opam update
opam switch create <switch_name> ocaml-variants.5.1.0~rc3+options <option_list>

    where <option_list> is a comma-separated list of ocaml-option-* packages. For instance, for a flambda and no-flat-float-array switch: 

    opam switch create 5.1.0~rc3+flambda+nffa ocaml-variants.5.1.0~rc3+options ocaml-option-flambda
ocaml-option-no-flat-float-array

    All available options can be listed with opam search ocaml-option.

Last Second Bug Fixes

  • #11284, +#12525: Use compression of entries scheme when pruning mark stack. Can decrease memory usage for some workloads, otherwise should be unobservable. (Tom Kelly, review by Sabine Schmaltz, Sadiq Jaffer and Damien Doligez)
  • #12486: Fix delivery of unhandled effect exceptions on s390x (Miod Vallat, report by Jan Midtgaard, review by Vincent Laviron and Xavier Leroy)

Ocaml.org tutorial revamping, cont’d - Basic Datatypes

Archive: https://discuss.ocaml.org/t/ocaml-org-tutorial-revamping-contd-basic-datatypes/12985/1

Cuihtlauac Alvarado announced

Update of the ocaml.org tutorials continues.

We’d like to have your opinion on this new one:

It addresses datatypes basic concepts. It covers the most common predefined datatypes, variants, and records.

Mutability and references are not covered, and polymorphic variants are neither (we’re planning to work on these as stand-alone tutorials).

We love feedback, we need it.

Printing, modular implicits and the Stdlib

Archive: https://discuss.ocaml.org/t/printing-modular-implicits-and-the-stdlib/13002/1

Emile Trotignon announced

I wanted to share thoughts I had on the thorny “printing question” with the community. I have written a small blog post here, and I would enjoy discussing the questions raised here.

Ocaml.org tutorial revamping, con’d - Values and Functions

Archive: https://discuss.ocaml.org/t/ocaml-org-tutorial-revamping-cond-values-and-functions/13005/1

Cuihtlauac Alvarado announced

Here is a new episode in the series of updates of the ocaml.org tutorials

It addresses values, functions, environment, scope, closures, shadowing and friends.

As for the previous one, we’d love to know what you think about it.

Old CWN

If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe online.