OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of October 24 to 31, 2023.

Table of Contents

asai 0.2.0 for compiler diagnostics

Archive: https://discuss.ocaml.org/t/ann-asai-0-2-0-for-compiler-diagnostics/13305/1

Favonia announced

I am happy to announce asai 0.2.0 for generating and printing compiler diagnostics. This is a major release: there are many improvements and (unfortunately) many backward-incompatible changes. Compared to the previous release, this version is significantly more stable, though we might still make big changes before 1.0.0. We have a đź”° quickstart tutorial for you to try out and would love to hear your feedback.

The most significant change is the introduction of structured reporters. The library can thus be used in two different modes:

  1. Use Reporter (as the old Logger): its API is biased towards free-form explanations.
  2. Use StructuredReporter: its API is biased towards fully structured messages.

Both modes share the same type of diagnostics, and thus, an application using structured messages can adopt a library using free-form explanations and vice versa. Great care has been taken to make sure the two modes can work together. The current tutorial focuses on Reporter and free-form explanations because that mode is slightly easier to set up.

The second most significant change is that the TTY handler got an overhaul. The output is now more concise, informational, and beautiful. You can trivially produce an error message like this:

3c19944f0880b524ac5d675c517136eba6908575.png

In case you are wondering, we intentionally did not implement the Rust-style ASCII art due to our obsession with Unicode emojis (see our design philosophy). However, our modular design makes it is very easy to connect our message reporting interface with other rendering libraries, for example the one tracked by this GitHub issue. (The author said we should wait a bit before they finish the renderer, and that’s why we have not implemented it yet!)

Thanks to @viritrilbia for many valuable suggestions.

Changelog

  • BREAKING CHANGES
    • Span is renamed to Range
    • Logger is renamed to Reporter and Reporter.Code is renamed to Reporter.Message
    • Range: the type position was changed to allow string (in-memory) sources
    • Diagnostic: the type diagnostic was changed and message was renamed to loctext
    • The LSP handler is separated out as a new package (not published yet)
  • Bug Fixes
  • Features

TSan support in OCaml 5.1 and blog post

Archive: https://discuss.ocaml.org/t/ann-tsan-support-in-ocaml-5-1-and-blog-post/13306/1

Fabrice announced

With ThreadSanitizer support to be part of OCaml 5.2, we (@otini and I) wanted to let you know about a couple of news on the subject.

If you don’t already know about it, ThreadSanitizer (also known as TSan) is a special instrumentation added to your program to detect data races. Quick pedantic reminder with the definition of a data race: :nerd_face:

A data race is when two or more threads access the same memory location concurrently, and at least one of the accesses is a write. Data races can lead to particularly hard-to-debug problems.

We are pleased to announce that we have backported TSan to OCaml 5.1, so that you can start using it right away. This version benefits from a more up-to-date compiler than the 5.0 backport, and will also perform better as many improvements have been made to the TSan integration. The impact on your program runtime under TSan will be lower. :rocket:

To get it running on your machine, it’s as simple as: 

opam update
opam switch create 5.1.0+tsan

:open_book: We had the opportunity to present TSan at the ICFP 2023 Ocaml Workshop, unfortunately the recording isn’t available yet. :face_holding_back_tears: But if you’re interested in learning more of about how TSan works, and the challenges we faced in integrating it with the OCaml compiler we’ve written an article at https://tarides.com/blog/2023-10-18-off-to-the-races-using-threadsanitizer-in-ocaml/!

:teacher: For a more hands-on experience, @jmid has also added a tutorial Transitioning to Multicore with ThreadSanitizer, which walks you through the steps of installing TSan, using it to detect data races in one example, and addressing them. It’s available at https://www.ocaml.org/docs/multicore-transition.

Happy data race hunting, and have a fun with parallelism! :ninja:

Melange 2.0

Archive: https://discuss.ocaml.org/t/ann-melange-2-0/13073/2

Antonio Nuno Monteiro announced

We have since released Melange 2.1 with the following fixes and improvements:

dream-html 1.0.0

Archive: https://discuss.ocaml.org/t/ann-dream-html-1-0-0/12787/2

Yawar Amin announced

[ANN] dream-html 1.1.0

Small update to add some minimal SVG support (PRs to add more SVG tags and attributes welcome). Also added fetchpriority attribute. E.g. 

open Dream_html

let osi_logo =
  let open SVG in
  svg [xmlns; viewbox ~min_x:0 ~min_y:0 ~width:100 ~height:100] [
    path [
      d "M34,93l11,-29a15,15 0,1,1 9,0l11,29a45,45 0,1,0 -31,0z";
      stroke "#142";
      stroke_width "2";
      fill "#4a5"] []
  ]

let unimportant =
  let open HTML in
  img [
    src "/images/in_viewport_but_not_important.svg";
    fetchpriority `low;
    alt "I'm an unimportant image!";
  ]

OCaml Platform Newsletter: September 2023

Archive: https://discuss.ocaml.org/t/ocaml-platform-newsletter-september-2023/13314/1

Thibaut Mattio announced

Welcome to the sixth edition of the OCaml Platform newsletter!

Dive into the latest updates from September and discover how the OCaml Platform is evolving. Just like in previous newsletters, it spotlights the recent developments and enhancements to the OCaml development workflows.

In addition to the updates on the Platform team’s progress highlighted below, don’t hesitate to share your feedback on the upcoming Platform roadmap. We’ve just updated it based on the most recent feedback and are aiming to adopt it in the coming weeks, barring new concerns from the community.

Happy reading!

  • Building Packages
    • [Dune] Exploring Package Management in Dune
    • [opam] Native Support for Windows in opam 2.2
    • [Dune] Dune Terminal User Interface
    • [Dune] Support on Niche Platforms
  • Generating Documentation
    • [odoc] Add Search Capabilities to odoc
    • [odoc] Syntax for Images and Assets in odoc
    • [Dune] Generate Dependencies Documentation with Dune
  • Editing and Refactoring Code
    • [Merlin] Support for Project-Wide References in Merlin
    • [Merlin] Improving Merlin’s Performance

Releases

Here are all the new versions of Platform tools that were released this month:

For detailed release notes and announcements, explore the OCaml Changelog.

Building Packages

  • [Dune] Exploring Package Management in Dune

    Contributors: @rgrinberg (Tarides), @Leonidas-from-XIV (Tarides), @gridbugs (Tarides), @kit-ty-kate (Tarides), @Alizter

    The biggest highlight from September is that the work to expose the compiler and libraries from packages installed by Dune to the rest of the project is now complete! This means that there is now a prototype of Dune package management that can be used to build projects that depend on (simple) opam packages! This is still an early prototype that’s not ready to be tested outside of the core team, but still a significant milestone: :tada:!

    In addition to this, work in September focussed on three areas:

    • Increasing coverage of opam features to support more opam packages from the opam-repository. This month, the Dune team added support for new fields, including build-env, setenv, and subst, and they also added support for patching.
    • Designing and implementing a string manipulation DSL for Dune configurations. This will allow users to express the same amount of dynamism found in opam filters in Dune package lockfiles, which is necessary for converting opam build and install commands into Dune expressions.
    • Started working on support for custom opam repositories by making the opam-repository configurable in dune-workspace. The next step is to experiment on how opam repositories are stored and accessed. One idea is that all opam repositories would be stored in one revision storage that would supply all the data. This has the advantage that incremental updates are small, which work like pulling via Git. The repo doesn’t need to be uncompressed, thus less storage and inodes used.

    Activities:

  • [opam] Native Support for Windows in opam 2.2

    Contributors: @rjbou (OCamlPro), @kit-ty-kate (Tarides), @dra27 (Tarides), @AltGr (OCamlPro)

    In preparation for the upcoming release of opam 2.2~alpha3, the work has focussed on better handling of path rewriting for the setenv and build-env opam fields.

    The proposed change will allow users to specify, in the opam file, the path separator and format they want for each environment variable in setenv~/~build-env. This ensures the environment variables are correctly set and usable on Windows.

    The PR is in review and not quite ready to be merged, but this is the last issue scoped for opam 2.2~alpha3.

    Activities:

  • [Dune] Dune Terminal User Interface

    Contributors: @Alizter, @rgrinberg (Tarides)

    Following the merge of the PR to port Dune TUI to Nottui in August, and the addition of a few features, @Alizter continued the work on building a full-on Terminal User Interface for Dune with two pull requests, namely the addition of a Jobs tab in tui mode, and support for multiline status lines.

    Activities:

  • [Dune] Support on Niche Platforms

    Contributors: @Alizter

    Dune now builds on both Haiku and Android (using Termux)! This means it is now possible to build and install both OCaml and Dune on these platforms, which should pave the way for more native OCaml development.

    For reference, here is a table of Dune’s platform support (with ? indicating that further testing is needed):

    Platform Support Watch TUI Cache Sandboxing
    Linux Full Yes Yes Yes Yes
    MacOS Full Yes Yes Yes Yes
    Windows (DKML) Full Yes No* Yes Copy only
    Windows (MinGW) Limited Yes Yes Yes Yes
    Windows (Cygwin) Limited Yes Yes Yes Yes
    Linux (Android) Limited Yes Yes ? ?
    FreeBSD Limited Yes Yes ? ?
    NetBSD Limited Yes Yes ? ?
    OpenBSD Limited ? ? ? ?
    Haiku Limited Yes Yes ? ?

    If you’re working on one of these platforms, don’t hesitate to open issues on Dune’s bug tracker if you encounter any problem!

    Activities:

Generating Documentation

  • [odoc] Add Search Capabilities to odoc

    Contributors: @panglesd (Tarides), @EmileTrotignon (Tarides), @julow (Tarides), @jonludlam (Tarides)

    Work continues on adding search capabilities to odoc in order to improve the documentation browsing experience.

    In September, the odoc team continued reviewing the different pull requests started in August. Peer-reviews suggested several improvements to to the CLI and the library API.

    They also worked on client-side performance improvements by loading the search script only when the user clicks on the search bar, and they made quite a lot of progress on the UI overall.

    Activities:

  • [odoc] Syntax for Images and Assets in odoc

    Contributors: @panglesd (Tarides)

    The effort to add support for images and assets to odoc and bring images to the OCaml.org package documentation continues!

    This month, @panglesd opened a PR with an implementation for asset references.

    The exact syntax for medias went through several designs, in particular whether a media is a block, a nestable block, or an inline element. At the end of the month, @panglesd created a PR that builds on the asset references PR in order to add support for medias.

    For some time, there has been no official convention on how documentation for opam-installed packages should be built. With the added complexity of having assets, it was a good time to solve this. A documentation PR was opened for this. Warm thank you to @dbuenzli for the thourough review and participating in establishing these conventions!

    Activities:

  • [Dune] Generate Dependencies Documentation with Dune

    Contributors: @jonludlam (Tarides)

    Currently, Dune only knows how to build the documentation for the packages in your Dune workspace, meaning that you can only read the documentation of your dependencies from the OCaml.org package site. Alternative odoc drivers, like odig, build documentation for all the packages in your switch and have been the recommended solution for users who prefer to read the dependencies’ documentation locally.

    In an effort to improve the documentation generation experience with Dune, @jonludlam worked on a new version of Dune rules to generate the documentation. With these rules, Dune will gain the additional ability to build the combination of the two: a coherent set of docs that cover both switch-installed libraries and local libraries.

    The PR is in review and is set to be merged in the coming weeks.

    Future plans for the new rules include better integration with the rest of the platform, improvements in capabilities to cover the use cases that dune build @doc covers, integration of source rendering, and integration of search (once it lands in odoc!).

    Activities:

Editing and Refactoring Code

  • [Merlin] Support for Project-Wide References in Merlin

    Contributors: @voodoos (Tarides), @trefis (Tarides), @Ekdohibs (OCamlPro), @gasche (INRIA)

    In August, the Merlin team opened the PR on the compiler that adds the necessary information in the Shapes to implement project-wide references.

    The PR received reviews, so the team worked on taking the feedback into account while also continuing work on the rest of the stack (build system rules, the indexer and new locate, and occurrences backends for Merlin).

    They also consolidated a release plan and timeline. The plan is to first release an experimental 4.14-based variant of the compiler in order to gather feedback on this eagerly awaited feature before the end of the year. The current aim is to provide official project-wide occurrences support in OCaml 5.2.

    Activities:

  • [Merlin] Improving Merlin’s Performance

    Contributed by: @pitag-ha (Tarides), @3Rafal (Tarides), @voodoos (Tarides), @let-def (Tarides)

    The Merlin team continues work on improving Merlin performance.

    Before diving into specific performance optimisation, they worked on a benchmarking CI to catch performance regressions and measure improvements. While at it, they’ve also worked on a fuzzy testing CI to catch behaviour regressions.

    In September, following the Proof of Concept (PoC) of the fuzzy-testing CI (from the work in July), the team continued their work on addressing the limitations of the current CI implementation that would prevent it from being merged in Merlin. Specifically, they focussed on finding a solution to store the fuzzy testing results in a way that wouldn’t bloat the Merlin repository. The current approach is to store the data in a separate Git repository and pull it when running the fuzzy-testing CI. They’ve created a GitHub action workflow that implements this behaviour.

    Next, the plan is to complete the work on the Merlin CI before gradually shifting focus to performance optimisations.

    Activities:

How-To: Buck 2 and OCaml - Build system

Archive: https://discuss.ocaml.org/t/how-to-buck-2-and-ocaml-build-system/13334/1

Roland Csaszar announced

I’ve just made a repo containing examples on how to use Buck 2 with OCaml using the ocaml-scripts:

OCaml-Buck-2-Examples - GitHub The example projects do also contain Dune files, so you can compare them.

All questions, suggestions and corrections are welcome, also for Meta’s OCaml scrips.

Why Buck 2 and not Bazel?

First: I have never tried Bazel (or the OCaml support), so I cannot say anything about it. I’ve chosen Buck 2 because OCaml support is included by Facebook itself, the Prelude contains all language rules of Buck 2, there are no “build in” and “external” languages as with Bazel. Buck 2 is written in Rust (which I use, so no extra tooling needed to build it), Bazel in Java. And Bazel is by Google.

But I would just look at the support of each for the languages you want to/need to/must use.

Henry Till said

Very cool.

Evidently, a couple of the creators of Buck2 gave a talk about this subject at ICFP this year.

https://icfp23.sigplan.org/details/ocaml-2023-papers/1/Buck2-for-OCaml-Users-Developers

The slide deck for the talk is available ~there~ here.

Hopefully we’ll see a video of the presentation on YouTube soon as well. :crossed_fingers:

Roland Csaszar said

Yes, this is a good idea. The official OCaml examples (which do not use Opam packages/the ocaml-scripts) which this talk is about are located in the Buck 2 repository, in the examples subdirectory OCaml examples.

Btw. Neil Mitchell is the person behind Shake and the Build Systems a la Carte paper.

First release of ortac-core, ortac-runtime and ortac-qcheck-stm

Archive: https://discuss.ocaml.org/t/ann-first-release-of-ortac-core-ortac-runtime-and-ortac-qcheck-stm/13335/1

Nicolas Osborne announced

We are very happy to announce the initial release of ortac-core, ortac-runtime~ and ortac-qcheck-stm.

Ortac is a runtime assertion checking tool for OCaml based on Gospel specification language.

ortac-core package exposes a library to translate a subset of Gospel specification language into OCaml terms. It also provides the ortac command-line tool. You will need to have one of Ortac plugins installed to use the ortac command-line tool.

ortac-runtime provides runtime environment that the translated terms depend on. In particular, it provides an implementation of the executable subset of the Gospel standard library.

ortac-qcheck-stm provides a plugin for Ortac. It generates QCheck-STM tests for a module specified with Gospel. QCheck-STM is a model-based testing framework and Ortac/QCheck-STM relies on the Gospel models you gave in the specifications to build the QCheck-STM tests. See the documentation for more details on how to write the specifications and how to uses the tool to test your libraries.

opam-publish 2.3.0

Archive: https://discuss.ocaml.org/t/ann-opam-publish-2-3-0/13337/1

Kate announced

Hi everyone,

We’re pleased to announce the release of opam-publish 2.3.0.

This release, apart from a couple of light improvements, mainly consists of the following new option:

  • You can now use opam-publish with the --no-confirmation argument for use in automated pipeline. Use this option with extreme caution if you do use it.

The full changelog is available here.

Enjoy, The opam team.

Old CWN

If you happen to miss a CWN, you can send me a message and I’ll mail it to you, or go take a look at the archive or the RSS feed of the archives.

If you also wish to receive it every week by mail, you may subscribe to the caml-list.