OCaml Weekly News

Examples using make:

Examples using dune:

Examples using Opam and something else:

Reference

Release-wide changes

• Switch to OCaml 4.14
  

  We are switching Base and all our packages, except sexplib0, to OCaml 4.14 and above.

  Previously, Base supported the four latest OCaml versions. The prime reason for it was to ensure that a single ppxlib release would work across multiple compilers. However, ppxlib now depends solely on sexplib0, not Base.

  Meanwhile, we are making a considerable effort to integrate the latest OCaml features. We’ve already added features like the In_channel module from the 4.14 standard library to Base, and we’re working on switching to the 5.0 compiler internally.

  As a result, we decided to remove the strict requirement for Base’s backward compatibility. Instead, we hope to provide a more comprehensive Stdlib replacement and better testing for recent compiler versions. We plan to eventually add support for OCaml 5 primitives in Base, starting with releases v0.17 or v0.18.

• 32-bit platform support
  

  As stated previously, we are no longer testing on 32-bit platforms. That said, we expect that our packages will mostly continue to function on 32-bit, and we strive to maintain compatibility. In particular, we will keep responding to bug reports and accepting 32-bit compatibility patches. We’re also hoping that Tarides’ work in this area will improve the testing of our packages across many platforms, including 32-bit bytecode.

• Deprecation of Caml in favor of Stdlib
  

  We are deprecating our Caml alias for the OCaml standard library, in favor of using the default module, Stdlib. This allows writing code that references the standard library regardless of whether Base is opened or not.

• Local allocations
  

  We’ve included annotations for local allocations in our public release code; however, these annotations are not recognized by the stock OCaml compiler. More information on local allocations can be found in our blog post at https://blog.janestreet.com/oxidizing-ocaml-locality/.

  For those interested in experimenting with local allocations, our OCaml fork with local allocation support can be found here. Please be aware that this is highly experimental and not guaranteed to function outside of Jane Street.

Future plans

• Shadowing Stdlib
  

  We plan to stop shadowing Stdlib in Base. We also plan to add a flag to Dune to enable -nopervasives during compilation, which prevents Stdlib from being opened automatically. The previous practice of shadowing Stdlib in Base has caused issues, and we hope this new approach will prove more resilient.

• OCaml extensions
  

  We intend to create an experimental Opam repository with our internal compiler for a more straightforward setup. This will allow users to experiment with our new language extensions before they are integrated into mainstream OCaml.

  At present, we convert our native syntax for OCaml extensions into annotations. However, once we transition to the experimental repo, we will release the unmodified code featuring the preferred local_ syntax.

Many, many other changes

New packages

2023/06/08

Releases

Building Packages

• Dune Exploring Package Management in Dune
  

  Contributors: @rgrinberg (Tarides), @Leonidas-from-XIV (Tarides), @gridbugs (Tarides), @kit-ty-kate (Tarides)

  Explorations continue on adding package management support to Dune. This month progress has been made on several fronts:

  • The work on the solver has been started, including vendoring the opam-0install solver for solving dependencies when generating Dune lockfiles. A work-in-progress implementation of lockfile generation is available on the main branch.
  • The source tree handling has undergone a refactor to eventually allow multiple context-specific lockfiles.
  • The source fetching implementation has seen improvements, including checksum handling and a better Fetch API. This results in a cleaner interface for building opam packages.
  • Work continues on prototyping the building of opam packages, which includes the addition of new Patch and Substitute actions. This has increased the subset of opam packages that can now be built.

  Activities:

  • Merged the PR that added the ability to build opam packages – ocaml/dune#7626.
  • Added safety mechanisms in lock directory regeneration – ocaml/dune#7832.
  • Introduced feature to set environment in build rules – ocaml/dune#7742.
  • Merge the PR that added conservative lockfile generation – ocaml/dune#7732.
  • Simplified entries in cookie – ocaml/dune#7701.
  • Fixed location handling for source copies – ocaml/dune#7697.
  • Improved checksum handling – ocaml/dune#7696.
  • Tested install action – ocaml/dune#7695.
  • Versioned lock directory format – ocaml/dune#7693.
  • Created a better API for fetch – ocaml/dune#7675.
  • Vendored opam-0install – ocaml/dune#7668.
  • Open a PR adding a feature to return the retrieved checksums on failure for checksum verification – ocaml/dune#5552.
  • Carried out a refactor to allow passing in a custom runner to OpamStd.Sys – ocaml/dune#5549.
• opam Native Support for Windows in opam 2.2
  

  Contributors: @rjbou (OCamlPro), @kit-ty-kate (Tarides), @dra27 (Tarides), @emillon (Tarides), @Leonidas-from-XIV (Tarides)

  Throughout May, the opam team has focused on reviewing and fixing remaining PRs for the 2.2 alpha release. This effort debugged and resolved issues found during testing, and the team is now working through the handful PRs pending review.

  A new release, opam 2.1.5 point release, has also been rolled out, backporting several fixes from the upcoming 2.2 release, and an important security fix. You can read the announcement on the OCaml Changelog.

  Activities:

  • Installed cygwin internally during init – ocaml/opam#5545.
  • Added cygwin support to depexts – ocaml/opam#5542.
  • Implemented fully revertible environment updates – ocaml/opam#5417.
  • Some windows shell updates – ocaml/opam#5541.
  • Better cygwin support in core – ocaml/opam#5543.
  • init: detect local cygwin installation – ocaml/opam#5544.
  • init: install cygwin internally - ocaml/opam#5545.
  • Used OCaml code to copy/move/remove directories instead of unix commands – ocaml/opam#4823.
  • Fix performance regression in opam install/remove/upgrade/reinstall – ocaml/opam#5503.
• Dune Improving Dune’s Documentation
  

  Contributors: @emillon (Tarides)

  Two Dune libraries now have a documentation page on OCaml.org: both dune-build-info and dune-configurator now have their API documentation directly on their package page, and for Dune itself, a link to its official documentation has been included.

  A new action: directive has been added to the Dune Sphinx domain, allowing for improved cross-referencing. Finally, the old .org format Dune example docs have been converted to Markdown to standardize the documentation format further.

  The improvements to the documentation have been published as part of the Dune 3.8 release. The new structure and many improvements can be viewed on Dune documentation.

  Activities:

  • Placed uncategorized pages under the most appropriate header – ocaml/dune#7683.
  • Converted README.org in example to markdown – ocaml/dune#7738.
  • Added API documentation for dune-build-info – ocaml/dune#7739.
  • Directed readers to the official docs in odoc – ocaml/dune#7746.
  • Added an odoc index for configurator – ocaml/dune#7749.
  • Corrected the documentation for (map_workspace_root) – ocaml/dune#7775.
  • Expanded actions documentation with a special directive – ocaml/dune#7804.
• Dune Composing installed Coq theories
  

  Contributors: @Alizter and @ejgallego (IRIF)

  Last month, the PR that brings support for composing Coq theories with Dune was merged.

  This is now available in the release of Dune 3.8.0! From this point onwards, Coq users can utilize Dune to build Coq projects even if they depend on Coq projects that use other build systems!

• Dune Running Actions Concurrently
  

  Contributors: @Alizter and @hhugo (Nomadic Labs)

  Last month, a couple of PRs were merged into Dune to add a new concurrent action and it was utilized to run inline tests concurrently.

  These patches are part of Dune 3.8.0 release, starting now, you can use the new concurrent action in your Dune rules:

  (rule
   (action
    (concurrent
     (run <prog> <args>)
     (run <prog> <args>))))
  

• Dune Benchmarking Dune on Large Code Bases
  

  Contributors: @gridbugs (Tarides), @Leonidas-from-XIV (Tarides)

  The quality of the dune benchmark results has been improved, averaging out the variance seen in short tests by running them multiple times. This enhancement aims to reduce the effect of the background noise inherent in the environment.

  The dune benchmarks also exposed two broken packages: the hash of ppx_rapper (3.1.0) artefact has been updated as it had changed in place, and ocamlcodoc artefacts have been added to the opam-source-archives as the original URL is no longer reachable.

  Moreover, a stack overflow in dune-rpc-lwt exposed by the benchmarks has been fixed.

  Activities:

  • Ran short monorepo benchmarks multiple times – ocaml/dune#7798.
  • Fixed issues in monorepo benchmarks – ocaml/dune#7786.
  • Added ocamlcodoc to opam-source-archives and updated opam-repository.
  • Opened an issue about ppx_rapper.3.1.0 package changing in place.

Compiling to JavaScript

• Dune Compile to JavaScript with Melange in Dune
  

  Contributors: @anmonteiro, @jchavarri (Ahrefs), @rgrinberg (Tarides)

  We’re thrilled to see the joint release of Dune 3.8.0 and Melange 1.0 this month!

  Melange is a compiler from OCaml to JavaScript with the vision of maintaining compatibility with OCaml and providing the best OCaml experience within the modern JavaScript ecosystem.

  Have a look at the Dune documentation and Melange documentation to learn how to get started using Melange to compile your OCaml projects to JavaScript.

  You can also refer to the template to get started.

Generating Documentation

• Odoc Add Search Capabilities to odoc
  

  Contributors: @panglesd (Tarides), @EmileTrotignon (Tarides), @trefis (Tarides)

  The odoc team is making steady progress on adding a search bar to odoc’s generated documentation.

  In May, an interface for interaction between odoc and search engines was designed. Sherlodoc, which can now run in the browser, was updated to use the new interface exposed by odoc. Improvements were also made to sherlodoc itself to enable searching for constructors and record fields, as well as in docstrings. The larger database from all this extra indexing prompted work on profiling it and implementing optimizations.

  Anticipating the June update, the odoc PR that was open a few days ago can be checked out.

  In parallel, a working prototype of counting occurrences in odoc was developed. The aim is to add usage statistics in the generated index, so that search engines can use it to sort search results. It will also allow for a “jump-to-documentation” feature in the rendered source code that was merged in March.

• Odoc Support for Tables in odoc
  

  Contributors: @gpetiot (Tarides), @panglesd (Tarides), @Julow (Tarides), @jonludlam (Tarides), @trefis (Tarides)

  The PRs adding support for a new syntax to create tables in Odoc have been merged!

  As a reminder, this new feature will enable the creation of tables using a syntax similar to Markdown:

  {t
    a  | b | c | d
    ---|:--|--:|:-:
    a  | b | c | d
  }
  

  Odoc will generate tables for different backends, including LaTex and HTML. This new syntax will be available in the upcoming release of Odoc 2.3.0. Stay tuned!

  Activities:

  • Merged the PR that adds a new syntax for tables in odoc-parser – ocaml-doc/odoc-parser#11
  • Merged the PR that adds support for tables to odoc – ocaml/odoc#893

Editing and Refactoring Code

• Merlin Support for Project-Wide References in Merlin
  

  Contributors: @voodoos (Tarides), @let-def (Tarides)

  Not much progress was made on the support for project-wide occurrences in Merlin this month due to the Merlin team’s focus on performance improvements, fixing user-reported bugs, compatibility with OCaml 5.1, and the release of Merlin 4.9

  The remaining issues, such as module aliases traversal and index filtering, were discussed, with the implementation of identified solutions set to commence soon.

• Merlin Improving Merlin’s Performance
  

  Contributed by: @pitag-ha (Tarides), @3Rafal (Tarides), @voodoos (Tarides), @let-def (Tarides)

  The final stages of work on benchmarking Merlin are in progress, with the Merlin team focusing on integrating the developed benchmarking tooling into Merlin’s CI using current-bench. The PR on merl-an has been opened to add a new current-bench compatible backend and work has started on adding memory usage information to Merlin’s telemetry.

  On the performance optimisations front, the PPX phase cache in Merlin was completed and merged. Issues were opened upstream on how tools that use Merlin can benefit from the new PPX cache.

  Activities:

  • Implemented a sketch of current-bench backend in merl-an – pitag-ha/merl-an#2.
  • Started work on adding memory usage information to Merlin’s telemetry.
  • Completed and merged the PPX phase cache PR – ocaml/merlin#1584. - Opened issues at dune and ocaml-lsp-server to keep record of the necessities to enable the PPX phase cache
    • In Dune: dune ocaml-merlin: enable PPX phase cache – ocaml/dune#7731.
    • In OCaml LSP: Handle Merlin’s PPX phase cache – ocaml/ocaml-lsp#1095.
  • Identified and fixed a memory “leak” related to the (F A).t syntax, causing uncontrolled memoization table growth – ocaml/merlin#1609.
• OCaml LSP Using Dune RPC on Windows
  

  Contributors: @nojb (LexiFi)

  In May, a couple of patches that build on the support for watch mode on Windows introduced in Dune 3.7.0 were merged in Dune and OCaml LSP to allow OCaml LSP to use Dune RPC. The aim is to enable Windows users to leverage Dune RPC and receive build statuses and more exhaustive build errors in the editor when Dune is running in watch mode.

  Dune 3.8.0 was released with the above patches and a release of OCaml LSP will follow in the coming weeks.

• OCaml LSP Upstreaming OCaml LSP’s Fork of Merlin
  

  Contributors: @voodoos (Tarides), @3Rafal (Tarides)

  The effort to upstream OCaml LSP’s fork of merlin continued. Necessary patches in Merlin have been reviewed and merged. Work also continued on the PR to use Merlin as a library in OCaml LSP.

  Activities:

  • Reviewed and merged the PR enabling configurable Merlin PP/PPX spawning – ocaml/merlin#1585.
  • Continued working on the PR to use Merlin as a library in LSP – ocaml/ocaml-lsp#1070.

Formatting Code

• OCamlFormat Closing the Gap Between OCamlFormat and ocp-indent
  

  Contributors: @gpetiot (Tarides) and @EmileTrotignon (Tarides), @Julow (Tarides), @ceastlund (Jane Street)

  The effort continued to tune the janestreet profile so that it aligns better with the output of ocp-indent. Despite encountering difficulties with formatting “cinaps” comments, considerable progress was made in May and work will continue throughout June.

  Activities:

  • Align pattern alias – ocaml-ppx/ocamlformat#2359
  • Dock fun~/~function only if it starts on the first line of the apply – ocaml-ppx/ocamlformat#2362
  • Align module arguments – ocaml-ppx/ocamlformat#2363
  • Remove extra newline in empty comments – ocaml-ppx/ocamlformat#2365