OCaml Weekly News

Hello

Here is the latest OCaml Weekly News, for the week of November 19 to 26, 20= 24.

OCaml 5.2.1 released

Archive: https://discuss.ocaml.org/t/ocaml-5-2-1-released/15634/1

octachron announced

We have the pleasure of announcing the release of OCaml 5.2.1, dedicated to= the memory of Niels Bohr and Paul =C3=89luard on the anniversary of their death= s.

OCaml 5.2.1 is a collection of safe but import runtime time bug fixes backp= orted from the 5.3 branch of OCaml to improve the stability of the 5.2 runtime wh= ile waiting for the upcoming release of OCaml 5.3.0.

The full list of bug fixes is available below for more details.

Happy hacking, Florian Angeletti, for the OCaml team.

Installation Instructions

The base compiler can be installed as an opam switch with the following com= mands:

opam update
opam switch create 5.2.1

The source code for the release is also directly available on:

GitHub<= /a>

Inria archive

Bug Fixes In OCaml 5.2.1 (18 November 2024)

Runtime System:
- #13207: Be = sure to reload the register caching the exception handler in caml_c_call and caml_c_call_stack_args, as its va= lue may have been changed if the OCaml stack is expanded during a callback. (Miod Vallat, report by Vesa Karvonen, review by Gabriel Scherer and Xavier Leroy)
- #13252: Rew= ork register assignment in the interpreter code on m68k on Linux, due to the %a5 register being used by GLIBC. (Miod Vallat, report by St=C3=A9phane Glondu, review by Gabriel Scherer and Xavier Leroy)
- #13268: Fix= a call to test in configure.ac that was causing errors when LDFLAGS contains several words. (St=C3=A9phane Glondu, review by Miod Vallat)
- #13234, #13267: Open runti= me events file in read-write mode on ARMel (ARMv5) systems due to atomic operations limitations on that platform. (St=C3=A9phane Glondu, review by Miod Vallat and Vincent Laviron)
- #13188: fix= races in the FFI code coming from the use of Int_val(...) on rooted values inside blocking questions / without the runtime lock. (Calling Int_val(...) on non-rooted immediates is fine, but any access to rooted values must be done outside blocking sections / with the runtime lock.) (Etienne Millon, review by Gabriel Scherer, Jan Midtgaard, Olivier Nicole)<= /li>
- #13318: Fix= regression in GC alarms, and fix them for Flambda. (Guillaume Munch-Maccagnoni, report by Benjamin Monate, review by Vincent Laviron and Gabriel Scherer)
- #13140: POW= ER back-end: fix issue with call to caml_call_realloc_stack from a DLL (Xavier Leroy, review by Miod Vallat)
- #13370: Fix= a low-probability crash when calling Gc.counters. (Demi Marie Obenour, review by Gabriel Scherer)
- #13402, #13512, #13549, #13553: Revise bytecode implemen= tation of callbacks so that it no longer produces dangling registered bytecode fragments. (Xavier Leroy, report by Jan Midtgaard, analysis by Stephen Dolan, review by Miod Vallat)
- #13502: Fix= misindexing related to Gc.finalise_last that could prevent finalisers from being run. (Nick Roberts, review by Mark Shinwell)
- #13520: Fix= compilation of native-code version of systhreads. Bytecode fields were being included in the thread descriptors. (David Allsopp, review by S=C3=A9bastien Hinderer and Miod Vallat)

smaws preview release, an AWS SDK for OCaml using eio

Archive: https://discuss.ocaml.org/t/ann-smaw= s-preview-release-an-aws-sdk-for-ocaml-using-eio/15635/1

Chris Armstrong announced

I'm pleased to announce the first preview release for the smaws library (0.1~preview1).




smaws provi=
des AWS bindings for OCaml using  the modern eio library for effects-based concurrency handling.



Links


Inst=
allation and Usage instructions
Examples




Whats in this release


This release includes SDKs for some AWS services and is intended to demonst=
rate its API.=20



It is not production ready, lacking important features such as full API doc=
umentation EC2/ECS instance metadata authentication, retry and timeout hand=
ling, etc. It also needs support for the other internal AWS API types to ex=
tend coverage across most AWS services.




Motivation


I wanted to build an AWS SDK using modern effects-based concurrency. I've b=
uilt similar bindings for ReScript and ReasonML in the past (some of the co=
de is in fact ported across) but this is the first OCaml-native bindings I'=
ve created.=20



Unlike similar projects in the OCaml ecosystem, it uses the newer Smithy de=
finitions to generate its bindings instead of the Python botocore definitio=
ns. These should be better supported by AWS in the future with richer API d=
efinitions.




What's next


My next task is to finish off API documentation generation, and then expand=
 support for all the authentication methods and other API types that will a=
llow this to be used with most AWS services.


ppx_deriving_ezjsonm


Archive: https://discuss.ocaml.org/t/ann-ppx-deriving-ezjsonm/15637/1


Patrick Ferris announced

I'm happy to announce the release of ppx_deriving_ezjsonm (bas= ed off of p= px_deriving_yaml). The two libraries share a common definition of a "value" which made the reuse of the existing deriver possible for= a simple JSON deriver.

opam update
opam install ppx_deriving_ezjsonm

The documentation is online.

This library may come in handy when your dependency cone already includes <= code>ezjsonm. If that is not the case, you would probably have bette= r luck in the yojson ecosystem of tools.

Happy JSON-ing :camel:

FUN OCaml now has a YouTube Channel

Archive: https://discuss.ocaml.org/t/fun-ocaml-now-has-a-youtube-c= hannel/15639/1

Sabine Schmaltz announced

I just created a YouTube channel for FUN OCaml. :sparkles: :camel:

The talk recordings from the conference in Berlin on September 16 + 17, 202= 4 are now available for viewing!

https://www.youtube.= com/@FUNOCaml/featured

If you can, commenting, liking, or subscribing helps us to make these video= s more visible and easier to find on YouTube, so big thanks for everyone wh= o helps us with this! :orange_heart: :camel:

For people who avoid YouTube: The videos will also be made available on wat= ch.ocaml.org.

Terrateam's open source Ocaml repository

Archive: https://discuss.ocaml.org/t/terrateams-open-source-oc= aml-repository/15645/1

Malcolm announced

A few years ago my friend and I started a company called Terrateam, which does infrastructure orchestration on Gi= tHub. Being that I am an Ocamler and we are a lean company, we chose to us= e Ocaml as our primary language. We recently went open source and I'm post= ing the link here to contribute an example of an actual company using Ocaml= . A real repository.

The code can be found = here.

There a few things to note about the repo:

It's a mono repo, so while many of the libraries in there are generic, = they are not really individually consumable as is.
We have our own concurrency framework (more on that below).
We use our own build library (pds, which is in opam).
The code is in flux all the time so things change rapidly.

Why did we build our own concurrency framework?

Disclaimer: Yet another concurrency framework? Yep! Do I expect anyone to = use it? Nope, and that's ok. It is designed for our needs. It's meant to = be maintainable by one person. It's not meant to compete with Lwt or Async= for mind share. If it grows, great, if it doesn't, I'm happy still.

Our concurrency framework is called "Asynchronous Building Blocks" (Abb). = It started over a decade ago when I was frustrated with a few things:

I wanted kqueue support in Async, but (at the time) Async required modi= fying a handful of repos to support it and it just wasn't obvious how.
Lwt supported kqueue, but for no good reason, I just didn't like Lwt. = Part of it was how failure worked in Lwt and other part is just it didn't f= it my aesthetic. That isn't a ding against Lwt, just personal preference.<= /li>
I wanted as much of it to be implemented in Ocaml as possible. As it s= tands now, the only C code is libkqueue which is a little shim= to to allow kqueue code to run on Linux, otherwise everything is in Ocaml.=
I didn't like how neither Async nor Lwt really supported cancelling ope= rations. I wanted that to be part of the framework, not an ad-hoc feature = per library. Coming from Erlang, cancelling is really important to me and = part of how I think about writing concurrent software. I was bummed that (= last I looked) Eio explicitly rejected cancelling.
I also wanted a little experiment of "what if the concurrency library e= xposed a syscall interface like an OS?" So a lot of the interface is meant= to look low-level (I don't think this idea really panned out or made Abb m= eaningfully different).
I also just like having my own frameworks.

Add a dash of naivete, "how hard can it be to build a concurrency framework= ?", I started my own. First commit was Mar 9, 2013.

Much of the concurrency monad is based on an unreleased library called Fut by @dbuenzli=20

Over time, Abb matured to where I could use it in my personal projects. An= d by the time we decided to make Terrateam, I felt it was good enough for p= roduction. And it's been running production traffic for a few years now.

One, unexpected, benefit of Lwt and Async existing in the community is that= adding a third one isn't that hard. Almost all libraries that want to be = used support both, and that usually means that they have a generic interfac= e. Cohttp and Dns are examples. So I could use existing libraries for thi= ngs I didn't want to or don't feel I could reasonably implement myself.

I've also used Abb as a foundation my web framework called Brtl (pronounced= Brutal) which is both a backend framework build on Cohttp and a frontend f= ramework built on Brr. It really doesn't do anything fancy, like Dream, it= 's pretty low level and focused on being simple.

The good:

It works! At least, for me.
Given that I wrote very single line of code, debugging and bug fixing (= which is less and less) is very easy. I also have a really great mental mo= del of how it works.
I like that I can just cancel a whole graph of async work if it's no lo= nger needed.
The future's library works in FreeBSD, Linux, and JavaScript.
The test coverage is pretty darn high. This is because it's a pretty i= ntricate thing to implement so I had to implement a lot of tests to stay sa= ne.

The bad:

Performance is not anything special. I don't think this is a fundament= al flaw, it's just that it is as fast as I need it to be right now.
Some of the API is a awkward if you don't know the system. Or names ar= e long, like Abb_future_combinators.
The multi-target build story kind of sucks. I think that might be a bi= gger issue with the pds build system but for now in the web framework you h= ave to use Abb_js rather than Abb for everything.=
There are definitely some corners cut in, especially around file IO, bu= t that's OK, we don't do much file IO.
Explicitly takes advantage in that everything runs in a single thread. = So implicitly going multi-threaded would probably break things.

Future work:

There really isn't a lot of future work. For the most part: Abb is done. = Or should I say the interface is done. Yes, it will need updates to fight = bit rot, but there isn't much more for it to do. It runs your code concurr= ently, the end.

However, as Ocaml5 becomes more of a thing, it will need to take advantage = of that. I haven't really thought about how to do it. One item I have in = my to-do list is to evaluate if Picos could be a base layer for Abb. Abb i= s a layered approach so really you only need to implement the Abb_int= f.S interface and everything above that should Just Work (given sing= le threaded semantics). I think any future work to support multi core will= probably need an explicit "this crosses a thread boundary" API. Abb will = get there, eventually, but right now it doesn't need to.

Effects will obviously have a big impact, I have no idea what that'll do fo= r Abb. I hope I can transition it slowly to supporting effects but I don't= want to look at effects until it's in the type system.

Some, perhaps, libraries of interest in the repo:

Abb_scheduler_kqueue - The most used scheduler. = It implements the Abb_intf interface.
Abb_scheduler_select - A simpler select-based sch= eduler. This is meant to be used any place kqueue is not supported and als= o as demo.
Pgsql_io - An implementation of the PostgreSQL protocol.
Githubc2 - An automatically generated GitHub REST library gen= erated from their JSON Schema. This actually has no Abb dependency, just i= mplements the API serializing/deserializing.
OpenAPI CLI - This generates a library (see Githubc2) from= an OpenAPI spec. It is, absolutely, a bit of a rats nest, but it works. = We chose to do code-gen for this because I didn't want to be blocked when c= ompiling based on different compiler versions as we're using Ast_help= er. Ocaml, the code, is more stable than Ocaml, the compiler API.

There is a bunch of other stuff in there. If you decide to poke around and= have any questions, feel free to ask. I can promise: not every decision i= n there is well thought out or coherent.

OUPS december 2024

Archive: https://discuss.ocaml.org/t/oups-december-2024/15654/1

zapashcanon announced

CAUTION: the time has been changed from 7pm to 6:30pm

The next OUPS meetup will take place on Thursday, 12th of December 2= 024. It will start at 6:30pm at the 4 place Jussieu in Paris.= It will be in the in the Esclangon building (amphi Astier).

Please, register on meetup as soon as possible to let us know how ma= ny pizza we should order.

For more details, you may check the OUPS= =E2=80=99 website .

This time we'll have the following talks:

Snapshottable Stores – Cl=C3=A9ment Allain & Gabriel Scherer = (@gasche)

We say that an imperative data structure is snapshottable or supp= orts snapshots if we can efficiently capture its current state, and res= tore a previously captured state to become the current state again. This is= useful, for example, to implement backtracking search processes that updat= e the data structure during search.

Inspired by a data structure proposed in 1978 by Baker, we present a sna= pshottable store, a bag of mutable references that supports snapshots. = Instead of capturing and restoring an array, we can capture an arbitrary se= t of references (of any type) and restore all of them at once. This snapsho= ttable store can be used as a building block to support snapshots for arbit= rary data structures, by simply replacing all mutable references in the dat= a structure by our store references. We present use-cases of a snapshottabl= e store when implementing type-checkers and automated theorem provers.

Our implementation is designed to provide a very low overhead over normal r= eferences, in the common case where the capture/restore operations are infr= equent. Read and write in store references are essentially as fast as in pl= ain references in most situations, thanks to a key optimisation we call = record elision. In comparison, the common approach of replacing referen= ces by integer indices into a persistent map incurs a logarithmic overhead = on reads and writes, and sophisticated algorithms typically impose much lar= ger constant factors.

The implementation, which is inspired by Baker's and the OCaml implementati= on of persistent arrays by Conchon and Filli=C3=A2tre, is both fairly short= and very hard to understand: it relies on shared mutable state in subtle w= ays. We provide a mechanized proof of correctness of its core using the Iri= s framework for the Coq proof assistant. [/details]

Safe, expressive and efficient programming of FPGAs circuits – Lo= =C3=AFc Sylvestre

FPGAs (Field-Programmable Gate Arrays) are reconfigurable digital circuits:= their behavior can be customized by logic synthesis of specification at th= e so-called register transfer level (RT level), in hardware description lan= guages such as VHDL or Verilog. FPGAs are well suited to implement reactive= systems, directly as synchronous circuits interacting with the external en= vironment via I/O pins =E2=80=93 the logic synthesizer ensuring that timing= constraints are met, given the FPGA clock frequency. FPGAs are also used t= o implement hardware accelerators ; however, RT-level descriptions of trans= formational systems (or =E2=80=9Ccomputations=E2=80=9D) =E2=80=93 with late= ncies of several clock cycles =E2=80=93 are difficult to debug, maintain an= d manually optimize. High-Level Synthesis (HLS) offers a simpler way of exp= ressing computations, using a programming language compiled at the RT level= . The advantage of this approach is to keep the implementation details hidd= en from the programmer, leaving the compiler responsible for scheduling com= putations over time. However, this leads to a loss of control over temporal= behavior and therefore safety and efficiency for the circuits generated. A= s embedded systems, especially those based on FPGAs, need to perform more a= nd more computations, while interacting with their environment, this thesis= proposes a programming model to combine hardware description (data-flow or= iented) and general-purpose parallel computation (control-flow oriented) us= ing a synchronous approach. This programming model forms the basis for the = design and implementation of Eclat, a functional-imperative, parallel and s= ynchronous programming language, compiled to VHDL. Eclat is sufficiently pr= ecise to describe synchronous circuits at the RT level. It facilitates the = programming of hardware accelerators, with a clear and predictable temporal= semantics by which to exploit time-space trade-offs. Any Eclat program is = reactive, with a mechanism for embedding computations within programs and t= hereby mix computation and interaction. Eclat also offers shared memory (in= the form of RAM blocks), with deterministic concurrency. It is used to dev= elop programming abstractions such as algorithmic skeletons and virtual mac= hine implementations for high-level languages. This addresses, at various l= evels, the need to run general-purpose algorithms within FPGA-based reactiv= e embedded applications.

After the talks there will be some pizzas offered by the OCaml Software Foundation and later on we=E2=80=99ll mov= e to a pub nearby as usual.

Dune dev meeting

Archive: https://discuss.ocaml.org/t/ann-dune-dev-meeting/14994/17

Etienne Marais announced

Hi camelers! :camel:=20 We will hold our regular Dune dev meeting on Wednesday, November, 27th at 1= 6:00 CET. As usual, the session will be one hour long.

Whether you are a maintainer, a regular contributor, a new joiner or just c= urious, you are welcome to join: these discussions are opened! The goal of = these meetings is to provide a place to discuss the ongoing work together a= nd synchronise between the Dune developers :ok_hand:=20

:calendar: Agenda

The agenda is available on the meeting dedicated page. Feel free to ask if y= ou want to add more items in it.

:computer: Links

Meeting link: zoom
Calendar event: google calendar
Wiki with information and previous notes: GitHub Wiki

Other OCaml News

From the ocaml.org blog

Here are links from many OCaml blogs aggregated at the ocaml.org blog.

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a loo= k at the archive or the <= a href=3D"https://alan.petitepomme.net/cwn/cwn.rss">RSS feed of the archive= s.

If you also wish to receive it every week by mail, you may subscribe to the= caml-list.

Alan Schmitt