OCaml Weekly News

The OCaml Users and Developers Workshop: ACM SIGPLAN Workshop September 9, 2023 (Saturday) Seattle, WA, USA and also online (two days after main ICFP)

Call for presentations: https://icfp23.sigplan.org/home/ocaml-2023 Talk proposal submission deadline: Thursday June 1st, 2023

(Please redistribute widely.)

The OCaml Users and Developers Workshop brings together the OCaml community, including users of OCaml in industry, academia, hobbyists and the free software community.

OCaml 2023 will be co-located with ICFP 2023, which will take place in Seattle, US. We aim to organize it as a hybrid event, so that people can attend and even give talks remotely: talks will be streamed in real-time, and virtual participants will be able to chat and ask questions in writing.

I’m pleased to announce the first release of OCaml-ldp, which includes

• the ldp package to build LDP (Linked Data Platform) applications, runnable either in standalone program (using packages ldp_tls or ldp_curl) or in the browser (using package ldp_js with js_of_ocaml).
• the solid package, providing a library to interact with SOLID pods,
• the solid_server package, providing an experimental SOLID server,
• the solid_tools package, including some basic command-line tools to interfact with SOLID servers and a library to develop other tools.

The packages should be available in opam soon.

We just released a point release that fixes some issues:

3.8.0 is planned to be released in May.

It’s my pleasure to announce the first release of the Cmarkit library.

Cmarkit parses the [CommonMark specification]. It provides:

• A CommonMark parser for UTF-8 encoded documents. Link label resolution can be customized and a non-strict parsing mode can be activated to add: strikethrough, LaTeX math, footnotes, task items and tables.
• An extensible abstract syntax tree for CommonMark documents with source location tracking and best-effort source layout preservation.
• Abstract syntax tree mapper and folder abstractions for quick and concise tree transformations.
• Extensible renderers for HTML, LaTeX and CommonMark with source layout preservation.

Cmarkit is distributed under the ISC license. It has no dependencies.

This first release benefited from a grant from the [OCaml software foundation]. Funding from my few but faithfull [donators] is also paramount to get these tedious bits out for release. Thank you all for your support.

• Homepage: https://erratique.ch/software/cmarkit
• Docs: https://erratique.ch/software/cmarkit/doc (or odig doc cmarkit)
• Install: opam install cmarkit (once this [PR] is merged)

Since someone is going to ask here’s my biased comparison between cmarkit and omd, the only other OCaml commonmark parser I’m aware of.

This is based on what I see here and what I fell on in using the omd tool.

1. cmarkit takes the whole document as input, omd can work line by line on input channels. Note however that in practice due to how CommonMark parsing works you need it in memory anyways and wait for the end of input to trigger inline parsing.
2. cmarkit should conform to the CommonMark spec, all conformance tests pass. I don’t think omd does since U+0000 doesn’t seem to be replaced by U+FFFD and in general it seems to forgo UTF decoding.
3. cmarkit provides location tracking and source layout information in the AST omd does not.
4. cmarkit fails on 3/22 (2/22 in OCaml 5) of the cmark pathological tests. omd fails on 17/22 of them.
5. From commonmark to html on a large 12Mo md file, cmarkit seems slightly faster (26%) than omd (even when locs and layout it’s still 10% faster). But no scientific benchmark was performed. Nor particular attention was paid to that. Nor is it likely to be important in practice (unless you are in charge of rendering all the READMEs of a code hosting platform).
6. cmarkit has renderers to CommonMark (layout preserving) and LaTeX. omd hasn’t, but it has one to sexp which cmarkit hasn’t.
7. cmarkit renderers are extensible and partially redefinable. omd ones aren’t.
8. cmarkit lets client customize link label definition and resolution which allows to embed data binding DSLs in the very flexible label syntax. omd has no such thing.
9. cmarkit’s AST is extensible. omd’s one is not.
10. cmarkit as per node extensible metadata. omd uses a polymorphic scheme.
11. cmarkit has AST mappers and folders. omd has no such thing.
12. cmarkit has no dependencies. omd depends on a bunch of other packages.
13. cmarkit and omd support different syntax extensions. It is unclear which ones are supported by omd, for cmarkit see the docs.
14. cmarkit reuses the CommonMark spec vocabulary and the docs are fully hyperlinked into the specification to help you understand the terrible morass you are dealing with.
15. cmarkit, the tool provided with the library, is a bit more featureful than the omd (or reference cmark) tool. Notably (with enough options specified :–) support is provided to output full HTML and LaTeX documents that are ready to read and render.

In general I’d say omd is fine if you are just interested in taking a CommonMark string to a default CommonMark rendering. If you are interested in making systems that integrate CommonMark as a medium that you process and play with you will be better off with cmarkit.

Finally it should be noted that omd was started in darker times when no CommonMark specification existed. Having spent a significant amount of time on cmarkit with a specification one can only appreciate the thoughness of the initial effort.

As a show-off for the recently announced cmarkit. I have devised ocamlmark, a bi-directional translation between ocamldoc and CommonMark with the least amount of CommonMark extensions possible.

It is defined in this document.

This unreasonable hack benefited from a grant of the OCaml Software Foundation. It was however not solicited by The Foundation and does not represent any endorsement by The Foundation.

A fully functional proof of concept[^1] implementation on odoc-parser can be found in this PR. If you want to try it you should be able to:

opam pin odoc --dev
opam pin add cmarkit https://erratique.ch/repos/cmarkit.git # or opam install cmarkit
opam pin add odoc-parser https://github.com/dbuenzli/odoc-parser#ocamlmark

It allows you to write your OCaml docstrings and .mld files in ocamlmark. Both ocamlmark and ocamldoc syntaxes are supported transparently by sniffing each comment for a syntax.

If ocamlmark is detected, cmarkit is used to parse the comment and the resulting abstract syntax tree is translated to ocamldoc’s one. Thanks to precise source location tracking on both sides all errors reported by odoc should be reported correctly in ocamlmark constructs.

This shows how to use cmarkit’s label resolvers to good effect. Namely to provide the counter part to ocamldoc extensive cross-reference linking capabilities by allowing to specify them with the same syntax – save for the braces – in CommonMark’s link labels.

Personally I’m not really convinced it’s a good idea, doc QA and readability are concerns, see the tradeoffs. But I’d just thought I’d put that on the table for discussion; sometimes it’s better to think we could have it rather than have it ;–)

[^1]: Except for missing support for @-tags and odoc latest addition (math and tables, but they are supported by cmarkit) it is reasonably complete.

I’m happy to announce that the kcas package now has a kcas_data companion package that provides implementations of compositional lock-free data structures implemented using kcas.

What is kcas?

kcas provides an implementation of atomic lock-free multi-word compare-and-set (MCAS), which is a powerful tool for designing concurrent algorithms.

On top of the efficient multi-word compare-and-set algorithm, kcas provides compositional transactional programming interfaces that can make it much easier to implement concurrent algorithms.

The first version of the new kcas_data package includes domain safe Hashtbl, Queue, and Stack data structures that all mimic the corresponding Stdlib module interfaces and can be more or less be used as drop-in replacements when domain safety is needed.

The kcas_data data structures also provide transactional interfaces that allow one to compose new lock-free operations with any other kcas based transactions. For example, given a queue and a stack one can atomically take an element from the queue and push it to the stack using a transaction written in direct style via explicit transaction log passing as follows:

let tx ~xt =
  match Queue.Xt.take_opt ~xt queue with
  | None -> ()
  | Some value ->
    Stack.Xt.push ~xt stack value
in
Xt.commit { tx }

Aside from offering composability, the provided data structures should give good performance and scalability compared to protecting all accesses of unsynchronized Stdlib data structures using locks.

Feel free to give the new kcas_data package a spin!

I would also like to encourage people to try and implement more compositional lock-free data structures using kcas. It is fairly straightforward to translate textbook imperative data structures using kcas to make them domain safe and lock-free and the project README comes with many examples.

Note that while the kcas_data package is currently in the kcas repository, it might later be moved to be a part of the repository that is currently called lockfree.

I’m the shepherd for the upstream compiler issue https://github.com/ocaml/ocaml/issues/12037 , which is about a potential code improvement noticed by gcc -fanalyzer. The issue has somewhat-detailed explanations of what could be done to solve it nicely.

If you have been looking for an opportunity to contribute to the upstream OCaml compiler, this may be one. This is in the runtime code, implemented in C, and extern.c is the marshaller (what runs when you use output_value or Marhsal.to_string).

Assuming you are already familiar with C and typical git+github workflows, but have never contributed to the OCaml compiler codebase, here would be a time estimate:

• 1h to get familiar with the github/ocaml repository (clone it, configure it, build it, etc.; I am assuming that you are not trying to program from a Windows machine, which may be harder), which includes skimming https://github.com/ocaml/ocaml/blob/trunk/CONTRIBUTING.md and https://github.com/ocaml/ocaml/blob/trunk/HACKING.adoc
• 1h-2h to write an initial patch that you are happy with and send a PR
• 1h-3h of addressing CI issues and review comments

Please join the Houston Functional Programming User Group next Wednesday, April 19 at 7pm U.S. Central time (midnight UTC) when Jonah Beckford (@jbeckford) will discuss his work on developing Diskuv OCaml (DkML).

As everybody who’s been following DkML’s development is aware, Jonah’s work on bringing OCaml and it’s toolchain to Windows is incredibly impressive. In this talk, he’ll discuss why he decided upon this path, the challenges that he faced and how he overcame them, and his ongoing and future work, including the OCaml SDK that he will release this year.

If you’re in the Houston area, you can join us in person. Otherwise, please join us via Zoom. Compete details, including his abstract and Zoom connection info are available on our website at https://hfpug.org.

Good morning everyone! :sunrise:

I’ve started learning OCaml only recently, and I would like to share my OSS project with everyone:

• :sparkles: zbg — Zero Bullshit Git

I’ve been using git for many years, and I came up with a workflow that works perfectly for me but it was tedious to type all the commands manually, and .gitconfig aliases weren’t powerful enough. So I decided to package my workflow into a CLI tool. I also added prettier versions of some commands. Maybe you’ll enjoy them too :hugs:

I started learning OCaml only recently; this is my first OSS project with OCaml. Before that, I was mostly using Haskell for the last 7 years. Still, I would greatly appreciate suggestions on improving the code, making it more idiomatic, utilizing OCaml tooling, and everything else. Feedback is welcome!

There’s still some work to be done in zbg, so if you want to have fun and join the development, have a look at open issues.