OCaml Weekly News

I am pleased to announce the release of a new library Blurhash 0.1.0!

https://github.com/ushitora-anqou/ocaml-blurhash

This library implements an encoder for BlurHash in OCaml. BlurHash allows us to convert an image into a short string that represents a lightweight, ’blurred’ version of the input image. It can be used as a placeholder on a Web page, for example.

The only function this library provides is Blurhash.blur_hash_for_pixels, which takes the same arguments as blurHashForPixels in the C reference implementation.

Although it does not depend on any image library, you can use (e.g.,) camlimages to load an image from a file.

let load_image_as_rgb24 ~path =
  match OImages.(load path [] |> tag) with
  | Rgb24 img -> img
  | Rgba32 img -> img#to_rgb24
  | Index8 img -> img#to_rgb24
  | Index16 img -> img#to_rgb24
  | Cmyk32 _ -> failwith "Not supported image type: Cmyk32"

let blurhash ~x_components ~y_components src =
  Blurhash.blur_hash_for_pixels ~x_components ~y_components ~width:src#width
    ~height:src#height ~bytes_per_row:(src#width * 3) src#dump

let test_encode_case1 () =
  let src = load_image_as_rgb24 ~path:"../../../test/test.ppm" in
  let hash = blurhash ~x_components:4 ~y_components:3 src in
  Alcotest.(check string) "test1" "LFE.@D9F01_2%L%MIVD*9Goe-;WB" hash;
  ()

Github link

I’ve recently released v4 of SZXX and it’s now built on Eio instead of Lwt.

It made me realize just how much complexity we’ve all accepted as the cost of doing asynchronous IO.

The code was so thoroughly simplified thanks to Eio’s non-monadic interface (no more colored functions that infect everything with `’a Lwt.t`) and its concept of Switch that I was able to implement complex features and optimizations that I had previously deemed too costly in terms of development time and added complexity.

v4 is more than 3x faster, much easier to use correctly, and offers stronger memory usage guarantees, all thanks to Eio.

SZXX is a *S*treaming *Z*IP, *X*ML and *X*LSX library.

It can stream data out of these 3 file formats even when reading from a network socket, either in constant memory or with user-defined memory usage guarantees.

All 3 formats are quite “quirky” to say the least. XLSX (aka. OOXML) is infamous for being difficult to stream. I could talk at length about all the different subspecies of ZIP files!

Whenever giving up the non-seekability requirement (network streams, etc.) brings benefits, SZXX offers both interfaces: an easier and/or more performant function that may “jump” around a file, and also a more advanced non-seeking function.

Hundreds of hours of benchmarking, optimization and testing have gone into this latest release to squeeze out as much performance as possible and I’m extremely pleased with the result.

I hope it proves useful to the OCaml ecosystem. Feel free to ask any questions about SZXX, Eio, XLSX, ZIP, XML, etc.

The OCaml bindings to LLVM 15 have been released on opam!

This is a milestone in work contracted by the OCaml Software Foundation to ensure the OCaml LLVM bindings remain up to date with both changes in OCaml and LLVM. First, I would like to thank @jjb for supervising me, and @Kit-ty-kate and @Kakadu on GitHub for their valuable help.

While previous versions of OCaml have supported naked pointers (pointers to outside the OCaml heap as OCaml values), OCaml 5’s new multicore runtime disallows naked pointers. Furthermore, OCaml 5 deprecated various runtime API functions. I had previously written https://reviews.llvm.org/D136400 to adapt the LLVM bindings on the main branch to these changes, but LLVM 15 had already been released by then, with naked pointers in use. Therefore, I backported D136400 to LLVM 15 on my fork, and this fork is what is published on opam. Therefore, the version of LLVM 15 on opam is compatible with OCaml 5.

Furthermore, the LLVM IR has been migrating from pointers which carry the types of their pointees to untyped “opaque pointers.” As a result, many LLVM API functions have been deprecated and replaced. For example, build_call has been deprecated with the replacement being build_call2. I have added deprecation alerts to the relevant functions and added missing bindings to construct opaque pointers.

The changes between the upstream LLVM 15 branch and the fork published on opam can be viewed at https://github.com/llvm/llvm-project/compare/release/15.x...alan-j-hu:llvm-project:aa44040b6096870d6bdecc7cc7c9e554f7b25a54.

Independently, @Kit-ty-kate had been working on building the LLVM bindings with Dune instead of the in-tree OCamlfind-based system at https://github.com/kit-ty-kate/llvm-dune. Her repo has a Git submodule that points to my fork of LLVM, and her repo contains the Dune scaffolding that the LLVM 15 release on opam uses. This change of build systems may have implications for selecting between dynamic and static linking to LLVM:

• Prior opam releases of LLVM use OCamlfind, which is supported via the in-tree META file. The package had a script that installed versions of the bindings that link statically or dynamically to LLVM (if each was available) and patched the META file to use the llvm.static predicate to choose between them.
• The llvm.static predicate can be set using OCamlfind but cannot be set from Dune.
• The LLVM 15 release uses Dune, which it supports by running a script that generates Dune files in the source tree. The script’s generated Dune files use the virtual library feature to define versions of the library that link statically or dynamically to LLVM (if each is available).
• OCamlfind-based packages don’t understand the Dune virtual library feature and therefore cannot use the “default implementation” and must select the static or dynamic implementation explicitly.
• @kit-ty-kate believes that the package builds faster and is easier to maintain with Dune.

Further context: https://discuss.ocaml.org/t/ask-for-suggestions-for-the-next-z3-package-release/8922/4 and https://discuss.ocaml.org/t/using-output-of-ocamlfind-params-only-show-in-a-dune-build/4187.

In the process of preparing this release, @Kit-ty-kate and @Kakadu tested my changes on their personal projects and helped uncover issues.

Please let me know if you have any questions.

We have the pleasure of celebrating the anniversary of Olympe de Gouges’ “Declaration of the Rights of Woman and of the Female Citizen” by announcing the release of OCaml version 5.1.0.

Some of the highlights in OCaml 5.1.0 are:

• Many runtime performance regression and memory-leaks fixes

(dynlinking, weak array, weak hash sets, GC with idle domains, GC prefetching)

• Restored support for native code generation on RISC-V and s390x architectures
• Restored Cygwin port
• Reduced installation size (50% reduction)
• Compressed compilation artefacts (.cmi, .cmt, .cmti, .cmo, .cma files)
• 19 error message improvements
• 14 standard library functions made tail-recursive with Tail-Recursion-Modulo-Cons (TRMC), such as List.append and List.map
• 57 new standard library functions
• More examples in the standard library documentation
• 42 bug fixes

OCaml 5.1.0 is still a relatively experimental release compared to the OCaml 4.14 branch. In particular:

• The POWER port is being tested in the dev version of the compiler.
• The Windows MSVC port is still unavailable.
• Ephemeron performances need to be investigated.
• GC compaction is a work in progress.
• statmemprof is a work in progress.
• There are a number of known runtime concurrency bugs (that trigger under rare circumstances).

We are planning to address those regressions, hopefully in time for the OCaml 5.2.0 release for some of them. Meanwhile, the OCaml 4.14 branch will be maintained, and the next release on the OCaml 4.14 branch, OCaml 4.14.2, should follow this release in the upcoming months.

Please report any unexpected behaviours on the OCaml issue tracker and post any questions or comments you might have on our discussion forums.

The full list of changes can be found in the changelog below.

A simple porting guide from Lwt to Eio as a definitive answer to my own question from years ago: https://github.com/ocaml-multicore/lwt_eio#porting-a-lwt-application-to-eio

Unicode 15.1.0 was released on September 12.

This is a point release for Unicode organisational reasons but it still adds 627 new characters to the standard and a few rules were changed in the segmentation standards. See the details on the announcement page.

Accordingly the libraries mentioned at the end of this message had to be updated. Consult the individual release notes for details. Both Uucd and Uucp are incompatible releases sinces a block enumerant had to be added and some property values changed their type. A few new properties related to identifiers, CJK and Indic breaking are also added, see the Uucp release notes for details.

As mentioned last year all the libraries and sample code have been changed to use the UTF decoders of the standard library rather than rely on the uutf package.

This has the following impact:

1. These new versions are only available for OCaml >= 4.14.0
2. The library name uunf.string is deprecated. The Uunf_string module is now simply part of the uunf library.
3. The library name uuseg.string is deprecated. The Uuseg_string module is now simply part of the uuseg library.

Regarding point 2. and 3. the libraries still exist but generate an ocamlfind warning if they are used. They are empty and simply require the base library. They will be fully removed at some point.

Two other less visible changes are:

• After waiting for too long if intra module link time dead code elimination would maybe make it in the compiler, Uucp was finally changed to use module aliases. This means that only the data modules you use get linked in your programs.
• Also after much reluctance, the repos now track generated data files for better source traceability, sandboxed pinning, and make it easier to dig them out when their data generation strategy breaks the compiler.

A big thanks for funding from the OCaml Software Foundation and from my donators.

I welcome and thank the new donator ahrefs.

And remember, OCaml :heart: Unicode.

• Uucd 15.1.0 Unicode character database decoder for OCaml. http://erratique.ch/software/uucd
• Uucp 15.1.0 Unicode character properties for OCaml. http://erratique.ch/software/uucp
• Uunf 15.1.0 Unicode text normalization for OCaml. http://erratique.ch/software/uunf
• Uuseg 15.1.0 Unicode text segmentation for OCaml. http://erratique.ch/software/uuseg

Hello everyone! First time poster and ocaml contributor here, I just release a library named “safemoney” which is intended to uphold type safety and enforce lossless operations for various kind of currency manipulations and offers convenient api for users. I started learning ocaml about 2 months ago so it was a great learning experience for me making this library and I must say I absolutely fell in love with ocaml and really think this is the best abstraction for writing elegant, functional and performant software, like rust + haskell supercharged but better ;) really glad I found this community and everyone seems very helpful and kind indeed.

You can find further info here:

• source code: https://github.com/gborough/safemoney
• documentation: https://gborough.github.io/safemoney/safemoney/safemoney/index.html

Please do forgive me for mucking up the documentation at this moment as I am still trying to figure out the quirks with writing and publishing docs, will fix it soon to the best of my ability. I used dune-release for publishing to gh-pages and seems like a quite a manual process, if you know a better way then please do let me know.

Thank you and have a lovely weekend!

I’m pleased to announce the release of ppx_optint.0.2.0, a small ppx for optint literals. Instead of writing Optint.minus_one or Optint.Int63.minus_one you can write -1i or -1I respectively. It may be handy if you write code with a lot of optint literals.

Version 0.2.0 tries to be smarter and will use Optint.of_unsigned_int32 and Optint.Int63.of_unsigned_int over _.of_string when possible.

DkML Workflows are GitHub Actions, GitLab CI/CD and desktop scripts. Version 2.0.3 is available today by clicking here. Who is this for?

• If you want to compile and distribute binaries for Windows, Linux and macOS, then the compilers workflows may be a good choice for you.
• If you are a package maintainer and want to test that your package compiles with Visual Studio MSVC, the Windows compilers workflow is a good choice.

I personally use the setup-ocaml GitHub Action for testing OCaml code, and additionally use DkML Workflows to distribute binaries in my own software like the Windows/Linux/macOS diskuvbox binaries.

The problems I was trying to solve:

• It has to work with MSVC. It is really easy to distribute binaries on Windows if you use MSVC.
• I grew really really tired of debugging issues remotely in vendor locked GitHub Actions. GitLab CI/CD is not as bad but still not good. I needed the ability to easily run the CI/CD directly on PCs.
• I am not a fan of static linking. IMHO, static linking is an overly complex solution in search of a problem that is only present on Linux (probably due to the 600+ Linux distributions). And among other things it complicates security. But …
• I do like static linking’s “easy to deploy” feature: you can static link an executable on your Linux desktop and scp to a Linux server. (See “Changes” below)
• I wanted to generate Apple Silicon binaries on Apple x86_64 CI/CD hardware.

Changes with 2.0.3:

• Finally implemented the “easy to deploy” feature. Now the steps for a working CI/CD pipeline are just: 1) edit .gitignore, 2) edit .gitattributes and 3) copy-and-paste into your terminal.
• Make use of the standardized, upgradable ./dk script to do all the fetching and creation of scripts.

Nomenclature: DkML Workflows is a part of DkML, helps to maintain the Windows ecosystem, and has been supported financially by the OCaml Software Foundataion, but is separate from the DkML Installer many people are aware of. I’ll try to distinguish them going forward.

Here is a feature comparison:

The ML workshop can be watched here: https://www.youtube.com/watch?v=M5M3f31pxns&t=2732s (someone has posted timestamps for the various talks in the comments of the video).