[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

Alan Schmitt
2024-10-15T13:31:19Z

[-- Attachment #1: Type: text/plain, Size: 14506 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of October 08 to 15,
2024.

Table of Contents
─────────────────

grep_cmt: structural search of OCaml code
Mutaml 0.1
ocaml-activitypub
Ortac/QCheck-STM 0.4.0 Dynamic formal verification beyond one system under test
Openbsd 1.0
Compsort - reorder files in archives for improved compression
Dune dev meeting
Other OCaml News
Old CWN

grep_cmt: structural search of OCaml code
═════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-poc-grep-cmt-structural-search-of-ocaml-code/15411/1>

Nicolas Ojeda Bar announced
───────────────────────────

As mentioned in a previous post:

<https://discuss.ocaml.org/t/ann-2nd-editor-tooling-dev-meeting-25th-of-july/14953/5?u=nojb>

I had promised to post back here when we had made the source code for
the "structural grep" tool that I presented, public.

This is now done:

<https://github.com/LexiFi/grep_cmt>

We added a `[POC]' marker to this post, because the code is not really
ready for public consumption (it is rough around the edges and may not
work in all circumstances). Our hope is to publicize the approach and
perhaps motivate interested hackers to take the code and develop it
further into a proper tool.

Mutaml 0.1
══════════

Archive: <https://discuss.ocaml.org/t/ann-mutaml-0-1/12639/2>

Jan Midtgaard announced
───────────────────────

I'm happy to share news of the Mutaml 0.3 release! :tada:
<https://github.com/jmid/mutaml/releases/tag/0.3>

Together with the recent 0.2 release, this brings Mutaml up to speed
with recent ppxlib releases and addresses a few issues reported by
brave, early users:

• Avoid mutations in attribute parameters #29
• Avoid polymorphic equality which is incompatible with Core #30
• Add support for ppxlib.0.28 and above #27
• Avoid triggering 2 mutations of a pattern incl. a `when'-clause
causing a redundant sub-pattern warning #22, #23

Happy testing! :smiley:

ocaml-activitypub
═════════════════

Archive: <https://discuss.ocaml.org/t/ann-ocaml-activitypub/15420/1>

Zoggy announced
───────────────

I'm glad to announce a first release of `activitypub*' packages,
implementing (well, trying to implement some flavor of) both
[server-to-server] and [client-to-server] activitypub protocols.

Documentation is available from the [web site].

The `activitypub_server' package installs [TAPS], an experimental
server, handling some common activities. Accounts hosted by this
server can at least follow and be followed by mastodon instances, and
post and receive activities (Create, Announce, Like, Undo, …).

The library of the `activitypub_client' package can be used by a
client application to post and receive activities to and from a server
(though it was only tested with TAPS). See a simple example [here].

A GUI client (based on [Stk] is installed by the `activitypub_gui'
package. It requires a client configuration file as described
[here]. You can drop IRIs/URLs of an actor in the window to open a tab
and be able to follow this actor. The GUI also allows to create and
post notes with attachments. This client is still very experimental
and will be developed more in the future.

EDIT: the package should be available soon in opam.

[server-to-server]
<https://www.w3.org/TR/activitypub/#server-to-server-interactions>

[client-to-server]
<https://www.w3.org/TR/activitypub/#client-to-server-interactions>

[web site] <https://zoggy.frama.io/activitypub/>

[TAPS] <https://zoggy.frama.io/activitypub/doc-taps.html>

[here] <https://zoggy.frama.io/activitypub/doc-client-example.html>

[Stk] <https://zoggy.frama.io/ocaml-stk/>

[here]
<https://zoggy.frama.io/activitypub//refdoc/activitypub_client/Activitypub_client/Main/index.html>

Ortac/QCheck-STM 0.4.0 Dynamic formal verification beyond one system under test
═══════════════════════════════════════════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-ortac-qcheck-stm-0-4-0-dynamic-formal-verification-beyond-one-system-under-test/15427/1>

Nicolas Osborne announced
─────────────────────────

I'm very pleased to announce this exciting new release of
`ortac-qcheck-stm.0.4.0'!

This new release brings some exciting new features, mostly the result
of Nikolaus Huber's contributions! Thank you Nik!

Ortac/QCheck-STM is a test generator based on the [QCheck-STM]
model-based testing framework and the [Gospel] specification language
for OCaml.

You can find the project on [this repo] and install the released
packages via `opam'.

It is also encourage to install `ortac-dune' to avoid having to write
too much dune boilerplate.

In particular, this release extend Ortac/QCheck-STM so that the
generated tests will include functions that can take multiple
System-Under-Tests as argument and/or that can return one. So now, if
we write Gospel specifications for `append'-like functions,
Ortac/QCheck-STM will include them in the generated tests!

Happy testing!

[QCheck-STM] <https://github.com/ocaml-multicore/multicoretests>

[Gospel] <https://github.com/ocaml-gospel/gospel>

[this repo] <https://github.com/ocaml-gospel/ortac>

Openbsd 1.0
═══════════

Archive: <https://discuss.ocaml.org/t/ann-openbsd-1-0/15434/1>

Sebastien Marie announced
─────────────────────────

I would like to announce a new (somehow niche) package [Openbsd],
which provides bindings for some specifics OpenBSD syscalls
[pledge(2)] and [unveil(2)].

These syscalls lets the kernel OS to know what the running processus
is expected to do, and so it is possible to restrict a processus to do
only filesystem or only network or only pure computation…

The package is designed to be installable on any platform and provides
simple method to check if `Pledge' or `Unveil' are supported. This
way, it is possible to easily write portable code using the package,
as it could be a turned on "no-operation" on Windows or Linux hosts
(or provides alternative code path for sandboxing).

• Homepage : [https://codeberg.org/semarie/ocaml-openbsd/]
• License : [ISC]
• Documented Interface : [lib/openbsd.mli]

[Openbsd] <https://ocaml.org/p/openbsd/latest>

[pledge(2)] <https://man.openbsd.org/pledge.2>

[unveil(2)] <https://man.openbsd.org/unveil.2>

[https://codeberg.org/semarie/ocaml-openbsd/]
<https://codeberg.org/semarie/ocaml-openbsd/>

[ISC] <https://en.wikipedia.org/wiki/ISC_license>

[lib/openbsd.mli]
<https://codeberg.org/semarie/ocaml-openbsd/src/tag/1.0/lib/openbsd.mli>

Examples
╌╌╌╌╌╌╌╌

┌────
│ let open Openbsd in
│ if Pledge.supported then
│ Pledge.promises "stdio rpath"
└────

┌────
│ let open Openbsd in
│ if Unveil.supported then (
│ Unveil.add "./lib" "r";
│ Unveil.add "./logs" "rwc";
│ Unveil.lock ())
└────

Compsort - reorder files in archives for improved compression
═════════════════════════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-compsort-reorder-files-in-archives-for-improved-compression/15436/1>

adrien announced
────────────────

I'm happy to announce the first release of compsort, a tool to reorder
files in an archive for better compression. It works by grouping files
according to a distance that is computed between every file pair. You
can install it with `opam install compsort' (requires ocaml 5.2 for
parallelism).

Website with more details and examples in `README.md', plus source:
<https://gitlab.com/adrien-n/compsort/>

The goal is not new but, AFAIK, the approach is. I am very interested
in prior or related art!

Results
╌╌╌╌╌╌╌

Compsort achieves improvements that would typically require larger
compression windows and therefore more memory. The improvements are
only a few percents but in the domain of compression, a few percents
is actually a lot.

With `xz' compression, a Ubuntu initrd on my machine is reduced by
more than 11.5%, while the best achievable improvement is 12.7% (the
reordering gives 90% of the best result). Similarly, the tree of
`linux-firmware.git' can be compressed 6% better, while the best
achievable improvement is 9.4% (the reordering gives 63% of the best
result).

Visualizations
╌╌╌╌╌╌╌╌╌╌╌╌╌╌

In order to better explain what it does, I quite like the
visualizations I have so far (there will be better ones in the
future), where the value of the pixel at `(x,y)' indicates how similar
are files `x' and `y'.

Before:

<https://gitlab.com/adrien-n/compsort/-/raw/main/doc/img/bettercomp_python3-django-horizon_noop.png>

After reordering:

<https://gitlab.com/adrien-n/compsort/-/raw/main/doc/img/bettercomp_python3-django-horizon_buckets.png>

Files that are very different from others are all packed at the end
and there's also an isolated cluster of files together similar but
different from everything else. One can also see that the distinct
row and column pattern from the first picture has disappeared: it
indicated that every 15 or so files in that region were similar and
were separated by disimilar files but that they're now grouped.

While there are certainly improvements possible, results are
good. It's a case where one might wonder why results are so good
considering all the approximations that took place.

[1] Most of the algorithms/libraries I've tried to use rely on having
an actual proper distance function which I don't have

Future work
╌╌╌╌╌╌╌╌╌╌╌

I'll try to improve the distance function. Currently it does some
steps of compression algorithms to detect redundancies but maybe
reusing a compression library would give better results if it can be
made fast enough (lz4 is borderline but it has tiny dictionaries
unfortunately).

Clustering could be better as the current algorithm is very basic (it
collects files that are 90% similar, then 80% similar, then 70%, …). I
tried several algorithms but I don't have a good-enough distance
function (for instance the triangular inequality probably doesn't
hold) and results were worse.

All this will benefit from better visualizations and I'd like to have
interactive plots that can be hovered on with the mouse to get the
distance value and full file name.

Oh and code isn't always pretty as it went through a lot of
experimental stages and low-level tweaks to improve performance.

Dune dev meeting
════════════════

Archive: <https://discuss.ocaml.org/t/ann-dune-dev-meeting/14994/13>

Etienne Marais announced
────────────────────────

We will hold our regular Dune dev meeting tomorrow, on _Wednesday,
October, 16th at 10:00 CET_. As usual, the session will be one hour
long.

Whether you are a maintainer, a regular contributor, a new joiner or
just curious, you are welcome to join: these discussions are opened!
The goal of these meetings is to provide a place to discuss the
ongoing work together and synchronise between the Dune developers !

Agenda
╌╌╌╌╌╌

The agenda is available on the[ meeting dedicated page]. Feel free to
ask if you want to add more items in it.

[ meeting dedicated page]
<https://github.com/ocaml/dune/wiki/dev-meeting-2024-10-16>

Links
╌╌╌╌╌

• Meeting link:[ zoom]
• Calendar event:[ google calendar]
• Wiki with information and previous notes:[ GitHub Wiki]

[ zoom]
<https://us06web.zoom.us/j/85096877776?pwd=cWNhU1dHQ1ZNSjZuOUZCQ0h2by9Udz09>

[ google calendar]
<https://calendar.google.com/calendar/embed?src=c_5cd698df6784e385b1cdcdc1dbca18c061faa96959a04781566d304dc9ec7319%40group.calendar.google.com>

[ GitHub Wiki] <https://github.com/ocaml/dune/wiki#dev-meetings>

Other OCaml News
════════════════

From the ocaml.org blog
───────────────────────

Here are links from many OCaml blogs aggregated at [the ocaml.org
blog].

• [The Uncertain Art of Accelerating ML Models with Sylvain Gugger]
• [Dune Package Management: Revolutionising OCaml Development]

[the ocaml.org blog] <https://ocaml.org/blog/>

[The Uncertain Art of Accelerating ML Models with Sylvain Gugger]
<https://signals-threads.simplecast.com/episodes/the-uncertain-art-of-accelerating-ml-models-with-sylvain-gugger-moYuL4Ps>

[Dune Package Management: Revolutionising OCaml Development]
<https://tarides.com/blog/2024-10-09-dune-package-management-revolutionising-ocaml-development>

Old CWN
═══════

If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].

If you also wish to receive it every week by mail, you may subscribe
to the [caml-list].

[Alan Schmitt]

[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <https://alan.petitepomme.net/cwn/>

[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>

[caml-list] <https://sympa.inria.fr/sympa/info/caml-list>

[Alan Schmitt] <https://alan.petitepomme.net/>

[-- Attachment #2: Type: text/html, Size: 26875 bytes --]