[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

[Alan Schmitt <alan.schmitt@polytechnique.org>]

[-- Attachment #1: Type: text/plain, Size: 14506 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of October 08 to 15,
2024.

Table of Contents
─────────────────

grep_cmt: structural search of OCaml code
Mutaml 0.1
ocaml-activitypub
Ortac/QCheck-STM 0.4.0 Dynamic formal verification beyond one system under test
Openbsd 1.0
Compsort - reorder files in archives for improved compression
Dune dev meeting
Other OCaml News
Old CWN


grep_cmt: structural search of OCaml code
═════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/ann-poc-grep-cmt-structural-search-of-ocaml-code/15411/1>


Nicolas Ojeda Bar announced
───────────────────────────

  As mentioned in a previous post:

  <https://discuss.ocaml.org/t/ann-2nd-editor-tooling-dev-meeting-25th-of-july/14953/5?u=nojb>

  I had promised to post back here when we had made the source code for
  the "structural grep" tool that I presented, public.

  This is now done:

  <https://github.com/LexiFi/grep_cmt>

  We added a `[POC]' marker to this post, because the code is not really
  ready for public consumption (it is rough around the edges and may not
  work in all circumstances). Our hope is to publicize the approach and
  perhaps motivate interested hackers to take the code and develop it
  further into a proper tool.


Mutaml 0.1
══════════

  Archive: <https://discuss.ocaml.org/t/ann-mutaml-0-1/12639/2>


Jan Midtgaard announced
───────────────────────

  I'm happy to share news of the Mutaml 0.3 release! :tada:
  <https://github.com/jmid/mutaml/releases/tag/0.3>

  Together with the recent 0.2 release, this brings Mutaml up to speed
  with recent ppxlib releases and addresses a few issues reported by
  brave, early users:

  • Avoid mutations in attribute parameters #29
  • Avoid polymorphic equality which is incompatible with Core #30
  • Add support for ppxlib.0.28 and above #27
  • Avoid triggering 2 mutations of a pattern incl. a `when'-clause
    causing a redundant sub-pattern warning #22, #23

  Happy testing! :smiley:


ocaml-activitypub
═════════════════

  Archive: <https://discuss.ocaml.org/t/ann-ocaml-activitypub/15420/1>


Zoggy announced
───────────────

  I'm glad to announce a first release of `activitypub*' packages,
  implementing (well, trying to implement some flavor of) both
  [server-to-server] and [client-to-server] activitypub protocols.

  Documentation is available from the [web site].

  The `activitypub_server' package installs [TAPS], an experimental
  server, handling some common activities. Accounts hosted by this
  server can at least follow and be followed by mastodon instances, and
  post and receive activities (Create, Announce, Like, Undo, …).

  The library of the `activitypub_client' package can be used by a
  client application to post and receive activities to and from a server
  (though it was only tested with TAPS). See a simple example [here].

  A GUI client (based on [Stk] is installed by the `activitypub_gui'
  package. It requires a client configuration file as described
  [here]. You can drop IRIs/URLs of an actor in the window to open a tab
  and be able to follow this actor. The GUI also allows to create and
  post notes with attachments. This client is still very experimental
  and will be developed more in the future.

  EDIT: the package should be available soon in opam.


[server-to-server]
<https://www.w3.org/TR/activitypub/#server-to-server-interactions>

[client-to-server]
<https://www.w3.org/TR/activitypub/#client-to-server-interactions>

[web site] <https://zoggy.frama.io/activitypub/>

[TAPS] <https://zoggy.frama.io/activitypub/doc-taps.html>

[here] <https://zoggy.frama.io/activitypub/doc-client-example.html>

[Stk] <https://zoggy.frama.io/ocaml-stk/>

[here]
<https://zoggy.frama.io/activitypub//refdoc/activitypub_client/Activitypub_client/Main/index.html>


Ortac/QCheck-STM 0.4.0 Dynamic formal verification beyond one system under test
═══════════════════════════════════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/ann-ortac-qcheck-stm-0-4-0-dynamic-formal-verification-beyond-one-system-under-test/15427/1>


Nicolas Osborne announced
─────────────────────────

  I'm very pleased to announce this exciting new release of
  `ortac-qcheck-stm.0.4.0'!

  This new release brings some exciting new features, mostly the result
  of Nikolaus Huber's contributions! Thank you Nik!

  Ortac/QCheck-STM is a test generator based on the [QCheck-STM]
  model-based testing framework and the [Gospel] specification language
  for OCaml.

  You can find the project on [this repo] and install the released
  packages via `opam'.

  It is also encourage to install `ortac-dune' to avoid having to write
  too much dune boilerplate.

  In particular, this release extend Ortac/QCheck-STM so that the
  generated tests will include functions that can take multiple
  System-Under-Tests as argument and/or that can return one. So now, if
  we write Gospel specifications for `append'-like functions,
  Ortac/QCheck-STM will include them in the generated tests!

  Happy testing!


[QCheck-STM] <https://github.com/ocaml-multicore/multicoretests>

[Gospel] <https://github.com/ocaml-gospel/gospel>

[this repo] <https://github.com/ocaml-gospel/ortac>


Openbsd 1.0
═══════════

  Archive: <https://discuss.ocaml.org/t/ann-openbsd-1-0/15434/1>


Sebastien Marie announced
─────────────────────────

  I would like to announce a new (somehow niche) package [Openbsd],
  which provides bindings for some specifics OpenBSD syscalls
  [pledge(2)] and [unveil(2)].

  These syscalls lets the kernel OS to know what the running processus
  is expected to do, and so it is possible to restrict a processus to do
  only filesystem or only network or only pure computation…

  The package is designed to be installable on any platform and provides
  simple method to check if `Pledge' or `Unveil' are supported. This
  way, it is possible to easily write portable code using the package,
  as it could be a turned on "no-operation" on Windows or Linux hosts
  (or provides alternative code path for sandboxing).

  • Homepage : [https://codeberg.org/semarie/ocaml-openbsd/]
  • License : [ISC]
  • Documented Interface : [lib/openbsd.mli]


[Openbsd] <https://ocaml.org/p/openbsd/latest>

[pledge(2)] <https://man.openbsd.org/pledge.2>

[unveil(2)] <https://man.openbsd.org/unveil.2>

[https://codeberg.org/semarie/ocaml-openbsd/]
<https://codeberg.org/semarie/ocaml-openbsd/>

[ISC] <https://en.wikipedia.org/wiki/ISC_license>

[lib/openbsd.mli]
<https://codeberg.org/semarie/ocaml-openbsd/src/tag/1.0/lib/openbsd.mli>

Examples
╌╌╌╌╌╌╌╌

  ┌────
  │ let open Openbsd in
  │ if Pledge.supported then
  │   Pledge.promises "stdio rpath"
  └────

  ┌────
  │ let open Openbsd in
  │ if Unveil.supported then (
  │   Unveil.add "./lib" "r";
  │   Unveil.add "./logs" "rwc";
  │   Unveil.lock ())
  └────


Compsort - reorder files in archives for improved compression
═════════════════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/ann-compsort-reorder-files-in-archives-for-improved-compression/15436/1>


adrien announced
────────────────

  I'm happy to announce the first release of compsort, a tool to reorder
  files in an archive for better compression. It works by grouping files
  according to a distance that is computed between every file pair. You
  can install it with `opam install compsort' (requires ocaml 5.2 for
  parallelism).

  Website with more details and examples in `README.md', plus source:
  <https://gitlab.com/adrien-n/compsort/>

  The goal is not new but, AFAIK, the approach is. I am very interested
  in prior or related art!


Results
╌╌╌╌╌╌╌

  Compsort achieves improvements that would typically require larger
  compression windows and therefore more memory. The improvements are
  only a few percents but in the domain of compression, a few percents
  is actually a lot.

  With `xz' compression, a Ubuntu initrd on my machine is reduced by
  more than 11.5%, while the best achievable improvement is 12.7% (the
  reordering gives 90% of the best result). Similarly, the tree of
  `linux-firmware.git' can be compressed 6% better, while the best
  achievable improvement is 9.4% (the reordering gives 63% of the best
  result).


Visualizations
╌╌╌╌╌╌╌╌╌╌╌╌╌╌

  In order to better explain what it does, I quite like the
  visualizations I have so far (there will be better ones in the
  future), where the value of the pixel at `(x,y)' indicates how similar
  are files `x' and `y'.

  Before:

  <https://gitlab.com/adrien-n/compsort/-/raw/main/doc/img/bettercomp_python3-django-horizon_noop.png>

  After reordering:

  <https://gitlab.com/adrien-n/compsort/-/raw/main/doc/img/bettercomp_python3-django-horizon_buckets.png>

  Files that are very different from others are all packed at the end
  and there's also an isolated cluster of files together similar but
  different from everything else.  One can also see that the distinct
  row and column pattern from the first picture has disappeared: it
  indicated that every 15 or so files in that region were similar and
  were separated by disimilar files but that they're now grouped.

  While there are certainly improvements possible, results are
  good. It's a case where one might wonder why results are so good
  considering all the approximations that took place.

  [1] Most of the algorithms/libraries I've tried to use rely on having
  an actual proper distance function which I don't have


Future work
╌╌╌╌╌╌╌╌╌╌╌

  I'll try to improve the distance function. Currently it does some
  steps of compression algorithms to detect redundancies but maybe
  reusing a compression library would give better results if it can be
  made fast enough (lz4 is borderline but it has tiny dictionaries
  unfortunately).

  Clustering could be better as the current algorithm is very basic (it
  collects files that are 90% similar, then 80% similar, then 70%, …). I
  tried several algorithms but I don't have a good-enough distance
  function (for instance the triangular inequality probably doesn't
  hold) and results were worse.

  All this will benefit from better visualizations and I'd like to have
  interactive plots that can be hovered on with the mouse to get the
  distance value and full file name.

  Oh and code isn't always pretty as it went through a lot of
  experimental stages and low-level tweaks to improve performance.


Dune dev meeting
════════════════

  Archive: <https://discuss.ocaml.org/t/ann-dune-dev-meeting/14994/13>


Etienne Marais announced
────────────────────────

  We will hold our regular Dune dev meeting tomorrow, on _Wednesday,
  October, 16th at 10:00 CET_. As usual, the session will be one hour
  long.

  Whether you are a maintainer, a regular contributor, a new joiner or
  just curious, you are welcome to join: these discussions are opened!
  The goal of these meetings is to provide a place to discuss the
  ongoing work together and synchronise between the Dune developers !


Agenda
╌╌╌╌╌╌

  The agenda is available on the[ meeting dedicated page]. Feel free to
  ask if you want to add more items in it.


[ meeting dedicated page]
<https://github.com/ocaml/dune/wiki/dev-meeting-2024-10-16>


Links
╌╌╌╌╌

  • Meeting link:[ zoom]
  • Calendar event:[ google calendar]
  • Wiki with information and previous notes:[ GitHub Wiki]


[ zoom]
<https://us06web.zoom.us/j/85096877776?pwd=cWNhU1dHQ1ZNSjZuOUZCQ0h2by9Udz09>

[ google calendar]
<https://calendar.google.com/calendar/embed?src=c_5cd698df6784e385b1cdcdc1dbca18c061faa96959a04781566d304dc9ec7319%40group.calendar.google.com>

[ GitHub Wiki] <https://github.com/ocaml/dune/wiki#dev-meetings>


Other OCaml News
════════════════

From the ocaml.org blog
───────────────────────

  Here are links from many OCaml blogs aggregated at [the ocaml.org
  blog].

  • [The Uncertain Art of Accelerating ML Models with Sylvain Gugger]
  • [Dune Package Management: Revolutionising OCaml Development]


[the ocaml.org blog] <https://ocaml.org/blog/>

[The Uncertain Art of Accelerating ML Models with Sylvain Gugger]
<https://signals-threads.simplecast.com/episodes/the-uncertain-art-of-accelerating-ml-models-with-sylvain-gugger-moYuL4Ps>

[Dune Package Management: Revolutionising OCaml Development]
<https://tarides.com/blog/2024-10-09-dune-package-management-revolutionising-ocaml-development>


Old CWN
═══════

  If you happen to miss a CWN, you can [send me a message] and I'll mail
  it to you, or go take a look at [the archive] or the [RSS feed of the
  archives].

  If you also wish to receive it every week by mail, you may subscribe
  to the [caml-list].

  [Alan Schmitt]


[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <https://alan.petitepomme.net/cwn/>

[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>

[caml-list] <https://sympa.inria.fr/sympa/info/caml-list>

[Alan Schmitt] <https://alan.petitepomme.net/>


[-- Attachment #2: Type: text/html, Size: 26875 bytes --]