[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

Alan Schmitt
2021-01-12T09:47:37Z

[-- Attachment #1: Type: text/plain, Size: 19040 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of January 05 to 12,
2021.

Table of Contents
─────────────────

Marshal determinism and stability
Sedlex + Menhir parser for both tty and file parsing
First release of awa-ssh
Introducing Feather: A lightweight shell-scripting library for OCaml
postdoc researcher and research engineer positions for CHERI and Arm verification
First ocb (OCaml Badgen) release
Release of OCaml-Git v3.0 and co
Other OCaml News
Old CWN

Marshal determinism and stability
═════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/marshal-determinism-and-stability/7041/28>

Continuing this thread, David Allsopp said
──────────────────────────────────────────

A couple of notes on `Marshal', which I don't think have been covered
• Although the guarantee is only between identical versions of OCaml,
the implementation actually goes to considerable lengths to maintain
backwards compatibility (so a value _written_ by older OCaml remains
_readable_ in newer OCaml). Our own testsuite, for example,
indirectly [includes a test which unmarshals a 3.12.1 value]. I
don't know exactly how far back the support goes.
• As it happens, the change which affected Unison in 4.08 was the
first breaking change to Marshal since either 4.00 or 4.01. The fact
that it doesn't break often (and that the two code paths - at least
at present - are small) meant I have suggested a few months back
that we could in future add an additional flag in the style of
`Compat_32' to allow values to be written in a way which should be
readable on older versions of OCaml. Indeed, it's small enough that
flags could be added for the changes in 4.08 ([PR#1683]) and in 4.11
([PR#8791]).
• Neither point undermines using alternative formats either for
network serialisation or persistent storage, for the many reasons
discussed above!

[includes a test which unmarshals a 3.12.1 value]
<https://github.com/ocaml/ocaml/blob/trunk/testsuite/tests/lib-hashtbl/compatibility.ml>

[PR#1683] <https://github.com/ocaml/ocaml/pull/1683>

[PR#8791] <https://github.com/ocaml/ocaml/pull/8791>

Sedlex + Menhir parser for both tty and file parsing
════════════════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/sedlex-menhir-parser-for-both-tty-and-file-parsing/7055/1>

Bernard Sufrin announced
────────────────────────

I am a great fan of Menhir, and have used it in several private
language projects, using the ulexing scanner generator to provide
Unicode-capable scanners.

Alarmed by the obsolescence of ulexing, and needing a utf8-capable
scanner in a hurry I decided to (teach myself to) use Sedlex. On the
whole the experience was very satisfactory, and I found it
straightforward to produce a variant of the sedlexing library which
supports buffers with variable chunk sizes, thereby enabling efficient
lexing on channels connected to files as well as immediately
responsive lexing on channels connected to terminals.

I also wanted to teach myself how to use the error-reporting,
incremental, interfaces to Menhir-generated parsers. In the hope that
it might be useful to others facing the same learning task, or the
problem of adapting Sedlex for efficient interactive use, I have
placed the example mock-S-Expression parser that resulted from this
excursion in:

[Git Repository: github.com/sufrin/InteractiveSedlexMenhirExample]

[Git Repository: github.com/sufrin/InteractiveSedlexMenhirExample]
<https://github.com/sufrin/InteractiveSedlexMenhirExample>

First release of awa-ssh
════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-first-release-of-awa-ssh/7057/1>

Hannes Mehnert announced
────────────────────────

I'm happy to announce that `awa-ssh'
(<https://github.com/mirage/awa-ssh>) has just been merged into
opam-repository. It is a pure OCaml implementation of the ssh (secure
shell, <https://en.wikipedia.org/wiki/SSH_(Secure_Shell)>) protocol.

This is the initial release, please report issues you encounter.

It was initially developed by Christiano Haesbaert in 2016, and
revived mid-2019 by myself and in 2020 it was migrated to the MirageOS
organization on GitHub for further development and maintenance.

Both client and server code are present in the library (pure code in
the main awa package), though the awa-lwt package implements only a
server, and the awa-mirage package implements only a client. Tests and
examples are in the test subdirectory.

The MirageOS client has been successfully used to clone git
repositories (on private servers, on GitHub, etc.). It supports apart
from RSA keys also ED25519 keys (and key exchanges).

Introducing Feather: A lightweight shell-scripting library for OCaml
════════════════════════════════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/introducing-feather-a-lightweight-shell-scripting-library-for-ocaml/7059/1>

Charles announced
─────────────────

I wrote a shell scripting library called [Feather]. I like idea of
writing bash-like code quickly, later being able to intersperse OCaml
to add more typeful components as needed. It's kind of like [Shexp]
but without the monadic interface and with Async
support. ([Feather_async])

There's a tutorial and some examples in the link above but here's a
quick taste:

┌────
│ open Feather
│
│ let lines = find "." ~name:"*.ml"
│ |. tr "/" "\n"
│ |. map_lines ~f:String.capitalize
│ |. sort
│ |. process "uniq" [ "-c" ]
│ |. process "sort" [ "-n" ]
│ |. tail 4
│ |> collect_lines
│ in
│ String.concat ~sep:", " lines |> print_endline
└────
Let me know if you have any feedback! And feel free to file bug
reports [here]. Hope it ends up being useful, entertaining, or both!

[Feather] <https://hg.sr.ht/~etc/feather>

[Shexp] <https://github.com/janestreet/shexp/>

[Feather_async] <https://hg.sr.ht/~etc/feather_async>

[here] <https://todo.sr.ht/~etc/feather>

postdoc researcher and research engineer positions for CHERI and Arm verification
═════════════════════════════════════════════════════════════════════════════════

Archive:
<https://sympa.inria.fr/sympa/arc/caml-list/2021-01/msg00023.html>

Peter Sewell announced
──────────────────────

We are looking for postdoctoral researchers and postdoctoral or
postgraduate research engineers to help develop semantics and
verification to improve the foundations and security of mainstream
computer systems, for CHERI and Arm system software verification, at
the University of Cambridge. OCaml expertise to help develop
verification tools will be especially welcome. Closing date 13 January
2021 - see the advert <http://www.jobs.cam.ac.uk/job/28012/>.

First ocb (OCaml Badgen) release
════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-first-ocb-ocaml-badgen-release/7073/1>

zapashcanon announced
─────────────────────

A few days ago, I released [ocb]. It's a library and a command-line
tool to generate SVG badges.

To get started quickly:

┌────
│ ocb --label Hello --color green --style flat --labelcolor white --status Goodbye
└────

Will gives this result: [SVG example].

My first use case was [To.ml] where I'm using [bisect_ppx] to generate
and deploy a [coverage report]. I wanted to display the coverage
percentage in the README and tried existing tools but wasn't fully
satisfied as they didn't work or were failing randomly. Now, [I'm
generating the badge directly in a GitHub action].

The project was inspired by [badgen]. I still have to add support for
icons and to improve the documentation but it's usable.

[ocb] <https://github.com/ocamlpro/ocb>

[SVG example]
<https://raw.githubusercontent.com/OCamlPro/ocb/master/example/cli.svg>

[To.ml] <https://github.com/ocaml-toml/To.ml>

[bisect_ppx] <https://github.com/aantron/bisect_ppx>

[coverage report] <https://ocaml-toml.github.io/To.ml/coverage/>

[I'm generating the badge directly in a GitHub action]
<https://github.com/ocaml-toml/To.ml/blob/6ac580848ad1d34ec3032da8672bbd9aca203cc4/.github/workflows/deploy.yml#L34>

[badgen] <https://github.com/badgen/badgen>

Release of OCaml-Git v3.0 and co
════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-release-of-ocaml-git-v3-0-and-co/7076/1>

Ulugbek Abdullaev announced
───────────────────────────

We, the [`ocaml-git'] team, are happy to announce a new major release
of `ocaml-git v3.0' and related libraries.

[`ocaml-git'] <https://github.com/mirage/ocaml-git>

Release Notes
╌╌╌╌╌╌╌╌╌╌╌╌╌

OCaml-Git v3.0
┄┄┄┄┄┄┄┄┄┄┄┄┄┄

[*OCaml-Git*] is a library that implements `git' format and protocol
implementation in pure OCaml. The library is used by libraries such as
[`irmin'], a git-like distributed database, or [`pasteur'], a MirageOS
unikernel-based snippet storage service.

[*OCaml-Git*] <https://github.com/mirage/ocaml-git>

[`irmin'] <https://github.com/mirage/irmin>

[`pasteur'] <https://github.com/dinosaure/pasteur>

Changes
┈┈┈┈┈┈┈

The main goal behind this major release was to get better
compatibility with various platforms, including
[~MirageOS~](mirage.io), 32-bit platforms, and `js_of_ocaml'. In order
to achieve that, we broke down `ocaml-git' into several components,
which are represented as sub-libraries. We will describe some of those
components later in this post.

Along with better support for various platforms, `ocaml-git 3.0' also
comes with SSH support for `fetch/push' and various bug fixes.

The rest of the changes are mostly internal and pave a way for
interesting features such as a full-blown `git' [garbage collector]
and wire protocol v2 ([announcment] and [spec]).

*References:*

• Full [changes list]
• [PR] that introduced the major rewrite of `ocaml-git'

—

In the new version of `ocaml-git', we try to have better separation of
concerns by breaking some of the `ocaml-git' components into
sub-libraries, which do not contain `git'-specific logic and can be
reused for other purposes.

[garbage collector] <https://git-scm.com/docs/git-gc>

[announcment]
<https://opensource.googleblog.com/2018/05/introducing-git-protocol-version-2.html>

[spec]
<https://github.com/git/git/blob/master/Documentation/technical/protocol-v2.txt>

[changes list]
<https://github.com/mirage/ocaml-git/blob/master/CHANGES.md>

[PR] <https://github.com/mirage/ocaml-git/pull/395>

Carton
┄┄┄┄┄┄

Git uses [PACK files] to store old git objects such as commits and
transfer objects over wire using git's wire protocols (`git-nss'
library mentioned below implements [v1] of the protocol; [v2]
implementation is in progress).

[*Carton*] is a library to work with PACK files. The library does not
contain git-specific code, so one can easily reuse the library and
PACK format for non-git objects. One can see how `ocaml-git' uses
`carton' for its purposes [here].

*References:*

• [PR] that introduces `carton'

[PACK files]
<https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt>

[v1]
<https://github.com/git/git/blob/master/Documentation/technical/pack-protocol.txt>

[v2]
<https://github.com/git/git/blob/master/Documentation/technical/protocol-v2.txt>

[*Carton*] <https://github.com/mirage/ocaml-git/tree/master/src/carton>

[here] <https://github.com/mirage/ocaml-git/tree/master/src/carton-git>

[PR] <https://github.com/mirage/ocaml-git/issues/375>

Git-NSS (Not So Smart)
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

When one wants to synchronize with a remote repository using git, they
need to use `git fetch/push'. Communication and
synchronization/negotiation is defined by git *wire protocol*, which
has two versions: older version 1 and newer leaner version 2. The
protocols are defined for four wire transports: HTTP(S), SSH, and
`git://' (TCP).

[`Not-So-Smart'] library is a library that allows for such
synchronization based on the git wire protocols but without
git-specific code, meaning that files being fetched do not need to be
git objects or that there is no assumptions on the "repository" that
one is synchronizing with. So, as well as `carton', the library aims
to be reusable for other purposes.

This release features support for SSH using [awa-ssh] by @hannesm (see
[the release]), support for [partial-clone] (of various `depth'), and
memory consumption fixes for unikernels.

*Note 1:* The library's name "Not so smart" is a play on the git's
"smart" protocol, a part of wire protocol v1 over HTTP(S) transport.

*Note 2:* only client side logic is implemented for wire
protocols. The server-side is planned but not yet implemented. One can
use `git' as the server for now.

[`Not-So-Smart']
<https://github.com/mirage/ocaml-git/tree/master/src/not-so-smart>

[awa-ssh] <https://github.com/mirage/awa-ssh>

[the release]
<https://discuss.ocaml.org/t/ann-first-release-of-awa-ssh/7057>

[partial-clone] <https://git-scm.com/docs/partial-clone>

Mimic
┄┄┄┄┄

[*Mimic*] is a small reimplementation of [`conduit'], a library that
helps to abstract over a transport protocol such as HTTP(S) or SSH. In
other words, the code using `mimic' can deal not with different types
that represent an HTTP or SSH connection, but just deal, e.g., read
from or write to, with a `flow' value, which hides protocol-specific
details under its hood.

—

There are several independent libraries that were upgraded along with
`ocaml-git 3.0'.

[*Mimic*] <https://github.com/mirage/ocaml-git/tree/master/src/mimic>

[`conduit'] <https://github.com/mirage/ocaml-conduit>

Duff v0.3
┄┄┄┄┄┄┄┄┄

[*Duff*] is a library that implements git's [`libXdiff'] (`xdiff'
algorithm) in OCaml. PACK files use a binary diff algorithm, `xdiff',
to compress binary data. More on the project [page] and release
[notes] for `ocaml-git 2.0'.

[*Duff*] <https://github.com/mirage/duff>

[`libXdiff'] <http://www.xmailserver.org/xdiff-lib.html>

[page] <https://github.com/mirage/duff>

[notes] <https://discuss.ocaml.org/t/ann-ocaml-git-2-0/2740>

Changes
┈┈┈┈┈┈┈

This release fixes the support for 32-bit architecture platforms.

Encore v0.7
┄┄┄┄┄┄┄┄┄┄┄

[*Encore*] is a library that can create an encoder/decoder based on
the format given. It also ensures isomorphism by construction.

[*Encore*] <https://github.com/mirage/encore>

Changes
┈┈┈┈┈┈┈

Extensive changes to the API. See the project page.

Decompress v1.2.0
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

[*Decompress*] is an OCaml implementation of certain decompression
algorithms such as `Zlib', `Gzip', etc.

[*Decompress*] <https://github.com/mirage/decompress>

Changes
┈┈┈┈┈┈┈

`ocaml-git 3.0' uses new version of `decompress' with extensive
performance improvements documented in *Tarides's* blog [API changes]
and [performance improvements].

We'd be happy to get your feedback or questions! :-)

[API changes]
<https://tarides.com/blog/2019-08-26-decompress-the-new-decompress-api>

[performance improvements]
<https://tarides.com/blog/2019-09-13-decompress-experiences-with-ocaml-optimization>

Other OCaml News
════════════════

From the ocamlcore planet blog
──────────────────────────────

Here are links from many OCaml blogs aggregated at [OCaml Planet].

• [How We Lost at The Delphi Oracle Challenge]
• [Tarides sponsors the Oxbridge Women in Computer Science Conference
2020]
• [Coq 8.12.2 is out]
• [First release of MetAcsl plugin]
• [Announcing MirageOS 3.10]
• [ ReScript 8.4]
• [Coq 8.13+beta1 is out]

[OCaml Planet] <http://ocaml.org/community/planet/>

[How We Lost at The Delphi Oracle Challenge]
<https://seb.mondet.org/b/0010-delphi-challenge-post-vivum.html>

[Tarides sponsors the Oxbridge Women in Computer Science Conference
2020]
<https://tarides.com/blog/2020-12-14-tarides-sponsors-the-oxbridge-women-in-computer-science-conference-2020>

[Coq 8.12.2 is out] <https://coq.inria.fr/news/coq-8-12-2-is-out.html>

[First release of MetAcsl plugin]
<https://frama-c.com/fc-plugins/metacsl.html>

[Announcing MirageOS 3.10]
<https://mirage.io/blog/announcing-mirage-310-release>

[ ReScript 8.4]
<https://rescript-lang.org/blog/bucklescript-release-8-4>

[Coq 8.13+beta1 is out]
<https://coq.inria.fr/news/coq-8-13beta1-is-out.html>

Old CWN
═══════

If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].

If you also wish to receive it every week by mail, you may subscribe
[online].

[Alan Schmitt]

[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <http://alan.petitepomme.net/cwn/>

[RSS feed of the archives] <http://alan.petitepomme.net/cwn/cwn.rss>

[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>

[Alan Schmitt] <http://alan.petitepomme.net/>

[-- Attachment #2: Type: text/html, Size: 32830 bytes --]