[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

Alan Schmitt
2020-09-22T07:27:28Z

[-- Attachment #1: Type: text/plain, Size: 15579 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of September 15 to
22, 2020.

Table of Contents
─────────────────

Liquidsoap 1.4.3
Simple63 v1: compression of integer sequences
bentov v1: streaming estimation of 1D histograms
opam-compiler 0.1.0
lua_parser 1.0.0
Merlin 3.4.0 : introducing external configuration readers
gRPC server and client in OCaml
Bitstring (and ppx_bitstring) 4.0.0
Old CWN

Liquidsoap 1.4.3
════════════════

Archive: <https://discuss.ocaml.org/t/ann-liquidsoap-1-4-3/6429/1>

Romain Beauxis announced
────────────────────────

I'm happy to announce that liquidsoap `1.4.3' is out at:
<https://github.com/savonet/liquidsoap/releases/tag/v1.4.3>

This is the 3rd bugfix release for the `1.4.x' branch. It contains
important fixes and a couple of new minor features. Update is
recommended and should be fairly safe.

Along we this release, we have now added builds for `arm64' debian
packages and docker-ready production images for `amd64' and `arm64'
architectures available at:
<https://hub.docker.com/repository/docker/savonet/liquidsoap>

Again, we would like to warmly thank all users, contributors and
reporters for helping us bring liquidsoap to the next step!

Also, please note that a couple of issues had to be left out to make
sure that the release comes out on time. These are listed [here] and
will be tackled as soon as possible.

Next for liquidsoap, we will focus on getting the current `2.x' branch
finalized and polished. We already have support for encoded content
and ffmpeg raw frames. We need to write a couple of inline encoders
and decoders and we'll have 90% of the features ready. This will be a
major update for us!

[here] <https://github.com/savonet/liquidsoap/milestone/7>

Simple63 v1: compression of integer sequences
═════════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-simple63-v1-compression-of-integer-sequences/6431/1>

Mika Illouz announced
─────────────────────

This is to announce Simple63, an opam package for compression of
integer sequences; similar to Anh and Moffat's Simple-8b. More details
found in:

• github: [https://github.com/barko/simple63]
• documentation: [https://barko.github.io/simple63/]

Feedback and bug reports welcome.

[https://github.com/barko/simple63] <https://github.com/barko/simple63>

[https://barko.github.io/simple63/] <https://barko.github.io/simple63/>

bentov v1: streaming estimation of 1D histograms
════════════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-bentov-v1-streaming-estimation-of-1d-histograms/6434/1>

Mika Illouz announced
─────────────────────

This is to announce bentov, a opam package that implements a 1D
histogram-sketching algorithm. For more details:

• github: [https://github.com/barko/bentov]
• documentation: [https://barko.github.io/bentov]

Feedback and bug reports welcome.

[https://github.com/barko/bentov] <https://github.com/barko/bentov>

[https://barko.github.io/bentov] <https://barko.github.io/bentov>

opam-compiler 0.1.0
═══════════════════

Archive: <https://discuss.ocaml.org/t/ann-opam-compiler-0-1-0/6442/1>

Etienne Millon announced
────────────────────────

On behalf of the opam maintainers, I'd like to announce the first
release of opam-compiler, a plugin to work with compiler variants,
branches and forks.

This can cover a pretty wide range of use cases, so the first version
is starting small with a single command to create a switch from a
branch or github PR:

┌────
│ % opam compiler create '#9921'
│ Opam plugin "compiler" is not installed. Install it on the current switch? [Y/n] y
│
│ ...
│
│ <><> Carrying on to "opam compiler create #9921" ><><><><><><><><><><><><><><><>
│
│ [ocaml-variants.4.12.0+trunk+no-flat-float-array] synchronised from
│ git+https://github.com/gasche/ocaml#Atomic.create
│ ocaml-variants is now pinned to git+https://github.com/gasche/ocaml#Atomic.create (version
│ 4.12.0+trunk)
│ % opam switch
│ ...
│ → ocaml-ocaml-9921
│ [opam-compiler] ocaml/ocaml#9921 - stdlib: rename Atomic.make into Atomic.create
└────

You can also override the arguments passed to `--configure'.

As you can see in the above snippet, it's an opam plugin so it will
auto-install if needed (assuming you ran `opam update' recently) and
will be available across all switches. Its sources and issue tracker
are available [here].

For the next releases, our plan is to add a user-friendly way to setup
a switch based on a local git clone, so that it's easy to test your
compiler fork with opam packages. You can find the other features we'd
like to add in the future in [the relevant part of the opam roadmap].

Thanks and have fun compiling compilers!

[here] <https://github.com/ocaml-opam/opam-compiler>

[the relevant part of the opam roadmap]
<https://github.com/ocaml/opam/wiki/Spec-for-working-with-the-OCaml-compiler>

lua_parser 1.0.0
════════════════

Archive: <https://discuss.ocaml.org/t/ann-lua-parser-1-0-0/6445/1>

Jason Nielsen announced
───────────────────────

I've release [lua_parser] which is now up on opam. It is a parser and
pretty-printer for lua 5.2. Actually it was developed with luajit in
mind which is lua 5.1 plus goto/labels (which syntactically for the
purposes of parsing and pretty-printing is lua 5.2).

[lua_parser] <https://github.com/drjdn/ocaml_lua_parser>

Merlin 3.4.0 : introducing external configuration readers
═════════════════════════════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-merlin-3-4-0-introducing-external-configuration-readers/6446/1>

vds announced
─────────────

I am glad to announce, on behalf of the Merlin team, the release of
Merlin `3.4.0' which brings some major changes in the way
configuration is handled.

As you might know, Merlin reads its configuration from the closest
`.merlin' file to the source file being edited. These files tell
merlin where to find other source files and build artifacts, but also
which flags should be passed to the compiler, which syntax extensions
are enabled and which packages are used by the project.

In this setting the configuration is the same for all the source files
of a folder, regardless of their specificities. In other words, the
configuration loaded for a single source file contains the union of
the dependencies of this file and of all its siblings which is not an
optimal behavior.

Starting with version `3.4.0' merlin will ship with two packages:
`merlin' and `dot-merlin-reader' which, as the name suggests, reads
configuration from `.merlin' files. Both are necessary for proper
function.

When a `.merlin' file is present in the source folder the Merlin
server will start a `dot-merlin-reader' process and communicate with
it via standard input and output following a simple protocol. These
processes are halted with the server.

*This change should not have any visible impact on users' workflows as
long as the `dot-merlin-reader' binary is correctly installed and in
the path*. (which should be the case in opam-based setups)

This change in itself will not solve the granularity problem mentioned
earlier, but it paves the way for such improvements: in a near-future
Dune will stop generating `.merlin' files and Merlin will obtain
file-based configuration directly from the build system using the same
protocol as the one used by `dot-merlin-reader'.

Changelog
╌╌╌╌╌╌╌╌╌

⁃ merlin binary
• fix completion of pattern matchings with exception patterns
(#1169)
• delegate configuration reading to external programs via a simple
protocol and create a new package `dot-merlin-reader' with a
binary that reads `.merlin' files. (#1123, #1152)

gRPC server and client in OCaml
═══════════════════════════════

Archive:
<https://discuss.ocaml.org/t/grpc-server-and-client-in-ocaml/6465/1>

blandinw announced
──────────────────

TL;DR <https://github.com/blandinw/ocaml-grpc-envoy/>

Hey, I'm new to OCaml after writing some Clojure, C++ and Haskell in
various contexts, including working at FB (relevant below).

After browsing this forum and Reddit for a bit, the assumption seems
to be that OCaml is not a good fit for gRPC, since there's no pure
implementation today. Now, this is something I have experience with,
so I thought I'd try and challenge this assumption.

As you may know, services inside FB use Thrift (both the format and
protocol) to communicate. The Thrift team worked primarily in C++ (for
good reasons), causing support for other languages to lag behind
despite their best efforts. Now, the interchange format (equivalent to
Protobuf) does not change very often so it's fine to have a
per-language implementation, but the client and server (equivalent to
HTTP2 + gRPC) frequently receive new features, optimizations and
fixes. After a valiant and continued effort to support most languages
used internally, the Thrift team came up with an idea. Instead of
maintaining multiple implementations and dealing with obscure FFI
bugs, ~FingerprintTrustManagerFactory~s and whatnot, they would focus
solely on the C++ implementation and provide a daemon to be ran
alongside whatever code you were trying to run. You could then use
simple IPC to exchange Thrift (the format) messages with that daemon,
and it would handle all the nitty-gritty of running a service at scale
(load balancing, connection pooling, service discovery, security,
retries, timeouts, network stats, hot restarts, etc.). Needless to
say, it worked remarkably well even at very high scale and everybody
was much happier.

I wanted to replicate this idea with OCaml and gRPC. We already have
support for protobuf thanks to the excellent `ocaml-protoc'. All we
need is a way to exchange protobuf messages reliably on the wire.
Instead of having an OCaml implementation that will have to stay
up-to-date and have its own set of bugs (the official `grpc/grpc-java'
repo has 4450 commits and 2400 issues at the moment), can we reuse
existing infra with already massive support and production time?

Fortunately, the people at Lyft built just that, open-sourced it and
contributed it to the Cloud Native Computing Foundation in late
2017. It is called Envoy and it is bliss.

I demonstrate how to fit these pieces together at
[blandinw/ocaml-grpc-envoy] to build a simple KV store, including a
gRPC client and server in 200 lines of OCaml code. The idea is to
spawn an Envoy process that will handle all gRPC communication for our
OCaml code. We use HTTP/1.1 to exchange Protobuf messages with it,
using for example `httpaf' and `Lwt'. This solution has the added
benefit that it is highly scalable from the start, allowing you for
instance to spawn one OCaml process per core and load balance between
them. You can also use Envoy (with proper config!) as your web reverse
proxy instead of say, nginx.

At the very least, this solution allows us to start writing gRPC code
today, and gracefully evolve towards HTTP/2, Multicore and maybe a
native OCaml implementation later.

I'm curious to hear your perspective on the future of building
services with OCaml, or your past experience like what went well, what
was missing, etc.

[blandinw/ocaml-grpc-envoy]
<https://github.com/blandinw/ocaml-grpc-envoy/>

Yawar Amin asked and blandinw replied
─────────────────────────────────────

Fantastic idea. So if I understand correctly, the only
thing that Envoy (server-side) is doing is translating the
Protobuf from gRPC HTTP2 transport to HTTP1, and
forwarding these Protobuf objects over HTTP1 to the OCaml
server? Envoy doesn't know to know about the actual gRPC
schema, because it doesn't touch the Protobuf objects
themselves, right?

That's correct. Envoy is only concerned with transporting bytes (along
with load balancing, routing, etc, etc). Only OCaml knows about the
Protobuf schemas.

In the OCaml server case, Envoy listens for HTTP/2 gRPC requests,
accesses the bytes payload with no knowledge of the actual
schema/layout and repackages these same bytes in a HTTP/1.1 request
that OCaml can process. OCaml then responds with bytes (an encoded
Protobuf response message) that Envoy sends back on the original HTTP2
connection.

Bitstring (and ppx_bitstring) 4.0.0
═══════════════════════════════════

Archive:
<https://discuss.ocaml.org/t/ann-bitstring-and-ppx-bitstring-4-0-0/6471/1>

xrguerin announced
──────────────────

Features
╌╌╌╌╌╌╌╌

• Add support for let bindings introduced in 4.08
• Switch to PPXLIB

Deprecations
╌╌╌╌╌╌╌╌╌╌╌╌

As PPXLIB requires `ocaml >= 4.04' support for earlier versions has
been dropped.

Breaking changes
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌

This release splits the library from the PPX to reduce runtime
dependencies. Projects using the PPX from bitstring will need to also
depends on ppx_bitstring from now on.

Rudi Grinberg added
───────────────────

The project is hosted [here] for those who are interested.There's also
some excellent [docs]

[here] <https://github.com/xguerin/bitstring>

[docs] <https://bitstring.software/documentation/>

Old CWN
═══════

If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].

If you also wish to receive it every week by mail, you may subscribe
[online].

[Alan Schmitt]

[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <http://alan.petitepomme.net/cwn/>

[RSS feed of the archives] <http://alan.petitepomme.net/cwn/cwn.rss>

[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>

[Alan Schmitt] <http://alan.petitepomme.net/>

[-- Attachment #2: Type: text/html, Size: 28846 bytes --]