[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

[Alan Schmitt <alan.schmitt@polytechnique.org>]

[-- Attachment #1.1.1: Type: text/plain, Size: 22620 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of November 26 
to
December 03, 2019.

Table of Contents
─────────────────

Irmin 2.0.0 release
How viable is delivering binaries linked to Cygwin to Windows 
customers?
Dune 2.0.0
Advanced C binding using ocaml-ctypes and dune
Upcoming breaking change in Base/Core v0.14
CI/CD Pipelines: Monad, Arrow or Dart?
Use of functors to approximate F# statically resolved type 
parameters
Old CWN


Irmin 2.0.0 release
═══════════════════

  Archive: 
  <https://discuss.ocaml.org/t/ann-irmin-2-0-0-release/4746/5>


Continuing this thread, samoht announced
────────────────────────────────────────

  And there is now a follow-up blog post, explaining how to use 
  the new
  GraphQL API available in Irmin2:
  <https://tarides.com/blog/2019-11-27-introducing-irmin-graphql>.


How viable is delivering binaries linked to Cygwin to Windows 
customers?
════════════════════════════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/how-viable-is-delivering-binaries-linked-to-cygwin-to-windows-customers/4775>


mbacarella asked
────────────────

  I’m in the early stages of planning a deliverable binary product 
  that
  will run on Linux, Mac and Windows.

  My brief sniff of the air around the OCaml ecosystem says I 
  should
  expect to target Cygwin to get Windows going (although there’s
  impressive work to get native Windows stuff done that can become 
  the
  preferred approach in a few years).

  My experience using Cygwin as an operating environment is that 
  it’s
  pretty darn sluggish compared to Linux on the same computer.

  Why is this? There’s an anecdote that says Cygwin can only fork 
  at
  about 30-50x a second on Windows, due to how it has to adapt it 
  to
  work within Windows’ task spawning model. (For contrast, Linux 
  can
  achieve thousands of forks per second if you play around with 
  it).

  I understand from another product developer that when they build
  binaries to deliver to Windows/Cygwin, they actually 
  cross-compile on
  Linux because of how slowly the toolchain runs on Cygwin.

  That sounds like bad news if you want to do UNIXy things, but 
  for a
  single standalone application this might not be so bad? I assume 
  if I
  ship a deliverable to Windows/Cygwin, the end user may enjoy 
  good
  performance, so long as I’m not spawning tons of processes or 
  relying
  on fork for multi-programming. Is this a safe assumptions?

  Any other gotchas when it comes to OCaml on Cygwin w.r.t. 
  performance?

  The app pretty much has real-time gaming requirements (though 
  it’s not
  a game so can side-step worrying about access to GPUs and
  what-not). Stated another way, although my application will 
  depend on
  the POSIX layer offered by Cygwin, I expect it not to crunch 
  POSIX
  related stuff in the main loop.

  How has your experience gone?


John Whitington replied
───────────────────────

  I have been shipping commercial binaries for Linux (32 and 64 
  bit),
  Windows (32 and 64bit) and OS X for years. For example:
  <https://github.com/coherentgraphics/cpdf-binaries>

  And even static or shared libraries in binary form:
  <https://github.com/coherentgraphics/cpdflib-binary>

  On OS X, you need to use MACOSX_DEPLOYMENT_TARGET or similar to 
  make
  sure your builds will run on older systems. And, in fact, you 
  need to
  use MACOSX_DEPLOYMENT_TARGET when asking OPAM to compile the 
  OCaml
  compiler itself. And, you will need to deal with codesigning and
  notarization. But it’s all doable.

  For linux, you may need to build under older linux versions, to 
  make
  sure that the glibc in use is old enough. This is not an
  ocaml-specific problem. I have a 64 bit and 32 bit VM with 
  old-ish
  glibc versions for this purpose.

  Under Windows, there are no such backward-compatibility 
  problems. I
  use the new OCaml for windows system, which comes with OPAM, and 
  is
  mingw-based. No cygwin remains in the final binary.

  For more obscure systems (AIX, HPUX, Sparc etc) customers 
  compile from
  source (with help from me). Not once in more than ten years has 
  anyone
  cared that it was written in OCaml.


dbuenzli also replied
─────────────────────

  remember that on the Windows native port, the Unix module 
  distributed
  with OCaml is your POSIX compatibility layer. There are a few 
  entry
  points to avoid though, the list is at the bottom of [this 
  page].


[this page] 
<https://caml.inria.fr/pub/docs/manual-ocaml/libunix.html>


nojb also replied
─────────────────

  At LexiFi our main application is developed and shipped on 
  Windows. We
  use the msvc port of OCaml. This means that you need Cygwin to
  develop, but the resulting application is fully native and does 
  not
  depend on the Cygwin DLL. As @dbuenzli mentioned, the Unix 
  module *is*
  the POSIX compatibility layer.

  Compilation speed is slower on Windows because process creation 
  is
  slower on Windows as a general rule, but it is manageable (our
  application has around 2000 modules + Js_of_ocaml + C bindings + 
  C#
  component).

  We don’t have any issues with runtime performance. The `Unix' 
  library
  mentioned above implements Windows support directly without 
  going
  through any compatibility layer and is quite efficient.


BikalGurung also replied
────────────────────────

  There is an editor being built in ocaml/reasonml which currently
  targets windows, linux and macos -
  <https://github.com/onivim/oni2>. However, the binary is native
  windows rather than cygwin derivative. So if you don’t have to 
  use
  cygwin dependencies then native windows binary could be the way 
  to go.

  Also esy - <https://github.com/esy/esy> makes developing
  ocaml/reasonml on windows viable.


keleshev also replied
─────────────────────

  *TLDR*: Install the [Mingw port of OCaml 4], freely use most 
  opam
   libraries, and compile to native Windows binaries, without 
   licensing
   issues.

  I recommend you read the “Release notes for Windows”:
  <https://github.com/ocaml/ocaml/blob/trunk/README.win32.adoc>

  To summarise, there are three Windows ports:

  • Native Microsoft port,
  • Native Mingw port,
  • Cygwin port.

  All three require Cygwin for development purposes. I recommend 
  using
  the Native Mingw, as:

  • it *doesn’t* require Visual Studio (it uses a mingw fork of 
  GCC that
    “cross-compiles” native Windows executables),
  • it *doesn’t* rely on the dreaded cygwin.dll
  • it has good opam support with opam-repository-mingw:
    <https://github.com/fdopen/opam-repository-mingw>
  • it has a convenient installer:
    <https://fdopen.github.io/opam-repository-mingw/> 5.

  To contrast, Native Microsoft requires Visual Studio, and 
  doesn’t have
  opam. You can still vendor pure OCaml packages, but as soon as 
  you
  want to use some C bindings you’re in trouble, because of the 
  “minor”
  differences between Visual C and GCC. And everything assumes GCC
  nowadays.

  Cygwin port is the one I don’t have experience with, but 
  re-reading
  the “Release notes for Windows” above it strikes me that it 
  mentions
  that Cygwin was re-licensed from GPL to LGPL with static linking
  exception. So it looks like the Cygwin port could be viable for
  commercial use, but I never tried to statically linked 
  `cygwin.dll',
  and I’m not sure what are the benefits of Cygwin port over the 
  Mingw
  port.


[Mingw port of OCaml 4]
<https://fdopen.github.io/opam-repository-mingw/>


dmbaturin also replied
──────────────────────

  With [soupault 4], I decided to ship prebuilt binaries for all
  platforms including Windows. Mostly to see if I can, all its 
  users I
  know of are on UNIX-like systems and know how to build from 
  source,
  but that’s beside the point. :wink:

  I can confirm everything @keleshev says: fdopen’s package just 
  works,
  opam works exactly like it does on UNIX, pure OCaml libraries 
  are
  trivial to install, and the binaries don’t depend on cygwin. 
  Note
  that “opam switch create” also just works, you can install 
  either
  MinGW or MSVC compiler versions as opam switches.  I only ever 
  start
  the Windows VM to make release builds, and the workflow is 
  exactly the
  same as on Linux where I’m actually writing code.

  My only obstacle on that path was that FileUtils lost its 
  Windows
  compatibility, but I wanted to use it, so I worked with 
  @gildor478 to
  make it cross-platform again. Uncovered a bug in the 
  implementation of
  Unix.utimes in the process, but it’s hardly a commonly used 
  function.

  You can also setup AppVeyor builds. It’s not as simple as I wish 
  it
  would be, but there are projects doing it that you can steal the 
  setup
  from.

  There’s also opam-cross-windows, but it’s very incomplete and 
  needs
  work to be practical. There are no big obstacles, it just needs
  work. While files in opam-repository-mingw are normally 
  identical to
  the default opam repository, the cross one needs small 
  adjustments in
  every package to specify the toolchain to use, so the required 
  work is
  mostly a lot of trivial but manual actions. I hope eventually it
  reaches parity with fdopen’s one and we’ll be able to easily 
  build for
  Windows without ever touching Windows.

  As of static Linux builds, @JohnWhitington’s approach can work, 
  but
  there’s a better option if you don’t need anything from glibc
  specifically and don’t link against any C libs: build statically 
  with
  musl. There’s a `+musl+static+flambda' compiler flavour. You 
  need musl
  and gcc-musl to install it, but after that, just build with 
  `-ccopt
  -static' flag and you get a binary that doesn’t depend on 
  anything.


[soupault 4] <https://soupault.neocities.org/>


Dune 2.0.0
══════════

  Archive: <https://discuss.ocaml.org/t/ann-dune-2-0-0/4758>


rgrinberg announced
───────────────────

  On behalf of the dune team, I’m delighted to announce the 
  release of
  dune 2.0. This release is the culmination of 4 months of hard 
  work by
  the dune team and contains new features, bug fixes, and 
  performance
  improvements . Here’s a selection of new features that I 
  personally
  find interesting:

  • New boostrap procedure that works in low memory environments
  • (`deprecated_library_name' ..) stanza to properly deprecate 
  old
    library names
  • (`foreign_library' ..) stanza to define C/C++ libraries.
  • C stubs directly in OCaml executables

  Refer to the change log for an exhaustive list.

  We strive for a good out of the box experience that requires no
  configuration, so we’ve also tweaked a few defaults. In 
  particular, `$
  dune build' will now build `@all' instead of `@install', and
  ocamlformat rules are setup by default.

  Lastly, dune 2.0 sheds all the legacy related to jbuilder and 
  will no
  longer build jbuilder projects. This change is necessary to ease
  maintenance and make it easier to add new features down the
  line. There are a few other minor breaking changes. Refer to the
  change log for the full list. We apologize in advance for any
  convenience this might cause.

  [Changelog]


[Changelog] <https://discuss.ocaml.org/t/ann-dune-2-0-0/4758>


Advanced C binding using ocaml-ctypes and dune
══════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/advanced-c-binding-using-ocaml-ctypes-and-dune/4805>


toots announced
───────────────

  I worked on a socket.h binding last summer and had a great 
  experience
  integrating ocaml-ctypes with dune, I thought that might be of
  interest to other developers so I wrote about it:
  <https://medium.com/@romain.beauxis/advanced-c-binding-using-ocaml-ctypes-and-dune-cc3f4cbab302>


rgrinberg replied
─────────────────

  This is a good article. I encourage anyone who writes C bindings 
  with
  ctypes to study it carefully.

  A little bit of advice to shorten your dune files:

  ┌────
  │ (deps    (:gen ./gen_constants_c.exe))
  └────

  This line isn’t necessary. Dune is smart enough to know that 
  running a
  binary in a rule incurs a dependency on it.

        dune has a truly amazing [support for cross-compiling],
        which we do not cover here, but, unfortunately, its
        primitives for building and executing binaries do not yet
        cover this use case.

  Indeed, we don’t have any primitives for running binaries on the
  target platform. Perhaps we should add some. However, we do in 
  fact
  have some features in dune to solve this concrete cross 
  compilation
  problem. As far as I understand, the goal is to obtain some 
  compile
  time values such as #define constants and field offsets for the 
  target
  platform. This does not in fact require to run anything on the 
  cross
  compilation target. In configurator, we have a primitive
  `C_define.import' to extract this information. The end result is 
  that
  these configurator scripts are completely compatible with cross
  compilation.

  Perhaps this could be generalized to work with ctypes generators 
  as
  well?

  Funny bit of trivia: The hack in configurator required to do 
  this is
  in fact something I extracted from ctypes itself. The original 
  author
  is [whitequark], who in turn wrote it to make ctypes itself 
  amendable
  to cross compilation.


[support for cross-compiling]
<https://dune.readthedocs.io/en/latest/cross-compilation.html>

[whitequark] <https://github.com/whitequark>


emillon then added
──────────────────

        This does not in fact require to run anything on the cross
        compilation target. In configurator, we have a primitive
        `C_define.import' to extract this information. The end
        result is that these configurator scripts are completely
        compatible with cross compilation.

  If anybody wants to know more about this bit, I wrote an article 
  about
  this last year: 
  <https://dune.build/blog/configurator-constants/>


Upcoming breaking change in Base/Core v0.14
═══════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/upcoming-breaking-change-in-base-core-v0-14/4806>


bcc32 announced
───────────────

  We’re changing functions in Base that used to use the 
  polymorphic
  variant type `[ `Fst of 'a | `Snd of 'b ]' to use `('a, 'b) 
  Either.t'
  instead. As well as enabling the use of all of the functions in 
  the
  Either module, this makes the functions consistent with other
  functions that already use `Either.t', (currently just
  `Set.symmetric_diff')

  The following functions’ types will change:
  • `Result.ok_fst'
  • `List.partition_map'
  • `Map.partition_map', `Map.partition_mapi'
  • `Hashtbl.partition_map', `Hashtbl.partition_mapi'

  The type of List.partition3_map will not change:

  ┌────
  │ val partition3_map
  │   :  'a t
  │   -> f:('a -> [ `Fst of 'b | `Snd of 'c | `Trd of 'd ])
  │   -> 'b t * 'c t * 'd t
  └────

  We don’t have a generic ternary variant, and it doesn’t seem 
  worth it
  to mint one just for this purpose.

  Since this change is pretty straightforward, we expect that a 
  simple
  find/replace will be sufficient to update any affected call 
  sites.


CI/CD Pipelines: Monad, Arrow or Dart?
══════════════════════════════════════

  Archive: 
  <https://roscidus.com/blog/blog/2019/11/14/cicd-pipelines/>


Thomas Leonard announced
────────────────────────

  In this post I describe three approaches to building a language 
  for
  writing CI/CD pipelines. My first attempt used a monad, but this
  prevented static analysis of the pipelines. I then tried using 
  an
  arrow, but found the syntax very difficult to use. Finally, I 
  ended up
  using a light-weight alternative to arrows that I will refer to 
  here
  as a dart (I don’t know if this has a name already). This allows 
  for
  static analysis like an arrow, but has a syntax even simpler 
  than a
  monad.

  <https://roscidus.com/blog/blog/2019/11/14/cicd-pipelines/>


Use of functors to approximate F# statically resolved type 
parameters
═════════════════════════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/use-of-functors-to-approximate-f-statically-resolved-type-parameters/4782>


cmxa asked
──────────

  I am learning OCaml coming from F#. In F#, to calculate the 
  average of
  an array whose element type supports addition and division, one 
  can
  write

  ┌────
  │ let inline average (arr: 'a[]) : 'a
  │     when ^a : (static member DivideByInt : ^a * int -> ^a)
  │     and  ^a : (static member (+) : ^a * ^a -> ^a)
  │     and  ^a : (static member Zero : ^a)
  │     =
  │     if Array.length arr = 0 then 
  (LanguagePrimitives.GenericZero) else
  │     LanguagePrimitives.DivideByInt (Array.fold (+) 
  (LanguagePrimitives.GenericZero) arr) (Array.length arr)
  └────

  My understanding is that in OCaml, one would have a module type 
  like
  so:

  ┌────
  │ module type Averagable = sig
  │   type 'a t
  │
  │   val divide_by_int : 'a -> int -> 'a
  │   val plus : 'a -> 'a -> 'a
  │   val zero : 'a
  │ end
  └────

  My question is how the corresponding function would be written:

  ┌────
  │ let average arr =
  │   ???
  └────


smolkaj replied
───────────────

  First, `Averagable' should look like this:

  ┌────
  │ module type Averagable = sig
  │   type t
  │   val divide_by_int : t -> int -> t
  │   val plus : t -> t -> t
  │   val zero : t
  │ end
  └────

  Then average will look something like this:

  ┌────
  │ let average (type t) (module A : Averagable with type t = t) 
  (arr : t array) : t =
  │   Array.fold ~init:A.zero ~f:A.plus arr
  └────

  (The code above uses Jane Street’s Base/Core library.)


ivg then added
──────────────

  While @smolkaj’s answer is a correct and direct implementation 
  of your
  F# code, it might be nicer if your code can interplay with 
  existing
  abstractions in the OCaml infrastructure. For example,

  ┌────
  │ open Base
  │
  │ let average (type a) (module T : Floatable.S with type t = a) 
  xs =
  │   Array.fold ~init:0. ~f:(fun s x -> s +. T.to_float x) xs /.
  │   Float.of_int (Array.length xs)
  └────

  and now it could be used with any existing numeric data in 
  Base/Core

  ┌────
  │ average (module Int) [|1;2;3;4|];;
  │ - : Base.Float.t = 2.5
  └────

  and even adapted to non-numbers,

  ┌────
  │ let average_length = average (module struct
  │     include String
  │     let to_float x = Float.of_int (String.length x)
  │     let of_float _ = assert false
  │   end)
  └────

  The latter example shows that we requested more interface than 
  need, a
  cost that we have to pay for using an existing definition. In 
  cases
  when it matters, you can specify the specific interface, e.g.,

  ┌────
  │ module type Floatable = sig
  │   type t
  │   val to_float : t -> float
  │ end
  │
  │ let average (type a) (module T : Floatable with type t = a) xs 
  =
  │   Array.fold ~init:0. ~f:(fun s x -> s +. T.to_float x) xs /.
  │   Float.of_int (Array.length xs)
  └────

  But we reached the point where using first class modules is 
  totally
  unnecessary. Our interface has only one function, so the 
  following
  definition of average, is much more natural

  ┌────
  │ let average xs ~f =
  │   Array.fold ~init:0. ~f:(fun s x -> s +. f x) xs /.
  │   Float.of_int (Array.length xs)
  └────

  it has type `'a array -> f:('a -> float) -> float' and computes 
  an
  average of `f x_i' for all elements in the array.


Old CWN
═══════

  If you happen to miss a CWN, you can [send me a message] and 
  I'll mail
  it to you, or go take a look at [the archive] or the [RSS feed 
  of the
  archives].

  If you also wish to receive it every week by mail, you may 
  subscribe
  [online].

  [Alan Schmitt]


[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <http://alan.petitepomme.net/cwn/>

[RSS feed of the archives] 
<http://alan.petitepomme.net/cwn/cwn.rss>

[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>

[Alan Schmitt] <http://alan.petitepomme.net/>


[-- Attachment #1.1.2: Type: text/html, Size: 41191 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]