[Caml-list] [ANN] Core Suite 109.27.00 + core_kernel

[Jeremie Dimino <jdimino@janestreet.com>]

I am pleased to announce the 109.27.00 release of the Core suite.

The following packages were upgraded:

- async
- async_core
- async_extra
- async_parallel
- async_unix
- comparelib
- core
- core_bench
- core_extended
- custom_printf
- jenga
- pa_ounit
- zero

Starting with this release the "core" package is split into two
packages: "core_kernel" and "core". core_kernel is a lightweight
subset of core. It contains all the non-Unix parts, it should be
easier to use with js_of_ocaml for example.

We couldn't make it C-free since there is a strong depency to bin_prot
which has C stubs. In addition core_kernel still contains a few C
stubs for bigstrings and also arrays.

Files and documentation for this release are available on our website
and all packages are in opam:

  https://ocaml.janestreet.com/ocaml-core/109.27.00/individual/
  https://ocaml.janestreet.com/ocaml-core/109.27.00/doc/

Here is the changelog since version 109.24.00:

## async_core

- Fixed `Monitor.catch_stream` to prevent missing a synchronous
  exception.

## async_extra

- Added function `Versioned_typed_tcp.Client.shutdown`.
- Added new module `Sequencer_table`, which is a table of
  `Throttle.Sequencer`'s indexed by keys.

## async_unix

- Fixed a performance problem in the scheduler due to repeated calls
  of `Timing_wheel.min_elt`.

  `Timing_wheel.min_elt` is an important part of async, since the
  scheduler calls it once per cycle to know when to timeout for
  `epoll` or `select`.  This causes a problem if `min_elt` is slow and
  called repeatedly, which happens in an application where the next
  clock event is a second out, and yet there are lots of cycles per
  second.

  `Timing_wheel.min_elt` now caches the minimum element, which
  eliminates the problem.
- Fixed async's clock to work on 32-bit machines.

  With the change to `Timing_wheel` in 109.22, async no longer worked
  on 32-bit machines, due to the clock overflowing.  This is because
  it is initialized to `Time.epoch`, and can only handle 6 days.

  The fix now in place is to start the clock at `Time.now ()` rather
  than `Time.epoch`.
- Added many functions to `Async.Sys` so that it looks more like
  `Core.Sys`.
- Changed `Reader.read_one_chunk_at_a_time_until_eof` to not destroy
  the reader buffer.

  Destroying the buffer failed if user code held on to the buffer.

## comparelib

- Changed how `with compare` treats `option`'s so that `None < Some`,
  like `Pervasives.compare`.

  Previously, `with compare` had treated `Some < None`.

## core

- Disabled use of `recvmmssg`, which isn't available on CentOS 5
  machines.
- Defined `Option.compare` using `with compare` so that their
  comparisons are consistent.
- Cleaned up the `Dequeue` module's interface and implementation.

  The interface now matches the conventions used elsewhere in `Core`.
  The new implementation is also cleaner and more efficient.
- Reimplemented the `Stack` module to improve performance, and renamed
  the old implementation as `Linked_stack`.

  The new `Stack` is implemented with this type:

  ```ocaml
  type 'a t `
    { dummy : 'a;
      mutable length : int;
      mutable elts : 'a array;
    }
  ```

  `Linked_stack` is implemented with this type:

  ```ocaml
  type 'a t `
    { mutable length : int;
      mutable elts : 'a list;
    }
  ```

  Using an array rather than a linked list is a more efficient and
  traditional implementation.  Pushing to the stack now does not
  require allocation, except in the rare case when the stack grows.

  One downside is that `Stack.clear` is now O(n) rather than O(1).

  This change also eliminates the `Stack.Empty` exception, so any code
  matching on that exception should fail to compile, and should be
  changed to depend on option-returning `top` and `pop` operations.
- Improved `Lock_file.Nfs`.
  * Allowed an arbitrary message to be stored and retreived.
  * Fixed a case where `create` might throw an exception.
  * Delete both files used for locking when we unlock.
- Split `Core` into `Core_kernel` and `Core`.
- `Core_kernel` is composed of all modules of `Core` that do not
  depend on unix or threads, and `Core` contains the rest and depends
  on `Core_kernel`.

  The goal is to have a very portable standard library that can
  especially run on exotic environment such as Javascript.

  So that code that directly refers to `Core` (rather than `Core.Std`)
  for modules that have moved to `Core_kernel`, we included "proxy"
  modules in `Core` that simply include the corresponding module from
  `Core_kernel`.

- Fixed `Core.Flags` to build on 32-bit machines.

  It had contained a unit test with an integer literal too large to be
  accepted by the compiler when building on a 32-bit machine.

## core_bench

- Added R^2 error estimation.

  Adding this metric should give us a sense of how closely the given
  values fit a line.  Even dots that are fairly scattered can give
  tight confidence intervals.  We would like to have to number to have
  a sense of how much noise we have.

## core_extended

- In module `Sexp`, changed and renamed `load_includes_in_sexp`.

  From:

  ```ocaml
  val load_includes_in_sexp : ?max_depth:int -> Sexp.t -> Sexp.t
  ```

  to:

  ```ocaml
  val load_sexp_with_includes: ?max_depth:int -> ?buf:string -> string -> Sexp.t
  ```
- Added function `Sexp.Diff.to_string`.
- Previously the only option was to print to `Out_channel`.

## custom_printf

- Added missing registration of `custom_printf`'s Camlp4 filter so
  that it works in the toplevel.

## pa_ounit

- Removed comments from test names displayed by `pa_ounit`.

  Before:
  ```
  File "iobuf.ml", line 141, characters 0-34: <<(** WHEN YOU CHANGE
THIS, CHANGE iobuf_fields `...`>> threw ("Iobuf.create got nonpositive
len" 0).
  ```

  After:
  ```
  File "iobuf.ml", line 141, characters 0-34: <<ignore (create ~len:
0)>> threw ("Iobuf.create got nonpositive len" 0).
  ```

## zero

- Added new hashtable module, `Pooled_hashtbl`.

  `Pooled_hashtbl` is a drop-in replacement for `Core_hashtbl`, and
  has the same interface.

  `Pooled_hashtbl` is a linked-chain hashtbl implemented using the
  pooling features of `Zero`.  In addition to pooling, by using
  `Flat_tuple_array` (and therefore `Obj_array`) it has much improved
  performance when working with immediate keys or values, as a test is
  performed and the `caml_modify` skipped when possible.

  The algorithm is a straightforward implementation, and as such will
  not perform well under heavy collisions that come from poor hash
  functions.  Additionally, since the pool of Key/Data nodes are
  created up front, rapidly creating and releasing `Pooled_hashtbl`s
  will not perform well.

  For a well-thought-out hash with a long-lived table and immediate
  values as keys (or data), `Pooled_hashtbl` will very likely
  outperform `Core_hashtbl`.
- Made pointers in `Pool.Obj_array` contain the unique id of the
  object they point to.

  This allows one to check in constant time check whether a pointer is
  valid.
- Improved the performance of `Timing_wheel` by internally exposing
  `Priority_queue.Elt_.t ` private int`.

  This allows the compiler to eliminate some `caml_modify`'s.  This
  made `nanoseconds_per_step` in the timing-wheel benchmark go from
  96ns to 79ns.
- Changed `Timing_wheel` to cache its minimum element.

  This is an important optimization for how async uses `Timing_wheel`.

-- 
Jeremie Dimino, for the Core team