Re: [Caml-list] ANNOUNCE : libsndfile-ocaml alpha

["Markus Mottl" <markus.mottl@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2104 bytes --]

On 1/2/07, Gerd Stolpmann <info@gerd-stolpmann.de> wrote:
>
> Just a quick and somehow interesting observation: Linus assumes that the
> read buffer is page-aligned in this comparison, something you cannot
> expect in most higher languages (of course you can arrange that in
> libsndfile - did you?).


Right, this is an unwarranted assumption.  Furthermore, for certain
applications with complex access patterns I'd expect it to be way easier for
the user to use mmap efficiently.  Even assuming that "read" were always
faster, implementing your own complex buffer management to cache data
efficiently for particular access patterns may not be easy.  Unless your
name is Linus, of course... ;-)

That reminds me that the OS guys live in another
> world. Would be interesting which function wins if you compare Unix.read
> (which does not care about alignment, and does an extra copy) with
> Bigarray.mmap.
>

I use mmap in a fileserver for performance reasons.  Compared to using
I/O-channels it requires, if I remember correctly, only about 50% of the
CPU-time, but OCaml-channels have to do an extra copy from the channel
buffer to the user buffer so this is not quite a fair comparison.  I'd still
expect mmap to reduce overall CPU-time over "read" for larger files due to
less data copying between kernel buffers and user space.

I have observed that mmap does not pay for small files (my cutoff point is
8192 bytes), possibly due to some setup overhead for memory mappings.  Total
time is not strongly effected, because we are generally I/O-bound, but if
you run the fileserver on the same machine as applications, the lower
CPU-time is a noticable advantage.

There is one caveat though regarding mmap with OCaml on 32bit platforms: the
GC has a bug which prevents it from reclaiming bigarrays aggressively
enough.  Though there is plenty of RAM (the kernel need not keep mapped
files in memory), the process might run out of address space.  I hope this
bug (0004108) will be fixed in the next release.

Regards,
Markus

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

[-- Attachment #2: Type: text/html, Size: 2814 bytes --]