Re: [Caml-list] scalable web apps

["Török Edwin" <edwintorok@gmail.com>]

On Mon, 26 Jul 2010 09:59:01 +0200
Jérémie Dimino <jeremie@dimino.org> wrote:

> On Mon, Jul 26, 2010 at 10:29:04AM +0300, Török Edwin wrote:
> > Lwt's Lwt_preemptive seems to allow one to use 'preemptive
> > threads' (which I assume are just usual OCaml threads).
> > So would it be possible to put the file I/O on Lwt_preemptive.detach
> > threads?
> 
> Yes.
> 
> > Has anyone tried to measure the performance of doing that?
> 
> Yes, i tried it some time ago with two small C programs which were
> just reading a file on the disk. One doing read operations in a
> simple loop, and one launching a thread for each read operation (or
> reusing the same thread). The threaded version was about 100 times
> slower than the non-threaded one.
> 
> BTW it is planed to add some kind of asynchronous file I/O support in
> Lwt by using mmap and mincore.

How about using linux's AIO and eventfd?

Manpage of eventfd says:
"When used in the kernel, an eventfd file descriptor can provide a
kernel-userspace bridge  allowing,  for  example, functionalities like
KAIO (kernel AIO) to signal to a file descriptor that some operation is
complete.

A  key point about an eventfd file descriptor is that it can be
monitored just like any other file descriptor using select(2),
poll(2), or epoll(7).  This means that an application can
simultaneously monitor the readiness of  "tra‐ ditional"  files
and  the  readiness  of other kernel mechanisms that support the
eventfd interface."

And looks like libaio-ocaml has support for eventfd in AIO:
val fd : context -> Unix.file_descr
  (** return eventfd associated with the context *)

I think there could be a Lwt_unix.read_aio that is just
like Lwt_unix.read, except it checks the eventfd descriptor for being
readable instead of the original fd, and use Aio.read instead of usual
read, etc.

Best regards,
--Edwin