Efficient I/O with threads

[Yaron Minsky <yminsky@gmail.com>]

We've been running into some interesting problems building highly
efficient I/O routines in threaded code in ocaml, and I'm curious if
anyone else has some thoughts on this.  The basic problem seems to be
that the locking and unlocking of the IO channels seems to take a
large fraction of the execution time.

A little bit of background first.  The data type we're outputting is
basically a simple s-expression, with the following type:

type sexp = Atom of string | List of sexp list

We write out an s-expression by writing a tag-byte to determine
whether the s-expression is an atom or a string.  If the s-expression
is an atom, we then write a 4-byte int, which is the length of the
string, and then the string.  If the s-expression is a list, we write
an atom which is the number of s-expression that are contained, and
then write those s-expressions.

It's very easy to write parsing and marshalling for this type of wire
protocol, but that code turns out to be quite inefficient, because you
end up making too many calls to the input and output functions, and
each one of those calls requires releasing and acquiring locks.  I
just can't think of a clean way of implementing a reader for this kind
of protocol.  (a writer could be done by writing stuff to a buffer
first, and then writing the whole buffer out at the socket at once.)

Any thoughts?
Yaron