Re: Ancient, concurrency, etc.

[Richard Jones <rich@annexia.org>]

On Sat, May 10, 2008 at 11:09:04AM +0200, Berke Durak wrote:
> On Sat, May 10, 2008 at 10:51 AM, Richard Jones <rich@annexia.org> wrote:
> 'Deep copying' of ancient data can be done just like deep copying any
> > other OCaml value.
> 
> As in: it can't be done polymorphically, unless you resort to Marshal, which
> doesn't work on ancient data.

I don't understand -- is there a case where you're using Ancient and
the lack of Marshal has stopped you from doing something?  Or is this
just a really abstract argument about limitations that real users will
never encounter.  I myself have used Ancient extensively and the lack
of Marshal has never been a problem.

> > Anyhow, the polymorphic primitives (like %compare) don't work, just
> > because they make the assumption that anything outside the normal
> > OCaml heap is incomparable
> 
> And that can be quite annoying, so we'd like a way to take values of out
> the ancient heap.

Lack of polymorphic compare is a limitation, but it's well understood
and very simple to work around, and the workarounds don't require
copying anything out of (or into) any heap.

> > , but you can certainly write your own
> > comparison functions to replace those, eg. comparing
> > character-by-character for strings.
> 
> Not. Polymorphic.

But why?  For what purpose?  Any OCaml structure I can think of lets
you define your own comparison or hashing function, so this is no
barrier to putting Ancient objects into Maps or Hashtbls or whatever.

> > This has nothing to do with 'marking'.
> 
> Well, walking over a value graph, either for complete
> hashing/comparison/copying or
> serialization requires marking unless your graph is a tree.

Yes, and marking works just fine with Ancient objects, since their
in-memory representation is *exactly the same* as ordinary heap
objects.  You can mark them just fine by playing with the GC-reserved
bits in the header or however else you normally do it.

> That's very annoying for what I had in mind: communication for concurrency.

Ancient is designed for sharing large mostly read-only data sets
between processes.

It happens that you can mutate simple data too -- eg. if lots of
worker threads are writing their results to a big shared matrix, that
will work fine too.

It also happens that you could use Ancient to pass messages between
processes too.  You'll have to use futexs to provide mutual exclusion
and you'll have to manually free messages after using them, but the
overhead of passing a message will be just one or two copies (from
Caml heap to ancient heap and if necessary from ancient heap to Caml
heap at the remote end).  [The real overhead is hidden in this
analysis since if the processes are running in different sockets then
there will be unavoidable hardware message passing, lots of nasty
cache effects and so on, that you must take account of].

> >   - they don't get garbage collected
> 
> Again, if you want to store an ancient value in a new one, you're toast.

Store an ancient value in a new what?

Anyway, I've had enough of this pointless back and forth.  If you are
using Ancient and have actual sensible questions about its use, please
contact me in the normal way for open source projects.

Rich.

-- 
Richard Jones
Red Hat