Re: [Caml-list] Shared data space: How to manage Ancient keys?

[Hugo Ferreira <hmf@inescporto.pt>]

Goswin von Brederlow wrote:
> Hugo Ferreira <hmf@inescporto.pt> writes:
>
snip...
> 
> How about a simple solution. Only one thread handles indexes.
> 
> You said you have worker processes. Add one controling process on top of
> it that spawns all the worker processes and keeps a communication line
> open with them, e.g. a pipe. Each worker would then tell the controller
> to remove an index or request an unused index to store a result. Its
> probably a good idea to request unused indexes in bunches, say 100 at a
> time to cut down on latenzy.
>

To be honest I have already considered (but not discarded) this option
(which may provide other advantages). This however has several issues:

1) I still need to deal with problem of managing indexes (identifying
    and reusing slots).
2) Each worker has to scan all sequences that have been generated but
    not consumed. So I will also need to generate and send a list of
    available indexes. This may be a very long list.
3) Interaction between the controller and workers need to exchange
    several types of messages:
    a) Request for a sequence to process
    b) Request for a list of sequences to scan
    c) Request an index to use
    d) Acknowledge a used index (maybe)
    e) Request an index to free
    f) Acknowledge a freed index (maybe)
    This implies a complicated protocol with state (worker aware).

Advantages:
1) The workers scan the sequences asynchronously. Scans may have
    various processes access data in read only. Insertion/deletion
    requires locking for read/writes. Use of a controller will
    facilitate this.
2) The controller may avoid having the same sequences scanned
    by two or more workers.

Needless to say these problems are more than I bargained for. 8-(

Thanks,
Hugo.

> MfG
>         Goswin
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>