[Caml-list] Bigarray question: Detecting subarrays, overlaps, ...

[Goswin von Brederlow <goswin-v-b@web.de>]

Hi,

I'm implementing a RAID in userspace using ocaml and NBD (Network Block
Device) as protocol to export the device. For this I'm using
Bigarray.Array1 as buffer for data and wrote myself the right read(),
write(), pread() and pwrite() stubs. The advantage of this (over
strings) is that Bigarrays are not copied around by the GC so they don't
need to copy data around between the Bigarray and a temporary buffer in
the C stub.

For efficiency each request stores all its data in a single
Bigarray.Array1. For reasons of the RAID implementation large requests
are split into 4k chunks using Bigarray.Array1.sub and grouped into
stripes. The stripes are then acted upon independently until all stripes
involved in a request are finished. Then the request is completed.

Now I get a problem when 2 requests come in that overlap. Say I get a
write request for blocks 0 - 6 and a read request for blocks 4-9 in
that order. Since I already have the data for block 4-6 in memory from
the write request I do not want to reread them from disk. On the stripe
level the data looks like this:

|W0 W1 W2 W3 W4 W5 W6 .  .  .  | write 0-6
|            W4 W5 W6 R7 R8 R9 | read  4-9

For the read request I need to copy W4-6 to R4-6 or send out W4-6 + R7-9
in two writes. I think I would like to avoid sending each stripe chunk
out seperately. On the other hand I could implement (p)readv() and
(p)writev() stubs.

Anyway, my question now is how to detect which subarrays in the stripes
are part of a common larger array? Do I need to write my own C stub that
looks into the internas of the arrays to see if they share a common
ancestor?  I think that would be preferable to tracking the origin of
each subarray myself.


On a similar note how does Bigarray.Array1.blit handle arrays that are
part of the same larger array and overlap?

----------------------------------------------------------------------
I'm missing functions like:

val super : ('a, 'b, 'c) t -> ('a, 'b, 'c) t -> ('a, 'b, 'c) t
    Create a merged sub-array of 2 adjacent sub-arrays of the same
    big array.

val same_parent : ('a, 'b, 'c) t -> ('a, 'b, 'c) t -> bool
    True if the 2 (sub-)arrays are part of the same big array.

val offset : ('a, 'b, 'c) t -> int
    Offset of the sub-array in the underlying big array.

MfG
        Goswin