mixed customizable external data types in Caml

mixed customizable external data types in Caml

[Basile STARYNKEVITCH]

[[English]]

When using caml as an embedding langage (concretly, caml used as a
scripting langage to existing numerical applications coded in Fortran
or C) it would be very nice to be able to have a Caml value containing 
both Caml values (ie pointers or tagged integers) and other stuff,
such (as a Fortran matrix or whatever).

Of course it is already possible to use an arbitrary pointer to
whatever C or Fortran data.

But it would be nice to be able to have all of Caml memory management
stuff (in particular garbage collection and marshalling) in mixed data 
types. I would wish that a special Mixed_tag exists, and that mixed
data would have the following C structure 

struct mixed_data_st {
   unsigned  caml_header;
   struct mixed_descriptor_st *descr;
   unsigned data[0] /* the actual data */;
} 

where the descriptor is a structure like 

struct mixed_descriptor_st {
   unsigned descriptor_magic; /* a constant magic */
   char *type_name;
/* finalizing routine */
   void (*finalizer_func) (struct mixed_data_st*data); 
/* scanning references routine */
   void (*refscan_func) (struct mixed_data_st *data, 
                         void(*scan_func)(value*,void*), 
                         void *clientdata);
/* moving routine */
   void (*mover_func) (struct mixed_data_st *dest, struct mixed_data_st *src);
/* marshalling routine */
   void (*marshall_func) (struct mixed_data_st* dest, char* data, int datalen);
/* unmarshalling routine - return the needed byte length */
   int (*unmarshall_func) (struct mixed_data_st* dest, char* data, int datalen);
/* etc ... */
};

So the descriptor contains routine to finalisz, scan the reference
fields (for the GC), move or copy (for the copying GC), marshall &
unmarshall; etc...

I know that from a Caml-centric point of view external data containing 
Caml references is a useless nuisance. But I think that it would
promote Caml usage in existing (legacy) applications. From my reading
of the runtime system code, I think that adding it would be rather
easy for the runtime system coder (essentially, it only deals with
hooks).


[[ Résumé français ]]

Il me semble que pour faciliter l'utilisation de Caml comme interprète 
embarqué dans des applications existantes, il serait souhaitable de
disposer de type (au runtime) mélangeant référence Caml et valeurs
autres. Une façon possible de ce faire serait d'avoir une étiquette
Mixed_tag avec des données contenant un descripteur. Celui-ci serait
une structure C contenant les routines appropriées pour le
ramasse-miettes, la finalisation, le codage/décodage, etc...


N.B. Any opinions expressed here are solely mine, and not of my organization.
N.B. Les opinions exprimees ici me sont personnelles et n engagent pas le CEA.


----------------------------------------------------------------------
Basile STARYNKEVITCH   ----  Commissariat à l Energie Atomique 
DRN/DMT/SERMA * CEA/Saclay bat.470 * 91191 GIF/YVETTE CEDEX * France
fax: (33) 01,69.08.85.68; phone: 01,69.08.40.66; home: 01,46.65.45.53
email: Basile . Starynkevitch @ cea . fr  (but remove white space)
I speak french, english, russian. Je parle français, anglais, russe.
----------------------------------------------------------------------

Re: mixed customizable external data types in Caml

[Damien Doligez]

>From: Basile STARYNKEVITCH <Basile.Starynkevitch@cea.fr>
>
>When using caml as an embedding langage (concretly, caml used as a
>scripting langage to existing numerical applications coded in Fortran
>or C) it would be very nice to be able to have a Caml value containing 
>both Caml values (ie pointers or tagged integers) and other stuff,
>such (as a Fortran matrix or whatever).
>
>Of course it is already possible to use an arbitrary pointer to
>whatever C or Fortran data.
>
>But it would be nice to be able to have all of Caml memory management
>stuff (in particular garbage collection and marshalling) in mixed data 
>types.

You can have garbage collection easily: put your foreign data in one
object, and your caml values in another object, along with a pointer
to the first object.

I'd say the real interest would be in marshalling, equality, and
ordering.  And I see a big problem with unmarshaling.  How does the
system find the unmarshaling function ?  There is no pointer to the
descriptor in the object, because the object does not exist yet.

-- Damien